METAL GEAR SOLID 4 : Guns of the Patriots

By Yoshiyuki Watanabe

June 12, 2008 saw the simultaneous worldwide release of Metal Gear Solid 4: Guns of the Patriots (MGS4) for the PlayStation 3. Gamers all over the world were waiting eagerly for this ultimate stealth mission game. No doubt they have already been captivated by its powerful gameplay.

MGS4 was a huge project that took Hideo Kojima, the virtuoso director, about three and a half years to complete. The Metal Gear series has been around for 21 years, with many people regarding it as a masterpiece in terms of its scenarios, gameplay and realism. Making a particular impact is its world-class graphics that go beyond the boundaries of a conventional game. These graphics give the action such realism and immediacy that players feel like they have been transported to a real battlefield.

The Metal Gear Solid series was created by Kojima Productions, a development unit at Konami Digital Entertainment. Kojima Productions has also given us popular series such as Zone of the Enders and Boktai: The Sun is in Your Hand. The team has three directors: Hideo Kojima, Yoji Shinkawa and Shuyo Murata. There is a program manager, design manager, script manager and sound manager, as well as many other artists and programmers.

In the development of MGS4, about 100 staff members were involved in the 3D content production. Extra personnel were added during busy periods, and by the final phase of development over 180 people were working on the project. By the end there were as many 40 programmers working on MGS4, including the work required for online compatibility. From the fact that such a large number of people were involved and it took three and a half years to finish, we can see that MGS4 truly was a mega-project.

The main tool used in this game with its world-class video graphics was SOFTIMAGE|XSI. We interviewed the Kojima Productions team after they had finished development and as the release date for MGS4 was approaching. In this interview, we were lucky to be able to speak with staff from all the major development units, including characters, background, machines, 2D, event demos, motions and programming. We learned a lot of interesting information, so this article has been split into two parts.

From a Single Storyboard...

MGS4 video was first previewed to the public in a trailer shown at the 2005 Tokyo Game Show. The main character, Snake, was shown looking around a wall in a ruined building. From this single storyboard, the whole project started. After four months, the team finished a spectacular trailer that was over 10 minutes long and made a huge impression around the world.

This high quality video that made such a strong impact fully utilized the real-time processing power of the PS3. To transfer this to the actual game without losing quality, it took a lot of effort for the team to import the data and create a proper balance. In the initial stages of the project, they were still getting used to the performance of the PS3, and there were no precedents to any of the work that they were doing. The data size required for the content was much bigger than anything that had come before.

Art Direction in Metal Gear Solid

Like the previous titles in the series, Yoji Shinkawa was in charge of art direction for all the character and machine designs in MGS4. In this game, the most important consideration was whether the design would pull the player into the game. It didn't matter whether it was expressed through an exaggerated effect or through a conventional illustration.

Mr. Shinkawa starts his design process by using a brush to make a drawing. The resulting design is a black and white image drawn in Mr. Shinkawa's distinctive brush style. Then, one of a number of different methods is selected to embody the black and white image. The original picture may be touched up, it may be transferred to a 3D figure, or if the design is complicated, it may be modeled out of clay.

In this way, the team developed a more concrete design from the original image, which they then used as a reference when modeling with XSI. For machines modeled from clay and some of the characters, they used 3D data obtained from calibration (image-based modeling). The team imported the data into XSI for editing.

It is also possible to use 3D scanning to create 3D data. But because the scanned data requires a lot of modeling correction work, the team decided not to use this 3D scanning method. They said that using the calibration method results in less correction work and that it was relatively easy to import accurate image data into XSI. The calibration workflow is described in more detail using images in the second half of this article.

In the design process, Mr. Shinkawa also selected all the color designs. He said that even when he was drawing the original pictures in black ink, he could see in his head the types of colors that he would use for the characters. He made his final selection after all the characters were ready, choosing colors that would achieve a good overall balance. He reproduced textures such as material colors, metallic expressions and complex patterns using the real-time shader. The textures that Mr. Shinkawa wanted were created one after another by programmers using the real-time shader. Sometimes he would show a prototype that he created in advance with a mental ray shader to share his ideas on texture with the programmers.

Yoji Shinkawa's design workflow for creating a 3D image from an original drawing

Many characters from previous titles in the series reappear in MGS4, which makes the fans very happy. But there were some characters that were being depicted in 3D for the first time, such as Naomi, who appeared in MGS1. The team worked hard to create attractive 3D characters that would not disappoint the hardcore fans of the series.

Mr. Shinkawa said, "When we first started to develop the Metal Gear Solid series, we just wanted to make a war game that we ourselves would really want to play. The fact that the series grew into one that is played by so many people is thanks to our loyal users. We are confident that MGS4 will live up to the expectations of these fans. But we are never satisfied with the way things are. We always want to work harder to push our creative abilities to the limit and create even better games in the future."

Character Production Workflow

Most of the characters that are animated on the console, including the main character, Snake, have been restricted to a data size (including the face model) of about 5,000 to 10,000 polygons. Further, characters are used that have the same polygon resolution in both the game action and the event demos. This means that the game screens and video clips are seamlessly connected, making it easier for players to become emotionally involved.

As mentioned above, with the exception of crowds, characters are used that have the same polygon resolution in both the game action and the event demos. Separate from the resolution model used on the console, high resolution data are also simultaneously modeled for generating the normal map. Details such as creases on clothes are then expressed with the normal map that was generated from the high-res model.

In terms of bones used for constructing the bodies of characters, about 21 joint bones were used that contained animation data and were activated through these data. But many auxiliary bones were also used to supplement movements such as the twisting of knees, elbows, legs and arms. These were not activated by animation data. Rather, they were linked to the values of the basic joints that were activated by animation.

The team used these specifications not only on XSI, but also on the console. They could perform the same control on the console simply by outputting an auxiliary bone definition file from XSI.
Because the auxiliary bones themselves do not contain motion data, the data size can be kept to a low level. Further, if auxiliary bones need to be added or deleted, the operation can be performed simply by changing the model data without having to reconvert the motion data.

Gator: An Essential Function

To ensure top quality in game development, repeated specification changes cannot be avoided. In response to this issue, the team frequently used the Gator function in their character production workflow. Often, they wanted to extract just one attribute from a specific model, or they wanted to reuse an attribute from a past model. For example, due to unavoidable circumstances, sometimes complex UV information must be rejected after creation. Gator is perfect when you want to reuse such painstakingly-created UV information. Although the topology and number of polygons will be different, the data can be reused on models with a similar shape.

To give another example, creating UV information for high-res models involves a lot of work. Using Gator reduces this work by a significant amount. You can even transfer UV information from a low polygon model with a completely different polygon count to a high-res model. The UV information is interpolated before being transferred. In MGS4, the main character, Snake, has about 10 different clothes patterns, including camouflage and costumes. Instead of performing envelope settings for these clothes on a case-by-case basis, the team could instantaneously transfer the character's existing envelope information just by clicking a button in Gator.

As an aside, the team mentioned that just one month before they were due to finish the project, they started work on Assassin's Solid, a collaboration between Ubisoft's Assassin's Creed and MGS4. Obviously, it was going to be difficult in terms of scheduling to complete such a novel idea in the final stages of the MGS4 project. But coincidentally, both Altair and Snake, the main characters in the two games, have bodies that are very similar in terms of limb length and body size.

Because of this, they could use Gator to assign Snake's rig structure to Altair's model. Their effective use of Gator enabled them to place Altair in the runtime battle scenes in MGS4 even though they had such limited time. This collaboration broke down barriers between game companies and made a big impression on the public.
It also brought great publicity benefits.

Operation Video of Efficient Rig Setup Using Gator

This video shows that with XSI's Gator, it was easy to reuse Snake's existing envelope for Altair. The Altair character is the model data received as-is from Ubisoft: modeling correction was not performed. This is a vivid operation video that shows how the setup was performed in an efficient and rational way.

>> View Flash Movie

Amazing Facial Animation

One of the main features of MGS4 is its world-class facial animation. How did the team create such realistic facial expressions?

Because lip synch work is performed in the localization stage, and to lessen work loads, audio analysis animation is used in the Metal Gear Solid series. For example, in MGS4, lip synching in the English & Japanese versions was performed using different types of audio analysis software. Emotions & facial expressions other than lip synching were added through manual animation. In most cases, facial expressions and phoneme elements do not interfere with each other, so both can be worked on in parallel.

It was this that allowed the simultaneous worldwide release of the title. When performing audio analysis, the facial expression components (such as anger or laughter) and the phoneme components in each language must be divided into separate parameters. These must then be reproduced as rig behavior. Although it is possible to create parameters for bone rotation and movement, the team said that this would make the rig too complicated and make it too difficult to predict bone changes as an envelope. In other words, there would have been two problems had they tried to perform facial animation using bone control only: it would have been difficult for the designer to perform intuitive operations, and difficult to create facial expression and phoneme parameters for bone behavior.

On the other hand, while shape animation has the disadvantage of producing linear interpolation animation, it is very easy to create parameters for phonemes and facial
expressions. But perhaps its most important benefit is that the designer can intuitively predict the results.

For these reasons, the team based the rigs in this project on parameters created with shapes, outputting the results as bone animation.

This setup allowed animation automation using audio analysis (automated animation was performed not just for the mouth, but also the tongue and throat) to coexist with a rich range of character emotions that were added manually. In the Flash movie below, you can see how smooth muscle movement is reproduced using a top-quality rig setup.

>> View Movie
Flow for Facial Rig Construction
1. Low polygon model activated with shape animation
2. Bones fixed on top
3. Enveloped polygon meshes for these bones
4. Tangent color
5. OpenGL display (wrinkles also expressed with a normal map)

The team created rich and varied facial expressions by using a large number of different parameter combinations.

Expressions, phonemes, eyes (and eyebrows) and shader wrinkle animation can be selected using the tabs.

Surprisingly, the team even developed a tool for the automatic creation of setups for facial rigs that were capable of this kind of advanced control. In their system, they prepared face model data and executed the tool to automatically identify the optimum bone positions. The tool also created control that included preset parameters for expressions such as laughing or angry faces. To perform automatic facial rigging, it was first necessary to make sure that the topology information for facial data was uniform. Setup was fully automated just by observing this rule. Then, all that was required to construct an environment in which facial animation could be performed was fine-tuning of the control by the designer.

They also used a tool to automatically generate the rig for controlling eyeball movement and the muscles around the eyes. Because the area around the eyes is also controlled using both shapes and bones, when the eyeball locator is moved, the muscles move smoothly just like they do for the mouth. Further, even if the shape is edited to redefine the eye edges, it does not spoil the blinking or brow furrow expressions at all.

It took these setup techniques and efficiency quality improvements to create the memorable game characters that appeal so much to the emotions of players.

High-Quality Normal Maps Generated with Ultimapper

Textures such as defuse maps, normal maps and incidence color maps were used for the face models. With incidence, shading is determined not by light and normal lines, but by cameras and normal lines. By making the outline part brighter, they could depict a softer texture.
Surprisingly, the entire texture size for characters was kept to a resolution of 512×512, to restrict the amount of VRAM that is used. This is a relatively low resolution for a next-generation console. In the initial stages, since they were designing for the next-generation PS3, they used larger data.

But after a while they reassessed whether they actually needed the volume of information that they were using. The result was that they decided to use their wide-ranging expertise and techniques to produce high quality at a lower resolution. As one example, for limb and body material, they used two symmetric materials for the left and right that could be reversed in a U direction. Although the portrayal suffers to a very small degree by dividing the material into two, the resulting memory that is freed up is a major advantage.

The 512 size was not exceptional for normal maps either. The team imported into XSI a face model to which details of about 500,000 polygons had been added with ZBrush. From this, they generated a normal map with Ultimapper. Hideki Sasaki said that to obtain the maximum effect at a low resolution, it was essential to raise the quality of high-res models. He said that if work is performed carefully using XSI's modeling function and subdivision surface, it should be possible to achieve sufficient quality even at low resolution.

To ensure that the normal map data generated with Ultimapper did not lose information through compression when output as game data, they were extremely careful when outputting the data from their own normal map editing software.

So how did they express the wrinkles on Snake's face? They used a technique that blended two normal maps; one with wrinkles and one without. The team also used this animation with blended normal map values in other facial animation work, as will be described later in this article.

It is surprising how many techniques the team used to delete data so that the processing load would not destroy the balance of the game. Snake has visibly aged in this game as compared with previous titles: the work described above shows how much effort was put into creating this new look.

XSI's Cluster Constraint Function: Turning Ideas into Reality

When we asked Mr. Sasaki which functions benefitted the project, he gave an unexpected response. "There were many, but I found cluster constraint extremely useful for animation." Even in the facial rig setup, he used cluster constraint to fix each point cluster of the shape target to the bones. Also, he said that his team used cluster constraint in the following situations.

To minimize processing in MGS4, they used a specification where the tangent color on the console only changes when bones are rotated. If they were fixed just to coordinates, the animation behavior would be correct but the tangent color would not change. In other words, the shading would not change from the default condition. In response to this issue, the team used cluster constraint to fix normal maps and tangents. By doing this, they could input an accurate rotation value to the bone and reproduce correct shading.

Further, shaking items attached to the characters' clothes were mostly expressed in both the game action and the event demos with simulations on the console. However, there were also some event demo scenes with extreme action where simulation control was difficult. In these cases, they used animation that was simulated with XSI's Syflex. The flow for the main operations in this process was as follows.

1. Run a simulation with Syflex to depict the shaking of the clothes.
2. Convert the cached simulation results as a shape target.
3. Use cluster constraint control to fix the bone to the point group of the object that     was controlled with the shape.
4. Perform bone envelope control for the actual model to be used on the console.     (The concept is the same as for the facial rig.)

The advantage of this control is that if there were slight deformations or defects in the simulation results, they could add an editing stage using "Secondary Shape Mode" to step 2. They could edit the shape form with apex movements, or make images smoother with smoothing or push operations. Naturally, the results of these intuitive corrections were then reflected in the envelope control console data.

Mr. Sasaki explained, "Cluster constraint can be set for all kinds of components, such as apexes, polygons and edges. The fact that the fixing function is available as standard even for normal maps and tangents is typical of the rich features of XSI. If it wasn't for the excellent functionality of cluster constraint, we could not have achieved our facial rig setup ideas or our technique for transferring cross-simulation to bones."