Stunning real-time rendering effects in Unreal Engine 5.
Lumen in the Land of Nanite, a live demo running on PlayStation 5.
On May 13, 2020, Epic Game unveiledUnreal Engine 5The mysterious veil released by the company shocked the global gaming industry with the demonstration video. The powerful performance of Unreal Engine 5 has completely conquered academic colleagues and players in the field of computer graphics. This technological leap makes the real-time detailed rendering in the game closer to movie CG effects and the real physical world.
Unreal Engine 5 has two new core technologies:Nanitevirtual micropolygon geometry technology andLumenDynamic global illumination technology. The emergence of Nanite virtual geometry technology means that film and television-level artworks composed of hundreds of millions of polygons can be directly imported into Unreal Engine. Nanite geometry can be streamed and scaled in real time, so there is no need to consider polygon number budgets, polygon memory budgets or draw times budgets; there is no need to bake details into normal maps or manually edit levels of detail (LOD). This must be a revolutionary leap in the field of graphics. As shown in Figure 1, each samurai statue in the demo has 33 million polygons, and its geometric details are exquisitely crafted. Its realism is breathtaking!
Figure 1. Each statue has 33 million polygons.
traditional method
In traditional computer graphics techniques, three-dimensional geometric shapes are represented by so-called triangular meshes. We densely sample the surface, then triangulate the sampling points, record the combined structural information of the triangular mesh, and record the three-dimensional position information of all vertices. We use this to express the surface shape. Compared with images, the combined structure of triangular meshes is usually irregular, and we cannot predict the area where the vertices are located through the sequence number of each vertex. This irregularity makes the triangular grid unable to be directly expressed and randomly accessed by hardware; in geometry processing software, the data structure of the triangular grid requires a large number of pointers and complex and sophisticated algorithms to process.
Figure 2. Surface shape is represented by a triangular mesh data structure.
The details of the surface geometry are expressed by triangle patches. Given the same original surface, rough geometry requires a lower sampling rate and fewer triangle patches (low-mode), and precise geometric representation requires a higher sampling rate and more triangle patches (high-mode). Low model increases the rendering speed, but reduces the picture quality; high-resolution triangular mesh reduces the frame rate of the rendered picture, but improves the picture rendering quality. In the game, in order to achieve the speed of real-time interaction, we can dynamically select triangular grids with different degrees of precision based on the distance of the character from the camera and the direction of the perspective. Low-mode and high-mode can be represented by a unified data format and the resolution can be changed dynamically. This geometric representation is called LOD (Level Of Detail) representation, invented by Hughes Hoppe, and is currently commonly used in game engines. Figure 3 shows different levels of geometric representations of the same old man’s face.
Figure 3. Level of detail (LOD) representation of the surface.
The shape of the surface is not enough to express all the information of the geometry. We need the color information of the surface, material information (such as metal texture), roughness, optical properties and other information. This information is often stored in so-called texture images. Each pixel corresponds to a point on the surface, and the pixel value represents the color, material, optical properties and other information at the corresponding point. These images are called texture images, and the mapping between the texture image and the surface is called the parameterization of the surface. The process of applying a texture image to a surface is called texture mapping. Figure 3 shows an example of a texture map. The first row is a "white model" with only geometry and no texture; the middle row is two different texture images; the lower row is the rendering effect of different texture maps.
Figure 3. Surface texture map.
Texture images can also be used to express other information, such as normal maps. As shown in Figure 4, we parameterize the monster model to a flat disk, and then store the normal vector of each point on the surface in the corresponding texture pixel. We use red, green and blue colors to represent the coordinates of the normal vector, thus obtaining the normal vector texture map. We use the high polygon to generate the normal vector texture map, and then paste it on the low polygon. The quality of rendering depends on the precision of the normal vector, while the rendering speed depends on the number of triangle patches. On the one hand, we want to improve the precision of the normal vector, and on the other hand, we want to reduce the number of triangular patches. Such a low-modulus and high-modulus generated normal vector texture map is an ingenious solution.
Figure 4. Normal vector map obtained from optimal transmission mapping.
To attach the normal vector texture generated by the high-module to the low-module seamlessly, it is necessary to establish a mapping between the high-module and the low-module. As shown in Figure 5, we align the high-module and the low-module as much as possible in the three-dimensional space, and then project the low-module to the high-module. Each low-module vertex finds the closest point on the high-modulus surface to establish a mapping. This process is called "baking".
Figure 4. Baking process.
We see that the traditional methods of geometric compression, establishment of LOD representation, normal mapping, and baking are all aimed at improving rendering quality and rendering speed. These traditional techniques rely on differential geometry principles and require complex algorithms to implement. Nanite's virtual micro-polygon geometry technology makes these technologies obsolete. It is groundbreaking and unparalleled!
Nanite geometric image method
So, what is so brilliant about Nanite’s virtual micro-polygon geometry technology? Why can technology that people have accumulated over decades be abandoned overnight? Many experts in the field of computer graphics have analyzed that Nanite's geometric technology is likely to be based on the SIGGRAPH paper "Geometric Images" [1] that Lao Gu collaborated with Professor Gortler and Hughes Hoppe 18 years ago.
As discussed above, we see that there are two main data structures in graphics: irregular triangular grids that represent geometry and regular images that represent textures. In order to process these two very different data, two pipelines are set up in the graphics processing unit (GPU) hardware, one is used to process the triangle mesh, and the other is used to produce the texture image. Lao Gu's idea at the time was to unify the two data structures into a texture image, thus abandoning the triangle grid pipeline and letting the texture image pipeline process geometry and texture data, thus simplifying the GPU hardware design and improving rendering speed.
Figure 5. Surface conformal parameterization.
Figure 6. Geometric image (left frame) and normal texture image (right frame) of the head of King David.
Figures 5 and 6 show the computational process of converting triangular meshes into geometric images. First, we parameterize the triangular mesh and map it to a planar rectangular area to minimize distortion; then, we resample on the plane, record the geometric position and normal vector information at the corresponding pixels of the image, and encode them in red, green, and blue. From Figure 6, we can see that the geometric image is much smoother than the normal texture.
Figure 7. Conversion of geometric images to 3D surfaces (MIPMAP).
The geometric image contains all the geometric information of the three-dimensional surface and can naturally be rendered into a curved surface. Of course, in practical applications, we can also express the displacement of the high-polygon surface relative to the low-polygon surface as a geometric image to achieve a relief effect. In summary, we use image data structures to unify geometry and texture, and each surface model is represented as several huge texture images. For example, the surface in Nanite technology includes several textures such as geometric images, color textures, metal material textures, and optical properties. This regular representation method has many advantages. For example, the resolution rate can be dynamically determined during the rendering process (mipmap pyramid), dynamic subdivision and encryption (subdivision) can be dynamically generated, and textures can be dynamically generated, such as storing intermediate calculation results of global illumination, etc. In particular, the format of geometric images makes the memory access pattern more regular and easier to predict, thereby improving memory access efficiency and greatly improving the speed of data scheduling.
So, why didn’t GPU adopt the technical route of geometric images 18 years ago? Lao Gu believes that an important reason is that surface parameterization technology had not yet appeared at that time, and the conversion from triangular meshes to geometric images was very difficult. Now, as the theory and algorithms for computing conformal geometry are mature, and computing optimal transmission mapping is also close to maturity, the problem of surface parameterization is no longer a bottleneck.
Outlook
From 2002 to the present, the article "Geometric Images" has been widely circulated in the field of graphics. Experts and scholars have conducted a large number of subsequent studies, with thousands of citations.无数的学者和工程师们耗费多年的心血，全面改进了技术，推动了这一领域的发展，日积月累，终于一鸣惊人。如果几何图像被工业界广泛接受，那么全局参数化、基于几何图像的渲染、基于几何图像的硬件设计会吸引人们更多的注意。 Accordingly, the theoretical foundation of differential geometry will become one of the required courses in the field of graphics. In the future, we look forward to more profound applications of basic mathematics and more revolutionary results in the field of graphics!
【1】 X. Gu, S. Gortler and H. Hoppe, Geometry Images, SIGGRAPH 2002, http://hhoppe.com/proj/gim/
Machine-translated from the Chinese original 虚幻引擎5技术解析：几何图像的思想 (published 2020-05-17). WeChat original ↗