Figure 1. Blowhole at Kiama.
The greatest luck in life is to meet a knowledgeable and far-sighted mentor when you are ignorant. With the help of the mentor, you can shape your value system, improve your knowledge structure, improve your academic taste, and especially determine long-term research directions. Lao Gu is such a lucky person: twenty-five years ago, Lao Gu had just started his graduate study career. When he met Mr. Qiu Chengtong for the first time, Mr. Qiu suggested that Lao Gu study the Monge-Ampère equation and handed his research manuscript to Lao Gu. At that time, Lao Gu's knowledge level was limited and he was in a rebellious period of youth. He could not subjectively see that such a complex mathematical theory would be applied to the nascent computer science. Therefore, throughout his doctoral period, Lao Gu did not include the Monge-Ampere equation theory within the scope of his research.
Twenty-five years later, Lao Gu's views completely changed, and he increasingly admired Mr. Qiu's foresight. The natural laws of physics and geometry are expressed in the language of partial differential equations, and the simulation of natural phenomena is equivalent to solving partial differential equations. The development of computer science must apply the theory of partial differential equations, from simple to complex, from linear to nonlinear. This is a historical necessity. Looking at the entire computer field, almost all geometric algorithms a few years ago were based on the linear Laplace equation. For example, the spectrum of the surface Laplace-Beltrami operator is used as a "shape fingerprint" and is widely used in almost all fields of engineering. These algorithms are essentially based on the spectral theory of Riemannian manifolds in the early years of Mr. Qiu and Mr. Zheng Shaoyuan, as well as the "thermal core" theory that extends Riemannian manifolds. The "level set" method directly applies the differential equation method, especially the fluid equation, to the field of image processing. In recent years, with the rapid development of medical imaging technology, wireless communication technology, 3D printing technology, VR/AR technology, and especially deep learning technology, the theory of nonlinear partial differential equations represented by the Monge-Ampere equation has begun to be widely used in computer science and other engineering fields.
The research of Lao Gu's team is also based on this historical development: the early developed holomorphic differential algorithm is used to calculate the differential form on the Riemann surface, which is based on the linear geometric Poisson partial differential equation; the later developed Ricci flow algorithm is used to construct the Riemannian metric from curvature, which is a nonlinear flow method; the recently developed convex geometry algorithm for the Monge-Ampère equation is used to calculate the transformation between probability distributions and construct isometric embeddings from curvature, which is based on the strongly nonlinear Alexandrov method. The programming difficulty and algorithm control difficulty of these algorithms also increase in sequence. The valuable reward is the rare and accurate results.
Monge-Ampere equation
The Monge-Ampere PDE is a completely nonlinear, degenerate elliptic partial differential equation. Its general form is
Even in the field of basic mathematics, due to its strong nonlinearity and high degree of technicality, theoretical research on the Monge-Ampere equation has always been in a low-key position. The Monge-Ampere equations are closely related to geometric problems, which means that most geometric problems in nature are nonlinear in nature. For example, the Minkowski problem, given the Gaussian curvature of a convex surface defined on the normal vector, how to reconstruct the surface; the Weyl problem, how to realize the local isometric embedding of the Riemannian metric surface; the affine sphere and affine maximum surface problems in affine geometry. In geometric optics, the Monge-Ampere equation is also the key to solving the design problems of reflective surfaces and refractive lenses. In economic theory and probability statistics, the basis of optimal transportation (Optimal Mass Transportation) theory is also the Monge-Ampere equation.
In recent years, with the explosive development of deep learning technology, optimal transmission theory has become a hot topic in the field of computer science, and the Monge-Ampere equation theory has begun to substantially penetrate into the center of computer science. Therefore, we feel that it is most appropriate to understand the Monge-Ampere equation from the perspective of optimal transmission. The intuitive understanding of the optimal transmission problem is as follows: when examining a country's territory, its grain productivity and consumption rate, the cost of transporting one ton of grain from the place of production to the place of sale is. The government hopes to design a transmission plan to transport grain from producers to consumers, while achieving two goals:
-
Supply and demand are balanced, that is, the transmission map satisfies the Jacobi equation
-
The transmission cost is minimal,
Such a mapping is calledoptimal transport mapping, the total cost of the optimal transmission mapping is said to be the probability distribution betweenWasserstein distanceaway.
When the transmission cost function is the square of the Euclidean distance, according to the Brenier theory, there is a convex function, the so-called Brenier potential energy function, and its gradient map gives the optimal transmission map. At this time, the Brenier potential energy function satisfies the classic Monge-Ampere equation:
The Monge-Ampère equation theory studies the existence, uniqueness and regularity (smoothness) of the solution. These properties are strongly dependent on the convexity, the smoothness of its boundary, the smoothness of the probability density function and the nature of the cost function.
Historically, the French school, the Russian school and the Chinese school have all made outstanding contributions to the Monge-Ampere equation.
A brief historical review
french traditionSince Napoleon formulated the national policy of making mathematics the foundation of the country, France has established a long mathematical tradition. French culture gives mathematicians a high social status. French mathematicians are even more shining stars.
The optimal transmission problem was first proposed by the French mathematician Monge in 1781, and Monge and Ampere started this theoretical direction. There is a subway station in Paris named after Monge to highlight his mathematical contributions. In the ancient city of Lyon, there is a street named after Ampere. Lyon is located at the confluence of the Rhône and Saône rivers. It has beautiful scenery and a long history. It has ancient Roman ruins and is also the end point of the Silk Road. Lyon has a developed economy and a rich collection of people. In the 1980s, Brenier further elucidated the relationship between optimal transmission and the Monge-Ampere equation. His student Cedric Villani applied optimal transmission theory to differential geometry and statistical physics, especially the proof of nonlinear Landau damping and the study of the convergence of the Boltzmann equation to equilibrium. He won the 2010 Fields Award. Villani's work is done in Lyon. Villani's student Alessio Figalli studied the regularity theory of optimal transmission mapping and its internal connection with the Monge-Ampere equation, especially the prior estimation of the second-order derivative of the solution to the Monge-Ampere equation, and the application of geometric inequalities. He won the Fields Medal in 2018.
Currently, Villani is a hot figure in French politics and has a lot of power in formulating science and technology policy. Not long ago, he persuaded President Macron to invest 1.5 billion euros in the development of artificial intelligence. This policy greatly promoted the development and application of optimal transmission theory, thus triggering an upsurge in research on the Monge-Ampere equation.
russian traditionThe French royal family once offered a reward for solving the optimal transmission problem. In the 1940s, the Russian mathematician Kantorovich proved the existence of understanding by relaxing the optimal transmission mapping into an optimal transmission scheme, that is, converting the mapping into a joint probability distribution, and transforming the optimal transmission problem into a linear programming problem.
由此，Kantorovich获得了1975年诺贝尔经济奖。 Kantorovich's formula is equivalent to its dual form,
Currently, the vast majority of work in the field of deep learning is optimized based on Kantorovich's dual form.
Figure 2. Alexandrov problem.
The proof of the existence of solutions to the Monge-Ampere equation was first given by Alexandrov. The Alexandrov problem is similar to the Minkowksi problem. Its discrete formulation is as follows: given the area and normal vector of each face of the convex polyhedron, determine the convex polyhedron. In fact, the Alexandrov convex polyhedron is the Brenier potential energy function, and Alexandrov's geometric formulation is more intuitive. Currently, in the fields of computer graphics, computer vision, and geometric modeling, algorithms for optimal transmission problems are all based on Alexandrov's geometric formulation. In addition, Pogorelov also gave an estimate of the regularity of the Monge-Ampere equation.
Levy, Nirenberg, Calabi, Caffarelli and others of the American school have also made outstanding contributions to the theory of the Monge-Ampere equation.
Chinese traditionUnder the leadership of Mr. Qiu Chengtong, Chinese mathematicians have made outstanding contributions in the field of Monge-Ampere equation theory. In the 1970s, Mr. Yau used the complex Monge-Ampere equation theory to prove the Calabi conjecture and affirmed the existence of Calabi-Yau space. Decades later, theoretical physicists developed superstring theory in an effort to unify general relativity and quantum mechanics. In superstring theory, there is a Calabi-Yau space as a fiber at every point in the universe. The topological and geometric properties of this space determine the origin of the universe.
In 1990, Mr. Yau proposed 100 open problems covering an extremely wide range of topics, from differential geometry to complex geometry to algebraic geometry. The 21st question clearly puts forward the relationship between reflective surface design and optimal transmission mapping as well as the Monge-Ampere equation. In 1982, Mr. Yau won the Fields Medal for his outstanding contributions in the fields of differential equations, the Calabi conjecture in algebraic geometry, the positive mass conjecture in general relativity, and the real and complex Monge-Ampere equations.
Mr. Qiu greatly promoted the development of the Monge-Ampere equation theory, and he spared no effort to call on Chinese mathematicians to devote themselves to this field. Many Chinese mathematicians were inspired by Mr. Qiu and made outstanding contributions to the Monge-Ampere regularity theory. For example, Professor Wang Xujia, an Australian academician, and his collaborators gave the Ma-Trudinger-Wang condition to limit the geometric properties of the density function, transmission cost and probability measure support, thereby ensuring the smoothness of the optimal transmission mapping; for another example, Professor Wang clarified that reflective surface design and free-form lens design are equivalent to the optimal transmission problem on the sphere, and are also equivalent to the Monge-Ampere equation on the sphere. Due to the unremitting efforts of these mathematicians, in recent years, a large number of young Chinese talents are growing in the field of Monge-Ampere equations, and many outstanding figures have begun to show their talents.
paradise on earth
From August 19 to 23, 2019, Academician Wang Xujia, his student Professor Liu Jiakun, and colleague Professor Neil Trudinger organized an academic conference on the Monge-Ampere equation in Kiama, Australia, to celebrate the 60th birthday of Professor John Urbas. The conference invited about forty internationally renowned experts and scholars, covering basic theory, numerical calculation and application fields. Everyone came from all over the world to have a week-long lively discussion in Kiama, which is like a "lost paradise".
Figure 3. Lost Paradise - Kiama.
Australia is a vast country with sparsely populated areas and a pristine natural environment that is pure and pristine. Kiama is also a famous tourist destination in Australia. The water ranges from blue to emerald green and is crystal clear. Everyone lives in a hut on the shore, which can be said to be facing the sea and the flowers are blooming in spring. At night, the waves outside the window are turbulent, and the waves are crashing on the shore. When you look up, the bright moon is in the sky, and the Milky Way is bright; at dawn, the waves are calm, the sea and sky are the same color, and the sunrise is gushing. Apart from humans, Australia's ecology lacks top predators. Wild animals do not need to pursue bloodthirsty violence, but evolve along a relatively peaceful path. Therefore, many primitive and gentle species have developed here, such as various marsupials, kangaroos, koalas and so on. Due to the lack of natural enemies, a large number of species have evolved cute, dull, and naive behaviors. At the same time, since they no longer develop violence, a large number of birds focus on developing beauty to win the competition for survival. For example, the bird of paradise in Papua has developed its beauty to the extreme, with its feathers being spectacular and colorful. There are a large number of parrots, cormorants, ospreys, seagulls and other seabirds gathered on the shore of Kiama. They are not afraid of humans and enjoy themselves leisurely on the shore. In sharp contrast, the seagulls along the coast of New York are sinister, violent, and bloody. Visitors to Long Island often go to the shore to fish for crabs, placing chicken legs in special cages to lure the crabs into traps. At the moment when the chicken legs are thrown into the sea, seagulls often fly by and swallow the whole chicken legs into their mouths. At the same time, dozens of seagulls rush forward and fight in the air. Australian animals are very gentle, simple and honest, and have nothing to do with the world. However, they are favored by the sky and enjoy the paradise-like natural environment. I heard that the living conditions in the inland deserts of Australia are harsh and extremely poisonous snakes and scorpions have evolved.
Figure 4. Parrot on the beach.
Australia's food is also unique in the world. Kiama's famous seafood is actually the ancestor of mankind, barramundi. The Australian king crab is huge, like an octagonal round table, and one is enough for ten adults to have a meal; the smaller Australian snow crab is snow-white, enough for two people to enjoy a meal. On both sides of the Australian highway, cattle and sheep are everywhere, wearing various blankets in winter, happily foraging for food. Professor Wang Xujia had a barbecue at his residence to entertain the participating scholars, and randomly bought Australian lamb chops from the supermarket, which tasted extremely delicious. Even in Michelin restaurants in Beijing or New York, it is difficult to taste such rich and pure lamb chops. The BBQ prawns are huge, plump and juicy, with a long aftertaste.
Kiama has a unique spectacle - the blowhole. The coast of Kiama is made of volcanic rock with a high iron content and is very hard. At the same time, volcanic rocks contain a softer mineral. Due to natural weathering and waves, the mountains on the shore were hollowed out to form a shaft structure. At the same time, a tunnel is formed in the underwater part of the mountain. Under the combined action of strong sea winds and seawater tides, waves can rush into the corridor and spurt out in the shaft, forming a spectacular fountain. We had heard about the name for a long time, and everyone went there almost every day during the meeting, waiting for the right wind direction and tide. But the weather was not good and we never waited for the right weather conditions. Finally one day, the sea breeze roared, the tide surged, and the blowhole spectacle reappeared. Standing near the blowhole, listen to the roar of the waves coming from far and near. In an instant, a sky-high pillar spurts out from the middle of the mountain, with waves splashing and colorful rainbows, which makes people excited and overwhelmed. The excited Professor Hong Minchun called it the "Monge explosion"!
Figure 5. Pure and beautiful “white beach”.
Australia's "white sand beaches" are famous all over the world. The sand is as white as snow and stretches for miles, the water is clear and the sky is blue. Wandering along the coast, I forget both things and myself, and feel relaxed and happy. Everyone talked about the various techniques for a priori estimation of the Monge-Ampere equation, and they all felt that it was a miraculous workmanship and a natural phenomenon. Dolphins swim in the sea, their streamlined bodies are smooth and supple, and their swimming postures are smooth and elegant. Dolphins are very understanding of human nature and interact with the cruise ship happily.
exchange of ideas
Pure mathematicians are the most naive and simple group in human society. They choose to live outside the world and spend most of their time living in the abstract world of mathematics, thus avoiding the world. Unlike politicians and businessmen, they rarely use political skills or capital tricks to gain profits. Instead, they gain recognition from the world through their ideological talents and use aesthetic values to inspire all living beings. It is always a refreshing experience to exchange ideas with pure mathematicians. They are low-key and humble, profound and meaningful, and always enlighten people in philosophy and mathematics. The theory of the Monge-Ampère equation is relatively difficult, and experts in this field all possess rare treasures and extraordinary skills. Academician Fang Fuquan, Lao Gu's uncle, once publicly praised Academician Wang Xujia for "having real skills"! I have benefited a lot from communicating with these mathematicians.
Senior Brother Professor Cao Huaidong once used the Keller-Rich flow method to give another proof of Calabi's conjecture, and supplemented Peralman's rough proof, using the Rich flow method to give all the details of the proofs of Poincaré's conjecture and Thurston's conjecture. Lao Gu asked Brother Cao for advice on how to generalize the Abel-Jacobi theorem from complex manifolds to real manifolds. Brother Cao suggested that Lao Gu study real algebraic geometry. Brother Cao praised Lao Gu's research style, which allows him to study whatever mathematical theories he needs. Compared with mathematicians who specialize in one direction, he is more knowledgeable. Lao Gu believes that the work of pure mathematicians is more profound and original. They apply mature mathematical theories to the field of engineering, and their depth and pioneering are lower than those of pure mathematicians.
Professor Yuan Yu from the University of Washington is a senior fellow in the Department of Applied Mathematics at Tsinghua University who graduated in 1986, and is three grades higher than Lao Gu. At that time, theoretical classes in the Department of Mathematics and Computer Science at Tsinghua University were held at the same time. Several girls from Lao Gu's mathematics class have become the wives of Yuan Yu's classmates, which proves that for the scarce resource of school girls, senior brothers are the top predators. Professor Yuan said that his first love was geometry, and he now makes a living from partial differential equations. He highly admires Mr. Hua Luogeng's philosophical thoughts on analysis and geometry: "When numbers are lacking in shape, it is less intuitive, and when there are few shapes, it is difficult to understand the subtleties." When he was young, he disapproved of multiple solutions to a problem, but he changed his views as he matured. He requires himself to solve every mathematical problem once with analysis and once with geometry, and try to use as few advanced analytical tools as possible, reduce the order of derivation as much as possible, and gain insights from a more essential level. This is very much in line with Lao Gu's wishes. For Lao Gu's work, the understanding from analysis to geometric level is in the preparatory stage, and the final transformation of mathematical theorems into computer algorithms is truly completed. Algorithm design is more based on the geometric nature of the problem. Many difficult analyzes have proven that the geometric ideas behind it are very natural and intuitive. On one level, the theory of the Monge-Ampere equation is a powerful example. Professor Yuan emphasized the important role of a priori estimation in the theory of partial differential equations: "On the one hand, a priori estimation helps to understand the properties of the solution to the equation in advance; on the other hand, a priori estimation helps to prove the existence of the solution. For example, we use the fixed point method to prove the existence of the solution, and the iteration should be carried out in the invariant space. Therefore, we need corresponding derivative estimation."
Professor Yuan summed up the optimal transmission mapping theory with just a few strokes on the note. From the Kantorovich dual functional, we get
Taking the derivative of x on both sides, we have
Substituting, we get the general form of the Monge-Ampere equation:
If the transmission cost is the square of the Euclidean distance, then the cost function is equivalent to
Substituting into the above formula, the optimal transmission mapping and its inverse mapping are given by the following formula
Thus we have
We draw the curve,
The areas below and above the curve are respectively
From this we proved
Pick any point on the curve and examine the derivative at this point, that is, the direction of the tangent, we have
In the same way, we regard x as a function of y, so we get
From this we prove that in general the Hessian matrix is reciprocal,
The words are concise and concise, and the power is clear from the back of the paper.
Very few of Lao Gu's Tsinghua classmates have obtained Ph.D. degrees, and even fewer have stayed in academia. The vast majority of computer science students end up going to Silicon Valley and Wall Street. Professor Yuan was deeply moved by this misfortune. These Tsinghua students all have great potential and should have great ambitions to make greater contributions to mankind. In fact, in modern business society, money has become the only measure of value. Adhering to idealism and becoming a pure mathematician requires great courage and strong determination, and often requires paying a huge price. For an individual, there is nothing wrong with going down the river and pursuing comfort; for a nation, it is essential to face difficulties and stay the center of things. Tsinghua University really needs to cultivate talented people like Senior Brother Cao and Senior Brother Yuan who are broad-minded and broad-minded.
Professor Guan Bo of Ohio University is from Zhenghuang Banner and has lived in Ningguta, the birthplace of the Manchu Qing Dynasty, for generations. Professor Guan studied engineering as an undergraduate and transferred to partial differential equation theory as a graduate student, making outstanding contributions in Weyl isometric embedding. At this meeting, Professor Guan explained another more direct proof of Weyl's theorem based on the Monge-Ampere equation. Lao Gu and Professor Guan discussed the similarities and differences between engineering disciplines and basic theories. The purpose of theoretical work is to broaden the boundaries of human knowledge and pursue eternal value; the purpose of engineering science is to extend the boundaries of human technology and emphasize current practical value. However, because they are too close to current technology, engineers' creations are easily abandoned by the times. There was once a talented student in the junior class of HKUST who was studying in our department how to use the Lattice Boltzman Machine to solve fluid mechanics equations and conduct fluid simulations. He spent six years designing a circuitous and exquisite algorithm based on the GPU hardware conditions at the time. However, on the eve of his PhD defense, a new version of GPU came out. Due to hardware improvements, the complex and circuitous algorithms he designed became worthless, which was a huge blow to him. Many scholars in the engineering field are unable to compile their papers into books in their later years because their research has become outdated and has lost value. This is a lifelong regret no matter what. From this perspective, mathematicians’ contributions to mankind are more lasting. At the same time, Lao Gu feels that mathematics has a stronger spiritual influence. When mathematicians meet, they always discuss mathematics with excitement and are deeply intoxicated by the beauty and profundity of mathematics. Although senior engineers are also obsessed with technology, they are more willing to discuss entrepreneurship and financing.
Professor Wang, Professor Wang's student Chen and postdoctoral fellow Huang Genggeng once proved the convergence of a numerical algorithm for the Monge-Ampere equation and estimated the convergence order. Chen’s experience is very shocking and admirable: due to a mathematics competition in high school, he was admitted to the Computer Science Department of Peking University. After studying computer vision and pattern recognition for two years, he felt that computer science was not as profound as pure mathematics, so he tried to transfer to the Department of Mathematics. At that time, Peking University’s system was relatively rigid and it was impossible to change departments. In the end, despite the hard advice of his family and teachers, Chen resolutely dropped out of school and retake the college entrance examination, and entered the Department of Mathematics of Zhejiang University to study. After graduating from the undergraduate program, he went to Professor Wang and learned the true teachings. "At that time, pattern recognition algorithms were relatively primitive, and it was hard to imagine that algorithms many years later would use the Monge-Ampere equation." Chen said lightly. Lao Gu knew very well how much courage it took for Mr. Chen to cut off his wrist! Now that I have completed my doctoral studies, my long-cherished wish has come true. Lao Gu asked with a smile: "Have you learned Academician Wang's unique skills and become familiar with them? From now on, if you go down the mountain and travel around the world, can you be defeated by thousands of people?" Classmate Chen smiled and did not answer.
Lao Gu was very moved by Chen's idealism. Just like Professor Wang Xujia, pure mathematicians rely on their own talents and creativity to become independent from the world. This requires decades of hard work, extraordinary courage and the right opportunities. The vast majority of students do not have strong confidence in their talents and pursue a stable and prosperous life, thus giving up the study of pure mathematics prematurely. However, after experiencing the turbulent youth stage and having a stable life, many professors in the field of engineering often regret that they cannot enter the field of modern mathematics. Lao Gu also met some students with undergraduate mathematics background. After switching to the computer field, they were very dissatisfied with the style of scientific research in the computer field. After interning in the research departments of top companies, they feel that a large amount of research only relies on "brainstorming" and cannot satisfy rigor and profundity. There are also students who cried on the day they graduated from the master's degree in computer science because they felt that they had not learned any profound knowledge. Of course, some students who have gone into industry regret that they did not graduate as early as possible and missed making money when they met in the future; many students who have gone into academia regret that they graduated too early and failed to fully understand many profound geometric and topological theories.
Mathematics has been developed for more than two thousand years, and computer science has only been developed for a few decades. The profundity and rigor of mathematics naturally far exceed that of computer science. However, the speed of computer development far exceeds the speed of mathematics development. With the explosion of technology, many abstract and profound modern mathematics are penetrating into computer science and becoming classic computer algorithms. In particular we are witnessing an explosion of deep learning methodologies, allowing computational optimal transport theory and the Monge-Ampere equation to rapidly become emerging branches of computer science.
technological explosion
medical imagesThe rise of medical imaging technologies such as MRI and CT scans has completely revolutionized the medical field. Medical diagnosis and treatment increasingly rely on medical images. How to compare two medical images has become an indispensable technology. If we regard the gray value of a medical image as a probability density function, comparing the two images is equivalent to obtaining a diffeomorphism, so that the probability density matches and the image distortion is minimized. This happens to be modeled by optimal transport mapping. It is conceivable that with the improvement of computing hardware conditions and improvement of algorithm efficiency, the Monge-Ampere equation will be increasingly used in the medical field.
wireless communicationWireless communication technology is developing rapidly, and human beings cannot leave their mobile phones for a moment. The Monge-Ampere equation can help design antenna equipment and improve communication efficiency. The combination of optimal transmission theory and information theory is expected to achieve fundamental breakthroughs at the theoretical level. The relationship between information entropy and optimal transmission, the relationship between entropy and Wasserstein space geodesic, the relationship between entropy and bottom manifold curvature, and the information theory explanation of Brunn-Minkowski inequality, these are all promising research directions.
Figure 6. Adaptive car light intensity distribution, Audi concept.
Automobile industryHalf of all car accidents currently occur at night. How to increase the light intensity of car lights while avoiding dazzling glare for oncoming drivers has always been a key issue in car light design. In recent years, with the maturity of new light source technology and the development of optical processing technology, adaptive vehicle lighting technology has begun to emerge. As shown in Figure 6, the light intensity distribution of car lights in the distance is no longer uniformly distributed, but is specified by the designer. This problem is called a reflector design problem. Reflector design is essentially an optimal transmission problem, which can be described and solved by the Monge-Ampere equation on a curved surface. Due to the development of 3D printing technology, the processing of free-form optical devices is becoming increasingly mature, and new car lights will be applied in the near future.
deep learningA type of natural data can be regarded as a probability distribution on a low-dimensional manifold in a high-dimensional data space. The core of deep learning is to learn the structure of the manifold, represented by encoding and decoding mapping, and learn the probability distribution at the same time. In the latent space, the data distribution is very irregular and cannot be characterized by the classical distribution with parameters. Instead, it can be represented by the optimal transmission mapping from the known Gaussian distribution to the data distribution. The discriminator in an adversarial generative model calculates the Wasserstein distance between the true distribution and the generated distribution, and the generator calculates the transfer map from the latent space white noise to the generated distribution. Therefore, we can use optimal transfer theory to explain and design deep learning models. This theoretical framework can perfectly explain the intrinsic reasons for mode collapse, improve training stability and efficiency, and thus make the black box of deep learning transparent.
For example, Academician Wang has made outstanding contributions in the regularity of the Monge-Ampere equation. However, for a long time, because computers can only express discrete data, the computer field has not paid much attention to the regularity of solutions to partial differential equations. However, recently we have discovered that the mode collapse problem of the GAN model is closely related to the regularity of the solution to the Monge-Ampere equation. In 1996, Academician Wang proved in an article on light reflection that if the target area is non-convex, there is a positive smooth distribution defined on the source area and the target area, making the optimal transmission mapping discontinuous. In 2005, Academician Wang extended this theorem to any cost function, including the Monge-Ampere equation, in an article with Mahinan and Trudinger. This means that if the support of the target measure is non-convex, then the optimal transmission map is no longer globally continuous and there is a set of singular points. Currently, deep neural networks can only express continuous mapping and cannot express transmission mapping. This contradiction causes model collapse. We found this strange collection on the face image manifold. (For details, please see the theoretical explanation of GAN mode collapse)
Outlook
There were approximately 40 scholars participating in this Monge-Ampère equation conference, which should be a fairly large-scale conference in the field of mathematics. International conferences on computer vision and deep learning have about 20,000 attendees. These two thousand people should all know about the Generative Adversarial Network (GAN), have heard of the Wasserstein distance, and naturally are eager to understand the Monge-Ampere equation. The popularization and development of the Monge-Ampere equation theory will surely reach a climax. This is an unprecedented situation in the field of nonlinear analysis, and it is a once-in-a-lifetime opportunity.
I believe that with the development of technology, the difficult Monge-Ampere equation theory will be mastered and appreciated by more scholars, enlighten more young students, discover more engineering applications, and thus increasingly enter thousands of households and integrate into the wave of modern science and technology!
(Thanks to Professor Wang Xujia, Professor Guan Bo, Professor Yuan Yu, Professor Liu Jiakun and classmate Chen for their help in revision and correction.)
Machine-translated from the Chinese original 技术爆炸中的蒙日-安培方程 (published 2020-05-03). WeChat original ↗