A mathematical system that you need to know if you want to climb higher.

Over the past year, I have been wandering in the sea of mathematics, and research has not made much progress. I have gained some progress in the world of mathematics. Why do I go deep into the world of mathematics as a computer student? I have no attempt to become a mathematician. I want to climb the shoulders of giants and hope to stand at a higher height so that I can see more things I have studied. When I first came to this school

Over the past year, I have been wandering in the sea of mathematics, and research has not made much progress. I have gained some progress in the world of mathematics.

Why go deep into the world of mathematics

As a computer student, I have no attempt to become a mathematician. I want to climb the shoulders of giants and hope to stand at a higher height so that I can see more things I have studied. When I first came to this school, I did not expect that I would have a journey into mathematics. My mentor initially wanted me to create a unified model for appearance and motion. This topic is nothing special in today's world of Computer Vision. In fact, the use of various Graphical models to combine various things into the framework is not uncommon in recent years.

I do not deny that the widely used Graphical Model is a powerful tool for modeling complex phenomena. However, I think it is not panacea, and it cannot replace in-depth research on the problems studied. If you study statistics to cure all diseases, there is no need for many "downstream" disciplines. In fact, at the beginning, I was also the same as many people in Vision who wanted to create a Graphical Model. my mentor pointed out that this practice was just to repeat some standard procedures, there is no great value. After a long period of repetition, another path was gradually established-we believe that an image is made up by a large number of "atomic" spatial distributions, the movement of an atomic group forms a dynamic visual process. In the microscopic sense, the movement of a single atom is deeply related to the transformation of the overall distribution in the macro sense. This requires us to explore.

In the process of in-depth exploration of this question, I encountered many problems. how do I describe a general motion process and how to establish a stable and widely applied atomic expression, there are many ways to portray the relationship between micro motion and macro distribution transformation. In this process, I found two things:

• My original mathematical foundation is far from applicable to my in-depth research on these issues.
• In mathematics, there are many ideas and tools that are very suitable for solving these problems, but they are not valued by many researchers who apply science.

As a result, I am determined to go deep into the vast sea of mathematics. I hope that when I come out again, I will have more powerful weapons to face these challenges.

My tour is not over yet, and my vision is still very narrow compared to the broad and profound world. Here, I just want to talk about how mathematics develops from elementary to advanced step by step in my eyes. What are the advantages of advanced mathematics for specific applications.

Set theory: the foundation of modern mathematics

Modern mathematics has countless branches, but they all have a common Foundation-set theory-because of it, the family of mathematics has a common language. Set theory has some basic concepts: set, relation, function, equivalence ), it is almost inevitable in the languages of other mathematical branches. Understanding these simple concepts is the foundation for further learning other mathematics. I believe that college students in science and engineering are not unfamiliar with this.

However, a very important thing is not so well-known-the Axiom of Choice ). This principle means that "any group of non-empty sets can definitely take out one element from each set ." -- It seems that it is obviously impossible to make a clear proposition. However, this seemingly ordinary principle can deduce some strange conclusions, such as the pinball theorem of he-Taki-"one ball can be divided into five parts, after performing a series of rigid transformations (translation rotation) on them, they can be combined into twoSame size". It is precisely because of these conclusions that are totally against common sense that the mathematics field once had a heated debate over whether to accept it for a long time. Now, mainstream mathematicians should basically accept it, because many important theorems of mathematical branches depend on it. In the subjects we will discuss later, the following theorem relies on the choice principle:

1. Topology: Baire Category Theorem
2. Real analysis (measure theory): the existence of the unmeasurable set of Lebesgue
3. Four major theorems of functional analysis: Hahn-Banach Extension Theorem, Banach-Steinhaus Theorem (Uniform boundedness principle), Open Mapping Theorem, Closed Graph Theorem

Based on the set theory, modern mathematics has two families: Analysis and Algebra ).As for others, such as Ry and probability theory, in the classical mathematical age, they are tied with algebra, but their modern versions are basically based on analysis or algebra, therefore, in the modern sense, they are not parallel to analysis and algebra.

Analysis: The magnificent building built on the limit

1. calculus: the classical age of analysis-from Newton to kherk

Let's talk about Analysis first. it evolved from calculus (Caculus). This is why some calculus textbooks are called "Mathematical Analysis. However, the scope of analysis is far more than that. the calculus we study in the first year of college can only be an entry to classical analysis. Many objects of analysis are studied, including derivatives, integral, differential equation, and infinite series, it is introduced in elementary calculus. If there is an idea that runs through it, it is the limit-this is the soul of the entire analysis (not just calculus.

A story that many people have heard of is the debate between Newton and Leibniz about the 'electric rights' of calculus. In fact, in their time, many calculus tools began to be applied in science and engineering, but the foundation of calculus was not really built. The ghost of an infinitely small volume that has been unclear for a long time has plagued the mathematics field for more than one hundred years-this is the second mathematical crisis ". It was not until kherk re-established the basic concepts of calculus from the perspective of the limit of the series that this discipline began to have a solid foundation. Until today, the entire analysis building is built on the foundation of the limit.

He provided a rigorous language for the development of analysis, but he did not solve all the problems of calculus. In the 19th century, the world of analysis still had some lingering dark clouds. The most important one is whether the function can be accumulated ". The point we learned in the current calculus textbook that we used to take the Matrix Area and the limit through "infinitely segmented intervals" was proposed by Riann in 1850, it is called a Riman integral. However, what function has a Cartesian integral )? Mathematicians have long proved that the continuous function defined in the closed interval is product-free by the family. However, this result is not satisfactory, and engineers need to integrate functions of piecewise continuous functions.

2. Real Analysis: A Modern analysis is established in the theory of real number and measurement.

In the middle and late 19th century, the problem of non-continuous functions is always an important issue of analysis. The research on the Cartesian integral defined on the closed interval shows that the key to the accumulation is that there are few discontinuous points ". Only finite discontinuous functions are product-able, but many mathematicians have constructed many non-sequential product functions in an infinite number. Obviously, when measuring the size of a point set, limitations and infinity are not an appropriate standard. During the discussion of the "point set size" issue, mathematicians found that the real number axis, a thing they once thought was fully understood, has many characteristics that they did not think. With the support of the limit idea, the real number theory is established at this time. it marks several equivalent theorems describing the completeness of real numbers (definite theory, interval set theorem, mean convergence Theorem, Bolzano-Weierstrass Theorem and Heine-Borel Theorem) -- these theorems clearly express the fundamental difference between a real number and a rational number: completeness (not strictly speaking, is closed to the limit operation ). With the deep understanding of real numbers, the problem of how to measure the size of a point set has also been broken through. leberger creatively considers the algebra of a set, combined with the concept of Outer content (that is, a prototype of "external Measure"), the measurement Theory is established ), in addition, we further established the Lebesgue Integral, an Integral based on measure ). With the support of this new point concept, the problem of accumulation becomes clear at a glance.

The Real number theory, measure theory, and leberger points mentioned above constitute the mathematical branch that we now call Real Analysis. some books are also called Real variable function theory. For application Science, real analysis does not seem as "practical" as classical calculus-it is difficult to obtain any algorithm directly based on it. Moreover, it is unrealistic for engineers to solve some "difficulties", such as non-continuous functions everywhere or continuous and non-traceable functions everywhere. However, I think it is not a pure mathematical concept game. its practical significance lies in providing a solid foundation for many modern branches of applied mathematics. Below, I will only list a few of its usefulness:

1. The functional space of the product is incomplete, but the functional space of the product is complete. Simply put, the function to which a column of the functions that can be accumulated by a column of the Cartesian model is not necessarily the product of the column, however, the function columns of the leberger product must converge to a function of the leberger product. In functional analysis and approximation theory, we often need to discuss "the limit of a function" or "the series of a function". if we use the concept of a Cartesian point, this discussion is almost unimaginable. We sometimes look at the Lp function space mentioned in some paper, which is based on leberger points.
2. Leberger points are the basis of Fourier transformation (which is everywhere in engineering. Many elementary teaching materials on signal processing may bypass leberger points and talk about practical things without having to talk about its mathematical basics. However, this is not always the case for in-depth research issues-especially those who want to do some work in theory.
3. As we can see below, the measurement theory is the basis of modern probability theory.

Topology: analysis is extended from the real number axis to the general space -- Abstract basis of modern analysis

With the establishment of the real number theory, we began to extend the limit and continuous analysis to a more general place. In fact, many concepts and theorems based on real numbers are not unique to real numbers. Many features can be abstracted and promoted to a more general space. The promotion of the real number axis facilitates the establishment of Point-set Topology. Many concepts that originally existed only in real numbers are extracted for general discussion. In topology, four C components constitute its core:

1. Closed set (Closed set ). In modern topology, open set and closed set are the most basic concepts. Everything comes from this. These two concepts are the promotion of open and closed intervals. their fundamental position is not recognized at the beginning. After a long period of time, people realized that the concept of open set is the foundation of continuity, and closed set is closed to limit operation-and the limit is the foundation of analysis.
2. Continuous function (Continuous function ). In calculus, a continuous function is defined in the epsilon-delta language. in topology, it is defined as "opening a set as a function of opening a set ". The second definition is equivalent to the first one, but is rewritten in a more abstract language. I personally think that its third (equivalent) definition fundamentally reveals the essence of a continuous function-"a continuous function is a function that maintains the limit operation"-for example, y is a series of x1, x2, x3 ,... F (y) is f (x1), f (x2), f (x3 ),... . The importance of continuous functions can be compared from other branches. For example, in group theory, the basic operation is "multiplication". for groups, the most important ING is "homomorphic ing"-to maintain the "multiplication" ING. In the analysis, the basic operation is "limit", so the position of continuous functions in the analysis is equivalent to that of homomorphic ing in algebra.
3. Connected set (Connected set ). The concept of Path connected is a little narrower than that of Path connected, which means that any two points in the set are connected in a continuous Path. In general, the concept of connectivity is slightly abstract. In my opinion, connectivity has two important advantages: one is to prove the general mediate Value Theorem, and the other is the algebraic topology, the topic Group theory and Li Qun theory discuss the order of the Fundamental Group (Fundamental Group.
4. Compact set ). Compactness does not seem to appear specifically in elementary calculus, but there are several real-number theorems related to it. For example, "a bounded number of columns must have convergence subcolumns" -- in compactness language -- "The real number space has a bounded closed set ". In topology, it is generally defined as something that sounds abstract-"arbitrary open coverage of a tight set has limited subcoverage ". This definition is very convenient when discussing the topology theorem. it can often help to implement the transition from infinite to finite. For analysis, we use another form of computation-"The number of computation columns in a tightly concentrated sequence must have a consortium subcolumn"-which reflects the most important "limit" in analysis ". Compactness is widely used in modern analysis and cannot be fully described. Two important theorems in calculus: Extreme Value Theory and Uniform Convergence Theorem can be used to promote them to the general form.

In a sense, the point set topology can be viewed as a general theory about the "limit". It is abstracted from the real number theory and its concept becomes a general language of almost all modern analysis disciplines, it is also the foundation of modern analysis.

Differential RY: Analysis on a manifold -- introducing a differential structure in a topological space

Topology promotes the concept of limit to a general topology, but this is not the end of the story, but the beginning. In calculus, we have differentiation, derivation, and points after the limit. These things can also be extended to the topological space and established on the basis of topology-this is the differential Ry. In terms of teaching, there are two different types of textbooks for differential ry. one is the "classical differential ry" based on the classical microcomputer ", it mainly involves the calculation of some geometric quantities in 2D and 3D spaces, such as curvature. Another is based on modern topology, which is called "modern differential ry". its core concept is manifold) -- adds a structure based on the topological space that can perform differential operations. Modern Differential ry is a very rich discipline. For example, generally, the definition of differentiation on a manifold is richer than that of a traditional one, and I have seen three equivalent definitions from different perspectives-this makes things more complex, however, in another aspect, it gives different understandings of the same concept, which often leads to different ideas when solving the problem. In addition to the concept of calculus, many new concepts are introduced: tangent space, cotangent space, push forward, pull back, fiber bundle, flow, immersion, submersion, and so on.

In recent years, manifold learning seems quite fashionable in machine learning. However, frankly speaking, to understand some basic manifold algorithms, or even "create" some manifold algorithms, there is no need for the basis of differential Ry. For my research, the most important application of differential ry is another branch built on it: Li Qun and Lie algebra-this is a beautiful marriage between two family analyses and algebra in mathematics. Another important combination of analysis and algebra is functional analysis and harmonic analysis based on it.

Algebra: an abstract world

1. Abstract algebra

Let's look back at another big family-algebra.

If classical calculus is an introduction to analysis, the entry points of modern algebra are two parts: linear algebra and abstract algebra) -- it is said that some textbooks in China are called modern algebra.

Algebra -- the name seems to be studying numbers. in my opinion, the main research is computational rules. In fact, an algebra abstracts some basic rules from a specific computing system, establishes an ordinary system, and then studies it on this basis. Adding a set of operation rules to a set constitutes an algebraic structure. In the main algebraic structure, the simplest is Group, which has only one reversible operation that conforms to the combination rate, usually called multiplication ". If this operation also matches the exchange rate, it is called the Abelian Group ). If there are two kinds of operations, one is addition, which satisfies the exchange rate and combination rate, and the other is multiplication, which satisfies the combination rate and satisfies the allocation rate. this richer structure is called Ring ), if the multiplication on the Ring satisfies the exchange rate, it is called a Commutative Ring ). If the addition and multiplication of a ring have all the good properties, it becomes a Field ). Based on the domain, we can establish a new structure that can be used for addition and multiplication to form a Linear algebra (Linear algebra ).

The advantage of algebra is that it only cares about the deduction of calculation rules, regardless of the objects involved in calculation. As long as the definition is correct, a cat can get a pig by taking a dog :-). All the theorems obtained based on abstract calculation rules can be applied to the cat/dog multiplication mentioned above. Of course, in practice, we still hope to use it to do something meaningful. All those who have learned abstract algebra know that based on the simplest rules, such as the combination law, A lot of important conclusions can be derived-these conclusions can be applied to all the places that satisfy these simple rules-where is the power of algebra, we no longer need to create so many new theorems for each specific domain.

Abstract algebra is based on some basic theorem. Further research is often divided into two schools: studying finite discrete algebra structures (such as finite groups and finite fields). This part is usually used in number theory and coding, and integer equations. Another school is studying continuous algebra structures, usually associated with topology and analysis (such as topology groups and Li Qun ). In my learning, the focus is mainly the latter.

2. linear algebra: basic position of "linearity"

For people who are engaged in Learning, vision, optimization, or statistics, linear algebra is the most commonly used. this is what we started Learning in the lower grades of college. Linear algebra includes various disciplines based on it. the two core concepts are vector space and linear transformation. The position of linear transformation in linear algebra is the same as that of continuous functions in analysis or homomorphic ing in group theory-it maintains the ing of basic operations (addition and multiplication.

In learning, there is a tendency to despise linear algorithms and advertise non-linearity. In many cases, we may need non-linearity to describe the complex real world, but at any time, linearity is fundamental. Without linear Foundation, it is impossible to have the so-called non-linear extension. Our commonly used nonlinear methods include manifold and kernelization, both of which require regression linearity at a certain stage. The ing between a manifold and a linear space must be established in each region, and many local linear spaces are connected to form a non-linear relationship; kernerlization maps the original linear space to another linear space by replacing the inner product structure, and then performs operations in the linear space. In the field of analysis, linear computation is everywhere. differential, integral, Fourier transform, Laplace transformation, and mean in statistics are all linear.

3. Functional Analysis: moving from finite dimension to infinite dimension

Linear algebra learned in college is simple mainly because it is carried out in a finite dimension space. because of its limitations, we do not need to resort to too many analysis methods. However, a finite dimension space cannot effectively express our world. most importantly, a function forms a linear space, but it is an infinite dimension space. The most important operations on functions are performed in an infinite dimension space, such as Fourier transform and wavelet analysis. This shows that in order to study functions (or continuous signals), we need to break the limitations of finite-dimensional space and enter the infinite-dimensional function space-the first step here is functional analysis.

Functional Analysis is a general linear space, including finite and infinite dimensions. However, many things seem very trivial in finite dimension, real difficulties often occur in the infinite dimension. In functional analysis, elements in space are still called vectors, but linear transformations are usually called operators ). In addition to addition and multiplication, some operations are further added here, such as adding a norm to express "vector length" or "element distance ", this space is called "normed space" and can be further added to the Inner product space ).

We found that when we enter the infinite dimension, many old ideas no longer apply, and everything needs to be reviewed again.

1. All finite-dimensional spaces are complete (convergence of the sequence of the keys), while many infinite-dimensional spaces are incomplete (such as continuous functions on closed intervals ). Here, a complete space has a special name: a complete norm space is called a real-name space (a real-name space), and a complete inner space is called a Hilbert space ).
2. In a finite dimension space, space and its dual space are completely homogeneous. in an infinite dimension space, they have subtle differences.
3. In a finite dimension space, all linear transformations (matrices) are bounded transformations, while in an infinite dimension, many operators are unbounded (unbounded ), the most important example is to evaluate the function.
4. In a finite dimension space, all bounded closed sets are tight, such as the unit ball. In all infinite dimension spaces, the unit ball is not tight-that is to say, you can scatter an infinite point in the unit ball without a limit point.
5. In a finite dimension space, the spectrum of a linear transformation (matrix) is equivalent to all the feature values. in an infinite dimension space, the structure of the operator's spectrum is much more complex than this, in addition to the point spectrum composed of feature values, there are also approximate point spectrum and residual spectrum. Although complicated, it is more interesting. Thus, Spectrum theory is a very rich branch ).
6. In a finite dimension space, any point always has a projection on any sub-space. in an infinite dimension space, this is not necessarily the case, the sub-spaces with such good characteristics have a dedicated Chebyshev space ). This concept is the foundation of modern approximation theory (approximation theory ). Function space approximation theory should play a very important role in Learning. However, there are not many articles that use modern approximation theory.

4. continue to the next step: he algebra, harmonic analysis, and Lie algebra

There are two important directions for basic functional analysis. The first one is the he Algebra (Banach Algebra), which introduces multiplication (which is different from multiplication) on the basis of the He space (a complete inner product space ). For example, a matrix, in addition to addition and multiplication, can also be used for multiplication. This forms a he algebra. In addition, the fully-qualified bounded operators and square product functions of the value domain can constitute the he algebra. The he algebra is the abstraction of functional analysis. Many conclusions are derived from bounded operators, and there are also many theorems in the operator spectrum theory. they are not only applicable to operators, but can actually be obtained from the general he algebra, it is also applied outside the operator. The he algebra allows you to look at the conclusions in functional analysis from a higher perspective, but I still need to think about what it can bring more in practical problems than functional analysis.

Another important direction that best integrates functional Analysis with actual problems is Harmonic Analysis ). I will list its two sub-fields, Fourier analysis and wavelet analysis. I think this shows its practical value. The core issue of its research is how to use base functions to approximate and construct a function. It studies the problems of function space and must be based on functional analysis. In addition to Fourier and wavelet, harmonic analysis also studies some useful function spaces, such as the hard space and sobolevspace, which have many good properties, it has important applications in engineering and physics. For vision, harmonic analysis is a very useful tool in signal expression and image construction.

When analysis and linear Algebra walk together, functional analysis and harmonic analysis are produced. when analysis and Group theory walk together, we have Lie Group and Lie Algebra ). They give algebra to elements in a continuous group. I always think this is a very beautiful mathematics: in a single system, topology, differentiation, and algebra come together. Under certain conditions, through the Association of Li Qun and Lie algebra, it turns the combination of geometric transformations into linear operations and converts subgroups into linear subspaces, this creates necessary conditions for the introduction of many important models and algorithms in Learning to the modeling of geometric motion. Therefore, we believe that Li Qun and Lie algebra are of great significance to vision, but learning it may be difficult. before it, we need to learn a lot of other mathematics.

Modern Probability theory: regeneration based on modern analysis

Finally, let's talk about the branch of mathematics that many Learning researchers are particularly concerned with: Probability theory. Since Kolmogorov introduced measure into probability theory in the 1930s S, the theory of measure has become the basis of modern probability theory. Here, probability is defined as measurement, random variables are defined as measurable functions, conditional random variables are defined as the projection of measurable functions in a function space, and the mean value is the integral of measurable functions for probability measurements. It is worth noting that many modern ideas start to look at the basic concepts of probability theory with the concept of functional analysis. Random variables constitute a vector space, the signed probability measure forms its dual space, where one party applies to the other party to form the mean value. Although the angle is different, the two methods share the same things, and the foundation is equivalent.

On the basis of modern probability theory, many traditional branches have been greatly enriched. the most representative include Martingale, a theory triggered by gambling, it is mainly used in finance (here we can see the theoretical connection between gambling and finance:-P) and Brownian Motion-the basis of the continuous random process, the Stochastic Calculus (Stochastic Calculus) created on this basis, including random points (points are obtained for the path of the random process, among which Ito Integral is representative )), and random differential equations. This knowledge is indispensable for the Study of applying continuous ry to establish probability models and transformation of distribution.