First, why go into the world of mathematics
As a computer student, I (the original author) did not have any attempt to become a mathematician. The purpose of my study of mathematics is to climb the shoulders of giants, hoping to stand at a higher altitude, to see what I have studied more deeply. Speaking of which, when I first came to this school, I did not anticipate that I would have an in-depth math journey. The first thing my mentor wanted me to do was build a unified model of appearance and motion. There is no special place in the world where the topic is blossoming in today's computer vision. In fact, the use of various graphical model to unite various things together the framework, in recent years in the paper is not uncommon.
I do not deny that the widely popular graphical model is a powerful tool for modeling complex phenomena, but I do not think it is panacea, and it cannot replace the deep delving into the problems studied. If the statistical learning package cures all ills, then many "downstream" disciplines will not be necessary to exist. In fact, at the beginning, I was just like many people in vision, thinking about doing a graphical model--my mentor points out that this is just a matter of repeating standard processes and not having much value. After a long period of repetition, another path is gradually established-we believe that an image is made up of a space distribution of a large number of "atoms", and the movement of the atomic Group forms a dynamic visual process. The single atom movement in the microscopic sense has a profound connection with the transformation of the whole distribution in the macroscopic sense--it needs to be explored.
In the course of exploring this topic, we have encountered many problems, how to describe a general motion process, how to establish a stable and widely applicable atom expression, how to characterize the relationship between micro-motion and macro-distribution transformation, and many more. In this process, I found two things:
My original mathematical foundation is far from being able to adapt to my in-depth study of these problems.
In mathematics, there are many ideas and tools that are well suited to solving these problems, but are not valued by many researchers of applied science.
So, I am determined to begin to delve into the vast sea of mathematics, hoping that when I come out again, I have a more powerful weapon to face the challenges of these problems. My travels are not over, my vision is still very narrow compared to this profound world. Here, I'm just saying, in my eyes, how mathematics moves from primary to advanced, and higher-level mathematics is good for specific applications.
Set theory: The Common foundation of Modern mathematics
There are countless branches of modern mathematics, but they all have a common foundation-the set theory-because it, the large family of mathematics, has a common language. There are some basic concepts in set theory: Set (set), Relationship (relation), functions (function), equivalence (equivalence), which are almost inevitable in the language of other branches of mathematics. The understanding of these simple concepts is the basis for further learning some other mathematics. I believe that science and engineering students are not unfamiliar with these.
However, there is a very important thing is not so well-known-that is, the "Axiom of choice" (Axiom of Choice). This axiom means "any group of non-empty sets that must be able to take an element from each set." "--seems to be clearly no longer the obvious proposition. However, this seemingly ordinary axiom can deduce some rather strange conclusions, such as the Tuskegee----"a ball, can be divided into five parts, a series of rigid transformations (translational rotation), can be combined into two of the same size of the ball."
It is precisely because these are completely contrary to common sense conclusion that the mathematics community has been in a long time to accept it has a heated debate. Now, mainstream mathematicians should be basically receptive to it, because many of the important theorems of mathematical branches depend on it. In the subject we are going to go back to, the following theorem relies on the axiom of choice:
Topology: Baire Category theorem
Real analysis (measure theory): The existence of Lebesgue set of non-measurable
Four main theorems of functional analysis: Hahn-banach Extension theorem, Banach-steinhaus theorem (Uniform boundedness principle), Open Mapping theorem , Closed Graph theorem
On the basis of set theory, modern mathematics has two families: Analysis and Algebra (Algebra). As for others, such as geometry and probability, they are tied to algebra in the age of classical mathematics, but their modern versions are basically based on analysis or algebra, so from a modern point of view, they are not parallel to analysis and algebra.
Third, analysis: building on the limit based on the magnificent building
1. Calculus : The Classical era of analysis-from Newton to Cauchy
Let's start with analysis, which is developed from calculus (caculus)--and that's why some calculus textbooks are called "mathematical analysis." However, the scope of the analysis is far more than that, and the calculus we study in the first year of college is only a primer on classical analysis. There are many objects of analysis, including derivatives (derivatives), integrals (integral), differential equations (differential equation), and series (infinite series)--these basic concepts, There are introductions in elementary calculus. If there is a thought that runs through it, it is the limit-the soul of the entire analysis (not just calculus).
A story that many people have heard is the controversy between Newton (Newton) and Leibniz (Leibniz) about calculus invention. In fact, in their time, many of the tools of calculus began to be used in science and engineering, but the foundation of calculus was not really established.
The spectre of "Infinity", which has been unexplained for a long time, has plagued the math world for more than 100 of years-a "second mathematical crisis". Until Cauchy re-established the basic concept of calculus with a limit point of view , the subject began to have a relatively solid foundation. Until today, the entire analysis of the building is based on the cornerstone of the limit.
Cauchy (Cauchy) provided a rigorous language for the development of the analysis, but he did not solve all the problems of calculus. In the 19th century, the world of analysis still had some lingering clouds. One of the most important things that is not solved is the question of whether the function can be integrable .
The kind of integrals that we have learned in our current calculus textbooks, through the "infinite partitioning interval, taking the limits of the Matrix area", was proposed by Riemann (Riemann) in 1850, called Riemann integrals. But what functions have Riemann integrals (Riemann integrable)? Mathematicians have long proved that a continuous function defined within a closed interval is a Boltzmann integrable. However, the results are unsatisfactory, and the engineers need to integrate the function of the piecewise continuous function.
2. Real Analysis : establish modern analysis in theory and measure of real number
In the middle and late 19th century, the problem of the integrable of discontinuous function is always an important issue in the analysis. The study of the Riemann integrals defined on the closed interval shows that the key to integrable is that "discontinuous points are small enough". Only finite discontinuous functions are integrable, but many mathematicians construct many integrable functions that are discontinuous in infinity. Obviously, when measuring the size of a point set, finite and infinite are not a suitable standard .
In the course of exploring the problem of "point set Size", mathematicians discovered that the real axes -the things they once thought were fully understood-had many features they did not think of . With the support of the extreme thought, the real number theory is established at this time, and its symbol is a few equivalent theorems describing the completeness of the real number (the definite reason, the interval set theorem, the Cauchy convergence theorem, the Bolzano-weierstrass theorem and the Heine-borel Theorem, etc.)--these theorems clearly express the fundamental difference between real and rational numbers : completeness (very not strictly speaking, is closed to the limit operation).
With the deep understanding of real numbers, the problem of how to measure "point set size" has also made breakthroughs, and Lebesgue creatively combines the concepts of algebra and outer content(which is an embryonic form of "outer measure") to set up a measure theory ( Measure theory), and further establishes a measure-based integral-Lebee (Lebesgue Integral). With the support of this new integral concept, the integrable issue becomes apparent at a glance.
The theory of real numbers, the measure theory and the Lebesgue integral, are the branches of mathematics wenow call real analysis, and some books are called real-variable functions . For Applied Science, real analysis does not seem to be so "practical" as classical calculus-it is difficult to get directly based on what algorithm it is. Moreover, some of the "puzzles" it needs to solve--such as a discontinuous function, or a function that is ubiquitous everywhere--is not realistic in the eyes of the engineer.
But, I think, it is not a pure mathematical concept game, its practical significance is to provide a solid foundation for many modern applied Mathematics branch . Below, I'll just cite a few of its uses:
1) The function space of Riemann integrable is not complete, but the function space of Lebesgue integrable is complete. In a nutshell, the function that a Lebesgue integrable function column converges to is not necessarily a Boltzmann integrable, but the function column of the integrable can certainly converge to a Lebesgue integrable function. In functional analysis and approximation theory, it is often necessary to discuss the "limit of function", or the "progression of functions", which is almost unthinkable if the concept of Riemann integrals is used. We sometimes look at some of the paper mentioned in the L^p function space, which is based on Lebesgue integrals.
2) Lebesgue integral is the basis of the Fourier transform (which is everywhere in the project). Many of the elementary textbooks on signal processing may have bypassed Lebesgue integrals, talking directly about what is practical and not about its mathematical underpinnings, but for deep-seated research issues-especially the hope of doing some work in theory-it is not always possible to get around.
3) In the following, we will also see that measurement theory is the basis of modern probability theory .
3. Topology : Analysis from the real axis to the general space--the abstract basis of modern analysis
With the establishment of the theory of real numbers, we begin to analyze the limits and the continuous extension to more general places. In fact, many concepts and theorems based on real numbers are not unique to real numbers. Many features can be abstracted and generalized into more general spaces. for the generalization of the real axis, the establishment of the point set topology (Point-set topology) is facilitated . Many of the original concepts that existed only in real numbers were extracted for general discussion. In topology, there are 4 C that make up its core:
1) Closed set closed set
In modern topological axiomatic system, open set and closed set are the most basic concepts. Everything is extended from here. These two concepts are the extension of opening and closing intervals, and their fundamental status is not recognized from the beginning. After a long time, it was realized that the concept of open set is the basis of continuity , and closed set is closed to the limit operation-and the limit is the foundation of analysis.
2) continuous function continuous functions
The continuous function has a definition in calculus that is given in Epsilon-delta language, and in topology it is defined as "the original image of the open set is the function of the open set". The second definition is equivalent to the first one, but is rewritten in a more abstract language. I personally think that its third (equivalence) definition fundamentally reveals the essence of continuous function--"continuous function is the function of maintaining the limit operation"-for example, Y is the sequence x1, x2, x3, ... , if f is a continuous function, then f (y) is F (x1), F (x2), F (x3), ... The limit. The importance of continuous functions can be compared from other branch disciplines. For example, in group theory, the basic operation is "multiplication", for the group, the most important mapping is called "homomorphic mapping"--keep the "multiplication" mapping. In the analysis, the basic operation is the "limit", so the position of the continuous function in the analysis, and the homomorphism mapping in the algebraic position is equivalent.
3) Connected set connected set
A slightly narrower concept, called path connected, is that there are contiguous paths connected to any two points in the collection--probably the concept that is understood by the general people. The concept of connectivity in general is somewhat abstract. In my opinion, connectivity has two important uses: one is to prove the general mean value theorem (intermediate value theorem), there is the algebraic topology, topological group theory and Lie group theory to discuss the order of the fundamental groups (fundamental group).
4) Compact Set compact set
Compactness seems to have no particular presence in elementary calculus, but there are several theorems on real numbers that are actually related to it. For example, "Bounded sequences must have convergent sub-columns"--in compactness language--"bounded closed sets in real space are tight". Its general definition in topology is something that sounds more abstract-"finite sub-coverage exists in the arbitrary open cover of a tight set". This definition is handy when discussing the theorem of topology, which in many cases can help to transform from infinity to finite. For analysis, more is used in its other form-the "tight-set sequence must have convergent sub-columns"-it embodies the most important "limit" in the analysis. Compactness is very widely used in modern analysis and cannot be described. Two important theorems in calculus: The Extremum theorem (Extreme Value theory), and the Uniform convergence theorem (Uniform Convergence theorem) can be generalized to a general form.
In a sense, the point set topology can be regarded as the general theory of "limit", which is abstracted from the theory of real numbers, and its concept becomes the universal language of almost all modern analytical disciplines and the foundation of the whole modern analysis.
4. Differential Geometry : Analysis on manifolds--introducing differential structures on topological spaces
Topology promotes the concept of limits to the general topological space, but this is not the end of the story, but just the beginning. In calculus, after the limit we have differential, derivative, integral. These things can also be extended to topological spaces, built on the basis of topology-that is, differential geometry. From the teaching, there are two different types of textbooks for differential geometry , one is "classical differential geometry" based on classical microcomputer, which is mainly about the calculation of some geometrical quantities in two and three dimensional space, such as curvature. another is based on modern topology , which is called "Modern differential Geometry"-Its core concept is "manifold" (manifold)-is a topological space based on the addition of a set of possible differential operation of the structure. Modern differential geometry is a very rich subject. For example, the definition of differential on a general manifold is richer than the traditional differential, and I myself have seen three equivalent definitions from different angles-which makes things more complicated, but in another aspect it gives different understandings of the same concept and often leads to different ideas when solving problems. In addition to promoting the concept of calculus, a number of new concepts have been introduced: tangent space, cotangent space, push forward, pull back, fibre bundle, flow, immersion, submersion Wait a minute.
In recent years, manifolds seem to be quite fashionable in machine learning. But, frankly, to understand some of the basic manifold algorithms, even "create" some manifold algorithms, does not require much of the basis of differential geometry. For my research, the most important application of differential geometry is another branch built on it: Lie groups and Lie algebras--a beautiful marriage between two big family analyses and algebra in mathematics. Another important combination of analysis and algebra is the functional analysis, and the harmonic analysis based on it.
Iv. algebra: An abstract world
1. About abstract algebra
Go back and say another big family--algebra.
If classical calculus is an introduction to analysis, then the entry point of modern algebra is two parts: linear algebra (linear algebra) and the underlying abstract algebra (abstraction algebra)-Some of the domestic textbooks are said to be called "Modern algebra." Algebra-The name of the study seems to be the number, in my opinion, the main research is the operational rules. One algebra, in fact, is to abstract some basic rules from some specific computing system, establish an axiom system, and then carry on the research on this basis. A set, coupled with a set of operational rules, constitutes an algebraic structure. In the main algebraic structure, the simplest is the group-it has only one kind of reversible operation that conforms to the binding rate, usually called "multiplication". If this operation is also in accordance with the exchange rate, then it is called the abelian group (Abelian group). If there are two operations, one called addition, satisfies the exchange rate and the binding rate, one called multiplication, satisfies the binding rate, and satisfies the distribution ratio between them, this kind of rich structure is called the ring, if the multiplication on the ring satisfies the exchange rate, it is called the commutative ring (commutative ring). If, the addition and multiplication of a ring has all the good properties, then it becomes a domain (field). Based on the domain, we can construct a new structure, which can make addition and multiplication, and then form the linear algebra (Linear algebra).
The advantage of algebra is that it only cares about the deduction of arithmetic rules, regardless of the object that participates in the Operation . As long as it is properly defined, it is entirely possible for a cat to get a pig:-) by a dog. All theorems based on abstract operation rules can be applied to the above mentioned cat and dog multiplication. Of course, in practical use, we still want to use it to do something meaningful. learned of abstract algebra know that, based on a few of the simplest rules, such as binding laws, you can derive a very significant number of important conclusions that can be applied to everything that satisfies these simple rules-this is the power of algebra, and we no longer need to re-establish so many theorems for each specific area.
abstract algebra on the basis of some basic theorems, further research is often divided into two schools : the study of finite discrete algebraic structures (such as finite groups and finite fields), which is commonly used in number theory, coding and integer equations these places Another genre is the study of continuous algebraic structures , often associated with topology and analysis (e.g., topological groups, Lie group). The focus of my study is mainly the latter.
2. Linear algebra : The fundamental position of "linearity"
For those who do learning, vision, optimization or statistics, there is no more contact than linear algebra-which we begin to learn in the early part of college. Linear algebra, including a variety of disciplines based on it, the core of the two concepts are vector space and linear transformation. The position of linear transformations in linear algebra, the position of continuous functions in the analysis, or the status of Homomorphic mappings in group theory is the same-it is the mapping that maintains the base operation (addition and multiplication).
There is a tendency in learning to despise linear algorithms and to advertise nonlinearity. Perhaps under many circumstances, we need nonlinearity to describe the complex real world, but at any time, linearity is fundamental. Without a linear basis, there is no such thing as a non-linear generalization. The nonlinear methods we commonly use include manifolds and kernelization, both of which need to be in a certain phase of the regression. The manifold needs to be mapped in each local and linear space, and by connecting many local linear spaces to form nonlinearity, the kernerlization is to map the original linear space "nonlinearity" to another linear space by means of the permutation inner product structure, and then perform the operations that can be performed in the linear space. In the field of analysis, the linear operation is ubiquitous, differential, integral, Fourier transform, Laplace transform, and statistical mean, all are linear.
3. Functional Analysis : Moving from finite dimension to infinite dimension
The linear algebra studied in the university, its simplicity mainly because it is in the finite dimensional space, because it is limited, we do not need to rely on too many analytic means. However, the finite dimensional space does not effectively express our world -the most important, the function forms the linear space, but it is infinite dimension. The most important operations performed on the functions are in infinite dimensional space, such as Fourier transform and wavelet analysis. This shows that in order to study the function (or continuous signal), we need to break the constraints of the finite dimensional space and go into the function space of the infinite dimension-the first step in this is functional analysis.
Functional analysis (functional analyses) is a general linear space, including finite and infinite dimensions, but many things appear very trivial under finite dimensions, and real difficulties often arise in infinite dimensions. In functional analysis, the elements in space are still called vectors, but linear transformations are often called " operators " (operator). In addition to addition and multiplication, there are further operations, such as adding a norm to express "the length of the vector" or "the distance of the element", so that the space is called "normed linear Space" (normed space), and further, can be added to the inner product operation, such a space called "inner product Space" (Inner Product space).
It is found that when entering the infinite dimension of time, many old ideas no longer apply, and everything needs to be revisited.
1) All finite dimensional spaces are complete (Cauchy sequence convergence), and many infinite dimensional spaces are incomplete (such as continuous functions on closed intervals). Here, the complete space has a special name: The complete normed space is called the Banach space, and the complete inner product space is called the Hilbert space (Hilbert spaces).
2) in a finite dimensional space, space and its dual space are completely isomorphic, but in the infinite dimensional space, they have subtle differences.
3) in finite dimensional space, all linear transformations (matrices) are bounded transformations, and in infinite dimensions, many operators are unbounded (unbounded), and the most important example is the derivation of functions.
4) in a finite dimension space, all bounded closed sets are tight, such as unit balls. In all infinite dimensional spaces, the unit ball is not tight-that is, you can scatter an infinite number of points within the unit sphere without a limit point.
5) in the finite dimensional space, the spectrum of the linear transformation (matrix) is equivalent to all the eigenvalues, and in the infinite dimensional space, the structure of the operator's spectrum is much more complex than that of the eigenvalues (point spectrum), and approximate points Spectrum and residual spectrum. Although complex, it is also more interesting. Thus, a fairly abundant branch- operator Spectrum theory (Spectrum theory)is formed.
6) in a finite dimension space, there is always a projection of any one subspace, and in an infinite dimension space, this is not necessarily the case, with this good feature of the subspace has a special name Chebyshev space (Chebyshev). This concept is the basis of modern approximation theory (approximation theory). The approximation theory of function space should play a very important role in learning, but there are not many articles on the application of modern approximation theory.
4 . Continue forward: BA-khz algebra, harmonic analysis, Lie algebra
Basic functional analysis go ahead and have two important directions. The first is the Banach algebra (Algebra), which is to introduce multiplication on the basis of the space of the Barnabas (the complete Inner space) (this differs from the multiply). For example, the Matrix--which, besides addition and multiplication, can do multiplication--constitutes a BA-khz algebra. Besides, the bounded operator and the square integrable function of the range are all able to form the BA-khz algebra. The BA-khz algebra is an abstraction of functional analysis, many of the conclusions about bounded operator derivation, and many theorems in operator spectral theory, which are not only applicable to operators, but they can be obtained from the general BA-khz algebra and applied to other places than operators. The Hittite algebra allows you to take a higher view of the knot theory in the functional analysis , but I have to think about how much more it can bring in practical problems than functional analysis.
Another important direction in which functional analysis and practical problems can best be combined is the harmonic analysis (Harmonic). I am here to enumerate its two sub-domains, Fourier analysis and wavelet analysis, which I think has been able to illustrate its practical value. The core problem of the study is how to approximate and construct a function using the base function . It studies the problem of function space and inevitably must be based on functional analysis. In addition to Fourier and wavelet, harmonic analysis also studies some useful function spaces, such as hardy Space,sobolev space, which have many very good properties and are very important applications in engineering and physics. For vision, harmonic analysis is a very useful tool in the expression of signals and in the construction of images.
When analysis and linear algebra come together, functional analysis and harmonic analysis are produced; when the analysis and the group theory go together, we have the Lie group and the Lie algebra (Lie Algebra). They give an algebraic structure to elements on successive groups. I always thought it was a very nice math: In a system, topology, differentiation, and algebra came together. Under certain conditions, through the relation between Lie group and Lie algebra, it makes the combination of geometric transformation into linear operation, and the subgroup into the line subspace space, which creates the necessary conditions for the introduction of many important models and algorithms in learning to the modeling of geometric motion. Therefore, we believe that the Lie groups and the Lie algebra are important to the vision, but learning its path can be very difficult, before it needs to learn a lot of other mathematics.
V. Present probability theory: regenerating on the basis of modern analysis
Finally, let's talk a lot about the branch of mathematics that learning researchers are particularly concerned about: probability theory. Since Kolmogorov introduced the measure into probability theory in the 30 's, the measure theory has become the basis of the modern probability theory . Here, the probability is defined as the measure, the random variable is defined as the measurable function, the conditional random variable is defined as the projection of the measurable function in a function space, and the mean is the integral of the measurable function for the probability measure. It is worth noting that many modern viewpoints begin to look at the basic concept of probability theory in the way of functional analysis, and the stochastic variable forms a vector space, while the symbolic probability measure forms its dual space, in which one party exerts a mean value on the other. The angles are different, but the two approaches are the same, and the basis for the formation is equivalent.
On the basis of modern probability theory, many traditional branches have been greatly enriched, the most representative including martingale theory (martingale)--The theory that is triggered by the study of gambling, is now mainly used in finance (here you can see the theoretical link between gambling and finance,:-P), Brown Movement (Brownian Motion)--the basis of a continuous stochastic process, and a random analysis (Stochastic calculus) based on this, including random integrals (integrating the path of a stochastic process, which is more representative of Ito Integral (Ito Integral)), and stochastic differential equations. The research on the probability model of continuous geometry application and the transformation of distribution is inseparable from the knowledge of these aspects.
Graphic source Network, copyright belongs to the original author all
"ZZ" mit Cow-man Interpretive mathematical system