Mathematics related to machine learning and Computer Vision

(The following is a space article written from an MIT cow. It is very practical :)

Author: Dahua

It seems that mathematics is always not enough. These days, in order to solve some problems in research, we held a textbook on mathematics in the library. From the university to the present, the number of Mathematics Courses in the classroom and the number of self-taught mathematics courses is not very small. However, during the study, we always find that new mathematical knowledge needs to be supplemented.Learning and visionIt is the intersection of many kinds of mathematics. Looking at the convergence of different theoretical systems, for a researcher, it is often a very exciting enjoyable thing. However, this also indicates that it is very difficult to fully understand this field and make meaningful progress. I remember that I mentioned learning-related mathematics in my blog two years ago. Today, I think I have taken a new look at the role of mathematics in this field. For the study of learning,
1,Linear Algebra (linear algebra) and Statistics (Statistics) are the most important and indispensable.This represents the basis of the two most popular methods in machine learning. One is an algebraic method that focuses on functions and transformations, such as dimension functions, feature extraction, and kernel. The other is a statistical method that focuses on the statistical model and sample distribution, for example, graphical model and information theoretical models. Although they are different, they are often used together. For algebraic Methods, statistical interpretation is often required. For statistical models, the specific calculation requires help from algebra. Starting from algebra and statistics, we will find that more mathematics is needed.
2,Calculus (calculus) is only the basis of the mathematical analysis system.Its fundamental role is self-evident. Most of the problems in learning research are carried out in a continuous measurement space. Regardless of algebra or statistics, the analysis of the differential or gradient of A ing is always inevitable when studying optimization problems. In statistics, marginalization and integration are even more inseparable-however, it is rare to export points in the form of analysis.
3,Partial Differential Equation (partial differential equation ),This is mainly used to describe a dynamic process or simulate a dynamic process. This discipline is used more than learning in vision and is mainly used to describe the motion or diffusion process of a continuous field. For example, Level Set and optical flow are typical examples.
4,Functional Analysis (Functional Analysis ),In general, it can be understood as the expansion of calculus from finite dimension space to infinite dimension space-of course, it is actually far more than that. In this place, functions and the dual relationships between the objects they act on play a very important role. So far, learning has been extending to the infinite dimension, from studying the finite dimension vector problem to taking the infinite dimension function as the research object. Kernel learning and Gaussian process are typical examples. The core concepts are kernel. Many people who do learning simply understand kernel as the use of kernel trick, which seriously weakens the significance of kernel. In the function, kernel (inner product) is the foundation for establishing the entire broad algebra system, from metric, transform to spectrum.
5,Measure theory (measure theory ),This is a subject with close relationships with actual analysis. However, measurement theory is not limited to this. In a sense, real analysis can deduce from Lebesgue measure (leberger measure). However, there are many other measurement systems-probability itself is a measure. Measure theory is fundamental to learning, and modern statistics is based on measure theory. Although Early Probability Theory textbooks are generally not introduced in this way. When you read some statistical articles, you may find that they will change the statistical formula to a measurement expression. There are two advantages to doing so: all the derivation and conclusions do not need to be written for the continuous distribution and discrete distribution respectively. Both of them can be expressed in the same measurement form: the continuous distribution points are expressed based on the Lebesgue measurement, the sum of discrete distribution is based on the count measure, and can also be extended to the non-discrete and non-discrete distribution (this is not a mathematician's game, but something that is already in use, it is often seen in dirchlet process or Pitman-Yor process ). In addition, even continuous points, if they are not carried out in the Euclidean space, but in a more general topological space (such as a differential manifold or a variant group ), therefore, the traditional riman points (that is, the first-year calculus course in a university) will not work, and you may need some promotion, such as Haar measure or Lebesgue-Stieltjes points.
6,Topology ),This is a basic academic discipline. It generally does not directly provide methods, but many of its concepts and theorems are the cornerstone of other mathematical branches. When reading a lot of other mathematics, you will often come into use with the following concepts: open set/closed set, Set basis, hausdauf, continuous function, metric space, kernel sequence, neighborhood, compactness, connectivity. Many of these may have been studied in the first year of college. At that time, they were obtained based on the concept of limit. If we have read topology, our understanding of these concepts will be fundamentally expanded. For example, the continuous function was defined by the epison method at the time, that is, no matter how small the positive number Epsilon is, there is XXX, making XXX. This requires metric to measure the distance. In general topology, continuous functions are not required to define the coordinates and distance. If a ing is used to make the open set as an open set, it is continuous. As for the open set, it is defined based on the set theory, not the meaning of the general open interval. This is just the simplest example. Of course, we do not need to study the system of justice behind these mathematical concepts in learning. However, breaking the limitations of the original definition is necessary on many issues-especially when you study something that is not in the Euclidean space-orthogonal matrix, transformation group, manifold, the space of probability distribution belongs to this.
7,Differential manifold (differential manifold ),In general, it studies smooth surfaces. A direct impression is whether it can be used to fitting a surface or something-of course, this is an application, but it is very preliminary. Essentially, a differential manifold studies a smooth topology. The basic element of a space that forms a differential manifold is local smoothing: From the Perspective of topology, it means that any part of it is in the same space as the Euclidean space. From the perspective of analysis, is a compatible local coordinate system. Of course, globally, it should not sum the Euclidean space with the same embryo. In addition to describing the smooth surface on a set, it is more important that it can be used to study many important sets. All K-dimensional subspaces of an n-dimensional linear space (K
8,Lie group theory (Li Qun theory ),In general, the group theory is not used much in learning, and the group theory is used mostly in learning, which is an important direction of Lie group. If the group is defined as a smooth and popular group and its operation is smooth, this is called Li Qun. Because learning and encoding are different, more attention is given to the continuous space, because Lie groups are particularly important to learning in various groups. All kinds of sub-spaces, linear transformations, and non-singular matrices are constructed based on matrix multiplication in the general sense. Ing, transformation, measurement, and division in Li Qun all have important guiding significance for the study of algebraic Methods in learning.
9,Graph Theory (graph theory ),Graph, because of its powerful ability to express various relationships, elegant theories, and efficient algorithms, it is increasingly popular in the learning field. Classical graph theory, one of the most important applications in learning is graphical models, which is successfully applied to analyzing the structure of the statistical network and planning the statistical inference process. Graph Theory is indispensable to the success of the graphical model. In vision, maxflow (graphcut) algorithms are widely used in image segmentation, and stereo also has various energy optimizations. Another important graph theory branch is algebraic Graph Theory (Algebraic graph theory), which is mainly used for graph spectrum analysis. The famous applications include Normalized Cut and spectral clustering. In recent years, semi-supervised learning has received special attention.