Mathematics related to machine learning and computer vision

Mathematics related to machine learning and computer vision

One of the math related to machine learning and computer vision

(The following is a space article to be transferred from an MIT bull, which is very practical:)

Dahua

It seems that mathematics is not always enough. These days, in order to solve some of the problems in the library, also held a mathematical textbook. From the university to the present, the classroom and self-study mathematics is not very small, but in the process of research always found that need to add new mathematical knowledge. Learning and Vision are the intersection of many mathematical fields. Looking at the intersection of different theoretical systems, for a researcher, is often very exciting enjoyable things. However, it also means that it is difficult to fully understand the area and make meaningful progress. Remember that in a blog two years ago, the mention of learning-related mathematics. Today, I have a new perspective on the role of mathematics in this field. For the study of learning,
1 , Linear Algebra (linear algebra) and Statistics (statistics) are the most important and indispensable.

This represents the basis of the two most mainstream methods in machine learning. One is an algebraic approach that focuses on research functions and transformations , such as Dimension reduction, feature extraction,kernel, etc., one of which is statistical methods , such as graphical model, information theoretical models , are studied for statistical models and sample distributions. Although they are different, but often common use, for algebraic methods, often need statistical interpretation, for statistical models, its specific calculation requires the help of Algebra . Taking algebra and statistics as a starting point and continuing to go deep, we will find that more mathematics is needed.
2 , Calculus (Calculus), is only the basis of the mathematical analysis system.

Its fundamental role is self-evident. Most of the problems in the learning study are in continuous metric spaces, whether algebraic or statistical, when studying optimization problems, the analysis of a differential or gradient of a map is always unavoidable. In statistics,marginalization and integrals are inseparable-however, it is not uncommon for an analytic form to lead an integral.
3. Partial differential equation (partial differential equation)

This is used primarily to describe dynamic processes , or to mimic dynamic processes . This discipline is used more than learning in vision, and is mainly used to describe the motion or diffusion process of a continuous field . For example , level set, Optical flow is a typical example of this.
4. Functional analysis (functional analyst)

In layman's terms, it can be understood that calculus extends from finite dimensional space to infinite dimensional space --and, of course, it's far more than that. In this place, the dual relationship between the function and the object it acts on plays a very important role. Since the development of learning, it is also extending to infinite dimension--from studying the problem of finite dimension vector to the function of infinite dimension as the research object . Kernel Learning and Gaussian Process are typical examples-the core concepts are Kernel. A lot of people do learning kernel simple understanding of the use of kernel trick, which makes the significance of kernel seriously weakened. Within the functional ,Kernel (Inner Product) is fundamental to building the entire broad algebraic system , from metric, transform to Spectrum is rooted in this.
5 , Measure theory (measure theory)

Lebesgue Measure (Lebesgue measure) , but there are many other measurement systems-the probability itself is a measure . The theory of measure is fundamental to the meaning of learning, and modern statistics is built on the basis of measure theory -although the primary probability textbook is generally not introduced in this way. When you look at some statistical articles, you may find that they will use the statistical formula instead of a measure to express it, there are two advantages: all the derivation and conclusion do not have to separate the continuous distribution and the discrete distribution of each to write again, both of these things can be expressed in the same measure form: The integral of the continuous distribution is based on the Lebesgue measure , the summation of the discrete distribution is based on the count measure , but also can be extended to the kind of discontinuous and non-discrete distribution (this thing is not a mathematician game, but is already in practical things, Often seen in dirchlet process or pitman-yor process). And, even if it is not in the Euclidean space, but in more general topological spaces (such as micro-manifolds or transformation groups), then the traditional Riemann integrals (that is, the kind of college first-year calculus Class) do not work, you may need some of their promotion, such as Haar Measure or lebesgue-stieltjes integral.
6 , Topology (topology)

This is a very basic subject in academia. It generally does not provide methods directly, but many of its concepts and theorems are the cornerstone of other branches of mathematics. When you look at a lot of other math, you will often come into contact with concepts such as Open set/closed set,set Basis,hausdauf, continuous function,metric space, Cauchy Sequence, neighborhood, compactness, connectivity . Many of these may have been studied in the first year of college, when they were based on the concept of limits. If, after reading the topology, the understanding of these concepts will be fundamentally expanded. For example, the continuous function, at that time, is defined by the Epison method, that is, regardless of the number of small positive epsilon, there are xxx, so xxx. This requires a metric to measure distance, in the general topology, for the definition of continuous function coordinates and distances are not required-if a mapping makes the original image of the open set , it is continuous -The open set is defined based on set theory and is not the usual meaning of the open interval . This is just the simplest example. Of course, our study of learning may not need to delve into the axioms behind these mathematical concepts, but the limitations of breaking the concept of the original definition are necessary for many problems--especially when you are studying something that is not in Euclidean space-- orthogonal matrix, transformation Group, manifold, The spatial of the probability distribution belongs to this.
7, differential manifold (micro-manifold)

In layman's words it studies smooth surfaces. A direct impression is whether it can be used to fitting a surface--of course it's an application, but it's very preliminary. In essence , the micro-manifold study is a smooth topological structure . The basic elements of a space-forming micro-manifold are local smoothing: from the perspective of topology , it is the same embryo in the Euclidean space, from the analytic point of view, is a compatible local coordinate system. Of course, in the global, it does not require the same embryo as Euclidean space. In addition to being used to characterize the smooth surfaces on a set, it is more important that it can be used to study many important collections. All K-subspace spaces in a n-dimensional linear space (k
8, Lie Group theory (Lie groups theory)

The general meaning of group theory in the learning is not a lot of use, group theory in the learning used more is its important direction Lie group. A group defined on a smooth manifold, and its group operations are smooth, then this is called the Lie groups. Because learning and coding are different, more attention is given to continuous space, because Lie group is especially important for learning in various groups. Each seed space, linear transformation, non-singular matrix are based on the usual meaning of matrix multiplication to form Lie groups. The mapping, transformation, measurement, division and so on in Lie groups are important for the study of algebraic methods in learning.
9 , graph theory (graph theory)

Figure, due to its strong ability to express various relationships and elegant theory, efficient algorithm, more and more popular in the field of learning. Classical graph theory, one of the most important applications in learning is the graphical models, which is successfully applied to the analysis of statistical network structure and planning statistical inference

and machine learning and computer vision-related math two

Transferred from: http://blog.sina.com.cn/s/blog_6833a4df0100nazk.html

1. Linear algebra (Linear Algebra):

I think the domestic students will learn this course, but not every teacher can carry out its essentials. This discipline is necessary for learning, and it is essential for its thorough mastery. I studied this course when I was in the first year of Hkust, and then I read the linear algebra again after I arrived in Hong Kong, and I read

Introduction to Linear Algebra (3rd Ed.) by Gilbert Strang.

This book is the textbook used by MIT's Linear Algebra class, and is also a classic textbook chosen by many other universities. Its difficulty is moderate, the explanation is clear, it is important to discuss many core concepts more thoroughly. I personally think that learning linear algebra, the most important is not to master the matrix operation and reconciliation of the equation--these in the actual work of Matlab can do, the key is to understand a few basic and important concepts: subspace (subspace), orthogonal (orthogonality) , eigenvalues and eigenvectors (eigenvalues and eigenvectors), and linear transformations (Linear transform). From my point of view, the quality of an online textbook is whether it can give sufficient attention to these fundamental concepts, and whether it can be made clear about their links. Strang's book is doing very well in this respect.

Moreover, this book has a unique advantage. The author of the book Teaches linear algebra classes (18.06) in MIT for a long time, and the course video is available on the MIT Open Courseware website. Friends who have time can watch the video of the teacher's lectures while studying or reviewing the textbook.

Http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/CourseHome/index.htm

2. probabilities and statistics (Probability and Statistics):

There are many introductory textbooks for probability theory and statistics, and I don't have any special recommendations at the moment. What I want to introduce here is a basic textbook on multivariate statistics :

Applied multivariate statistical analysis  (5th Ed.) by Richard A. Johnson and Dean W. Wichern

This book was used for learning when I was in contact with vector statistics, and the basis for my study in Hong Kong was to lay it down. Some students in the lab also borrowed this book to learn vector statistics . This book does not specifically pursue the depth of mathematics, but in an easy-to-understand way to tell the main basic concepts, read very comfortable, the content is very practical. For Linear regression, factor analysis, principal component Analysis (PCA), and canonical component analysis (CCA) This Some of the basic methods of learning have also begun to be discussed.

the Bayesian statistics and graphical modelscan then be further studied. An ideal book is

Introduction to graphical Models (draft version). by M. Jordan and C. Bishop.

I don't know if this book has been published (not to be confused with learning in graphical models, it's a collection of essays, not for beginners). This book from the basic Bayesian statistical model has been deep into the complex statistical network estimation and inference, in simple, statistical learning many important aspects of this book have a clear exposition and detailed explanation. Inside MIT you can access, as well as the outside, it seems to have an electronic version.

3. Analysis :

I think everyone has studied calculus or mathematical analysis in college, and the depth and breadth vary with each school. This field is the foundation of many disciplines, and the recommended textbook is

Principles of mathematical analysis, by Walter Rudin

A bit old, but absolutely classic, deep and thorough. Disadvantage is more difficult-this is Rudin's book's consistent style, suitable for a certain basis to look back.

In analyzing this direction, the next step is functional analysis (functional).

Introductory functional analysis with applications, by Erwin Kreyszig.

Suitable as the basic teaching material of the functional, easy to cut into without losing all-round. I especially like it. Special attention is paid to spectral theory and operator theory , which is particularly important for the study of learning. Rudin also has a book on functional analysis, the book may be more profound in mathematics, but not easy to get started, the content and learning the relevance of the book.

In the analysis of this direction, there is an important subject is the measurement theory (Measure theory), but I have read the book is not yet felt particularly worthy of introduction.

4. topology (topology):

The basic topology books I have read have their own characteristics, but in general, I would most highly recommend:

Topology (2nd Ed.) by James Munkres

This book is the work of Professor Munkres's long-term coaching of the MIT topology class. There is a comprehensive introduction to general topology and a modest discussion of algebraic topologies (algebraic topology) . This book does not require special mathematical knowledge can begin to learn, from the most basic set theory concept (many books disdain to speak this) to Nagata-smirnov theorem and Tychonoff theorem the more deep theorems (many books avoid this) are covered. The narrative way is very strong , for many theorems, in addition to the proof process and guide you to think behind the principle of the context, a lot of amazing highlights-I often read to forget hunger, do not want to addictive. Many of the exercises are quite standard.

5. manifold theory (manifold theory):

For the topology and analysis of certain certainty, you can begin to learn the manifold theory, otherwise the study can only flow in the superficial . The book that I use is

Introduction to Smooth manifolds. by John M. Lee

Although the title has introduction the word, but in fact the book involved in very deep, in addition to teaching the basic manifold, tangent space,bundle, Sub-manifold and so on, also explored such as the outline theory (Category theory), the dram on the homology (De Rham cohomology) and the integral manifold Some of the more advanced topics. There are quite a lot of discussions about Lie groups and Li algebra . It is popular and rigorous, but it needs to be familiar with certain marking methods.

Although Lie groups are built on the concept of smooth manifolds , it is possible to learn the Lie groups and Lie algebras directly from the Matrix-a method that may be more practical for a friend who is in urgent need of solving problems with Lie groups. Moreover, it is beneficial to deepen understanding of a problem from different perspectives. The following book is an example of this direction:

Lie Groups, Lie algebras, and Representations:an elementary Introduction. by Brian C. Hall

The book starts from the matrix and introduces the concept of the matrix Lie groups from the algebraic rather than the geometrical point of view. And the exponential mappingis established by defining the Operation , and the Lie algebra is introduced in this way. This method is more acceptable than the traditional way of defining Lie algebra through the " left invariant vector field (left-invariant vector field)", and it is easier to reveal the meaning of the Lie algebra . Finally, there is a special discussion linking this new definition to the traditional way.

————————————————————————————

Whether it is research vision, learning or other disciplines, mathematics is ultimately the foundation . Learning Maths Well is the cornerstone of good research . the key to learning math is ultimately own efforts , but the choice of a good book is still of great benefit. Different people have different knowledge backgrounds, thinking habits and research direction, so the choice of the book also varies with each person, just to suit themselves, do not have to insist on the same. The above book is only from my personal point of view, my reading experience is very limited, there are likely to be better than their books (may also tell me, first say thank you).

Mathematics related to machine learning and computer vision