[Turn] machine learning and computer vision----mathematical basis

Http://blog.sina.com.cn/s/blog_6b99cdb50101ix0l.html

One of the math related to machine learning and computer vision

(The following is a space article to be transferred from an MIT bull, which is very practical:)

Dahua

It seems that mathematics is not always enough. These days, in order to solve some of the problems in the library, also held a mathematical textbook. From the university to the present, the classroom and self-study mathematics is not very small, but in the process of research always found that need to add new mathematical knowledge. Learning and Vision are the intersection of many mathematical fields. Looking at the intersection of different theoretical systems, for a researcher, is often very exciting enjoyable things. However, it also means that it is difficult to fully understand the area and make meaningful progress. Remember that in a blog two years ago, the mention of learning-related mathematics. Today, I have a new perspective on the role of mathematics in this field. For the study of learning,
1 , Linear Algebra (linear algebra) and Statistics (statistics) are the most important and indispensable.

This represents the basis of the two main categories of methods in Machinelearning. One is an algebraic approach that focuses on research functions and transformations , such as dimensionreduction, Featureextraction,kernel, etc. statistical methods, such as graphical model, Informationtheoreticalmodels and so on, are used to study statistical models and sample distributions. Although they are different, but often common use, for algebraic methods, often need statistical interpretation, for statistical models, its specific calculation requires the help of Algebra . Taking algebra and statistics as a starting point and continuing to go deep, we will find that more mathematics is needed.
2 , Calculus (Calculus), is only the basis of the mathematical analysis system.

Its fundamental role is self-evident. Most of the problems in the learning study are in continuous metric spaces, whether algebraic or statistical, when studying optimization problems, the analysis of a differential or gradient of a map is always unavoidable. In statistics,marginalization and integrals are inseparable-however, it is not uncommon for an analytic form to lead an integral.
3. Partial differentialequation (partial differential equation)

This is used primarily to describe dynamic processes , or to mimic dynamic processes . This discipline is used more than learning in vision, and is mainly used to describe the motion or diffusion process of a continuous field . For example , Levelset, Optical flow is a typical example of this.
4. Functional analysis (functional analyst)

In layman's terms, it can be understood that calculus extends from finite dimensional space to infinite dimensional space --and, of course, it's far more than that. In this place, the dual relationship between the function and the object it acts on plays a very important role. Since the development of learning, it is also extending to infinite dimension--from studying the problem of finite dimension vector to the function of infinite dimension as the research object . Kernellearning and Gaussian Process are typical examples-the core concepts are kernel. A lot of people do learning kernel simple understanding of the use of Kerneltrick, which makes the significance of kernel seriously weakened. Within the functional ,Kernel (innerproduct) is fundamental to building the entire broad algebraic system , from metric,transform to Spectrum is rooted in this.
5 , Measure theory (measure theory)

lebesguemeasure (Lebesgue measure) , but there are many other measurement systems--the probability itself is a measure . The theory of measure is fundamental to the meaning of learning, and modern statistics is built on the basis of measure theory -although the primary probability textbook is generally not introduced in this way. When you look at some statistical articles, you may find that they will use the statistical formula instead of a measure to express it, there are two advantages: all the derivation and conclusion do not have to separate the continuous distribution and the discrete distribution of each to write again, both of these things can be expressed in the same measure form: The integral of the continuous distribution is based on the Lebesgue measure , the summation of the discrete distribution is based on the count measure , but also can be extended to the kind of discontinuous and non-discrete distribution (this thing is not a mathematician game, but is already in practical things, It is often seen in dirchletprocess or pitman-yorprocess). And, even if it is not in the Euclidean space, but in more general topological spaces (such as micro-manifolds or transformation groups), then the traditional Riemann integrals (that is, the kind of college first-year calculus Class) do not work, you may need some of their promotion, such as Haarmeasure or lebesgue-stieltjes integral 6 , Topology (topology)

openset/closed Set,setbasis,hausdauf, continuous function,metric space, Cauchysequence, neighborhood, Compactness,connectivity . Many of these may have been studied in the first year of college, when they were based on the concept of limits. If, after reading the topology, the understanding of these concepts will be fundamentally expanded. For example, the continuous function, at that time, is defined by the Epison method, that is, regardless of the number of small positive epsilon, there are xxx, so xxx. This is the need for a metric to measure distance, in generaltopology, for the definition of continuous function coordinates and distances are not required-if a map to make the original image of the open set , it is continuous -The open set is defined based on set theory and is not the usual meaning of the open interval . This is just the simplest example. Of course, our study of learning may not need to delve into the axioms behind these mathematical concepts, but the limitations of breaking the concept of the original definition are necessary for many problems--especially when you are studying something that is not in Euclidean space-- orthogonal matrix, transformation Group, manifold, The spatial 7, differentialmanifold (micro-manifold)

In layman's words it studies smooth surfaces. A direct impression is whether it can be used to fitting a surface--of course it's an application, but it's very preliminary. In essence , the micro-manifold study is a smooth topological structure . The basic elements of a space-forming micro-manifold are local smoothing: from the perspective of topology , it is the same embryo in the Euclidean space, from the analytic point of view, is a compatible local coordinate system. Of course, in the global, it does not require the same embryo as Euclidean space. In addition to being used to characterize the smooth surfaces on a set, it is more important that it can be used to study many important collections. All K-subspace spaces in a n-dimensional linear space (k
8, Lie Group theory (Lie groups theory)

The general meaning of group theory in the learning is not a lot of use, group theory in the learning used more is its important direction Liegroup. A group defined on a smooth manifold, and its group operations are smooth, then this is called the Lie groups. Because learning and coding are different, more attention is given to continuous space, because Liegroup is particularly important for learning in a variety of groups. Each seed space, linear transformation, non-singular matrix are based on the usual meaning of matrix multiplication to form Lie groups. The mapping, transformation, measurement, division and so on in Lie groups are important for the study of algebraic methods in learning.
9 , graph theory (graph theory)

Figure, due to its strong ability to express various relationships and elegant theory, efficient algorithm, more and more popular in the field of learning. Classical graph theory, one of the most important applications in learning is Graphicalmodels, which has been successfully applied to the analysis of statistical network structure and planning statistical inference

and machine learning and computer vision-related math two

Transferred from: http://blog.sina.com.cn/s/blog_6833a4df0100nazk.html

1. Linear algebra (Linear Algebra):

I think the domestic students will learn this course, but not every teacher can carry out its essentials. This discipline is necessary for learning, and it is essential for its thorough mastery. I studied this course when I was in the first year of Hkust, and then I read the linear algebra again after I arrived in Hong Kong, and I read

Introduction to Linear Algebra (3rdEd.) by Gilbertstrang.

This book is the textbook used by MIT's Linear Algebra class, and is also a classic textbook chosen by many other universities. Its difficulty is moderate, the explanation is clear, it is important to discuss many core concepts more thoroughly. I personally think that learning linear algebra, the most important is not to master the matrix operation and reconciliation of the equation--these in the actual work of Matlab can do, the key is to understand a few basic and important concepts: subspace (subspace), orthogonal (orthogonality) , eigenvalues and eigenvectors (Eigenvaluesand eigenvectors), and linear transformations (Linear transform). From my point of view, the quality of an online textbook is whether it can give sufficient attention to these fundamental concepts, and whether it can be made clear about their links. Strang's book is doing very well in this respect.

Moreover, this book has a unique advantage. The author of the book has long taught linear algebra (18.06) at MIT, and the course video is available on the MIT OpenCourseWare website. Friends who have time can watch the video of the teacher's lectures while studying or reviewing the textbook.

Http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/CourseHome/index.htm

2. probabilities and statistics (Probabilityand Statistics):

There are many introductory textbooks for probability theory and statistics, and I don't have any special recommendations at the moment. What I want to introduce here is a basic textbook on multivariate statistics :

Applied multivariatestatistical Analysis (5thEd.) by Richard A. Johnson Anddean W. Wichern

This book was used for learning when I was in contact with vector statistics, and the basis for my study in Hong Kong was to lay it down. Some students in the lab also borrowed this book to learn vector statistics . This book does not specifically pursue the depth of mathematics, but in an easy-to-understand way to tell the main basic concepts, read very comfortable, the content is very practical. For linearregression, factor analysis, principal component Analysis (PCA), and canonical component analysis (CCA) The basic methods of these learning have also been discussed preliminarily.

Bayesian Statistics and Graphicalmodelscan then be further studied. An ideal book is

Introduction to Graphical Models (draftversion). by M. Jordan and C.bishop.

I don't know if this book has been published (not to be confused with learning in Graphicalmodels, it's a collection of essays, not for beginners). This book from the basic Bayesian statistical model has been deep into the complex statistical network estimation and inference, in simple, statisticallearning many important aspects of this book have a clear exposition and detailed explanation. Inside MIT you can access, as well as the outside, it seems to have an electronic version.

3. Analysis :

I think everyone has studied calculus or mathematical analysis in college, and the depth and breadth vary with each school. This field is the foundation of many disciplines, and the recommended textbook is

Principles of mathematical analysis, by Walterrudin

A bit old, but absolutely classic, deep and thorough. Disadvantage is more difficult-this is Rudin's book's consistent style, suitable for a certain basis to look back.

In analyzing this direction, the next step is functional analysis (functionalanalysis).

Introductory functional analysis with applications, Byerwin Kreyszig.

Suitable as the basic teaching material of the functional, easy to cut into without losing all-round. I especially like it. Special attention is paid to spectral theory and operator theory , which is particularly important for the study of learning. Rudin also has a book about functionalanalysis, the book may be more profound in mathematics, but not easy to get started, the content and learning the relevance of the book.

In the analysis of this direction, there is also an important subject is the measurement theory (measuretheory), but I have read the book is not yet felt particularly worth introducing.

4. topology (topology):

The basic topology books I have read have their own characteristics, but in general, I would most highly recommend:

Topology (2nd Ed.) Byjames Munkres

This book is the work of Professor Munkres's long-term coaching of the MIT topology class. For general topology (Generaltopology) There is a comprehensive introduction, and the Algebraic Topology (Algebraictopology) also has a modest discussion. This book does not require special mathematical knowledge can begin to learn, from the basic concept of Set theory (many books disdain to speak this) to Nagata-smirnovtheorem and Tychonofftheorem the more deep theorems (many books avoid this) are covered. The narrative way is very strong , for many theorems, in addition to the proof process and guide you to think behind the principle of the context, a lot of amazing highlights-I often read to forget hunger, do not want to addictive. Many of the exercises are quite standard.

5. manifold theory (manifoldtheory):

For the topology and analysis of certain certainty, you can begin to learn the manifold theory, otherwise the study can only flow in the superficial . The book that I use is

Introduction to Smoothmanifolds. by John M.lee

Although the title has introduction the word, but in fact the book involved in very deep, in addition to teaching the basic manifold, tangentspace,bundles, Sub-manifold and so on, also discussed such as the theory (categorytheory), the dram on the homology (Derham cohomology) and the integration manifold Some of the more advanced topics. There are quite a lot of discussions about Lie groups and Li algebra . It is popular and rigorous, but it needs to be familiar with certain marking methods.

Although Lie groups are built on the concept of smooth manifolds , it is possible to learn the Lie groups and Lie algebras directly from the Matrix-a method that may be more practical for a friend who is in urgent need of solving problems with Lie groups. Moreover, it is beneficial to deepen understanding of a problem from different perspectives. The following book is an example of this direction:

Lie Groups, Lie algebras, and Representations:anelementary Introduction. Bybrian C. Hall

The book starts from the matrix and introduces the concept of the matrix Lie groups from the algebraic rather than the geometrical point of view. The exponentialmappingis established by defining the Operation , and the Lie algebra is introduced in this way. This method is more acceptable than the traditional way of defining Lie algebra through " Left invariant vector field (Left-invariantvectorfield)", and it is easier to reveal the meaning of Lie algebra . Finally, there is a special discussion linking this new definition to the traditional way.

————————————————————————————

Whether it is research vision,learning or other disciplines, mathematics is ultimately the foundation . Learning Maths Well is the cornerstone of good research . the key to learning math is ultimately own efforts , but the choice of a good book is still of great benefit. Different people have different knowledge backgrounds, thinking habits and research direction, so the choice of the book also varies with each person, just to suit themselves, do not have to insist on the same. The above book is only from my personal point of view, my reading experience is very limited, there are likely to be better than their books (may also tell me, first say thank you).

[Turn] machine learning and computer vision----mathematical basis