The application of factor space theory in big data
Wang Peizhuang
Liaoning University of Engineering and Technology
(Speeches on the theme forum on big Data and data science progress, collated)
China's data and machine intelligence Science workers shoulder the task of leading the big Data era, this is about whether we can successfully achieve the Chinese dream of the event. No matter how difficult, we must strive for the forefront. As a veteran in the field of information revolution, I used to build a theory and wait for this day to undergo a new test, the theory is the factor space.
First, historical contribution of the factor space
July 87, Japanese scholars in the International conference Hall of Fuzzy Systems held in Tokyo, there is a machine, clearly written the words of the fuzzy computer, with a inverted pendulum control for the demonstration. Japan's Asahi Shimbun reported three days in a row, saying it was a computer in the post-five dynasties. In May 88, "Guangming Daily" reported that "Professor Beijing Normal University Wang Peizhuang, a doctoral student in the development of the international second fuzzy Inference machine", speed from 10 million times per second to 15 million deduction, the volume reduced to one-tenth of his. It was a beautiful battle that China had fought in the international information revolution. The direct reason of the victory is that I use the factor space to establish the random set and the fuzzy Falling Shadow Theory, which is far stronger in mathematics than in Japan.
 60 when I was teaching probability, I drew an image of a factor space on the blackboard. If you lose a coin, why can't you determine which side it is facing up to? The reason is that some factors such as ' hand movements ' are difficult to master and control. When the dimension of the factor space can be manipulated is not sufficient, the randomness occurs due to the breaking of the law of causality. Factor space is the bridge between randomness and certainty. They can be transformed from one another to the perspective of how the dimensions of the factor space are controlled. The basic space proposed by Kolmogorov is a factor space, without the thought of factor space, it is impossible to define a random variable as a necessity mapping, only to talk about classical probability and not induce distribution function and distribution density, there is no modernization of probability theory. 70 's I made fuzzy set research, Zadeh only the fuzzy set as a curve in the domain, the domain is regarded as an undefined noun and nobody, I regard the domain as a factor space. Use it to explore the true meaning of fuzziness. Factor space has become a bridge between ambiguity and clarity. I find that there is a special relationship between the two bridges: it is more difficult to describe subjective factors in mathematics than objective factors, to rise to a level, to raise the set theory to its power, that is, the set of sets, the image said, from the ground to mention the sky. I suggest that the fuzziness of the earth can be transformed into the randomness of the heavens. Such subjective measures as membership degree, reliability and so on are not as satisfying as probabilities, and they are all non-additive measures. All need to go around the sky to make an additive measure, and then fall to the non-additive measure. In 85, in the book "Fuzzy set and random set Falling Shadow" published by Beijing Normal University publishing house, I put the three basic mathematical structures of order, topology and measure into power on the basis of the difficult work, set up the general mathematics theory of the subjective measure, occupy the theoretical commanding point of qualitative things quantification, until now it has not been surpassed by foreign countries. The reliability theory of Shefer and the random set of Matheron in my book is only a small amount of space to be a little bit clear. The three super topologies currently in the world are covered in 8 of my hyper-topologies. So, I personally experience the importance of factor space theory.
This victory is actually the money old guidance, the money old on February 13, 1986 wrote a letter to me, said: "The manuscript received the work of mountains and rivers, he is also studying the problem of intelligent machine, also thought that fuzzy reasoning is a way, and the development of components." So we need someone in our country to do the components, is there anyone in your school? Do you have any idea what people are doing with fuzzy components? "It was in fact under the instruction of the old money that we did it." When this thing was done, the money always called me and several graduate students to his office, while watching the video and talking. He said: "The 50 's two bombs is a theory to do the problem, now make intelligent computer than then difficult, the most nerve-racking thing is that there is no real theory." Ai has been engaged for so many years and has not yet developed a real theory. "Money always emphasizes mathematics, because the mathematical preparation of the Industrial revolution is Newton's calculus (and his predecessors), the computer is the first mathematical idea to get out." Ai has been engaged for many years while he speaks, but the mathematical tool that really works is still a statistical method based on probability theory, which cannot last. I understand that money is old and far-sighted, from the money old speech, I will lead graduate students to focus on research factors space. The main results are published in the following three books:
Book 1 introduces the factor space theory, book 2 is the factor space for the neural network, Book 3 introduces the fuzzy set and factor space in the fuzzy inference machine and fuzzy computer development of the comprehensive application. We try to refine the mechanism of fuzzy inference, expecting to add a central processor of fuzzy inference outside the computer's numerical Computing center processor, in order to develop the intelligent computer. Some of the basic smart device mathematical ideas are written, such as the emergence of the corresponding physical and chemical components.
As we focus on the development of the central processor, a new wave has crept in and started the new era of intelligent networks. What do you call a computer when the world's computers are networked? This seems to need to be redefined. In any case, the central processor has been marginalized, and the data software that has been set up and subordinated is dominated by the transmission and operation of information. The KDD and subsequent data mining that emerged in 1989 marked a shift in the focus of machine intelligence from the development of the fifth generation computer to data intelligence. The rapid momentum of the big data wave is the power of the era of intelligent network performance and the Prelude, the core competition in the era of intelligent network is the birth of human-machine cognitive body. A variety of human-computer cognitive bodies will form a self-organizing ecosystem, mastering and affecting all aspects of human life, which is not with people will for the transfer of the grim fact. You can't stop it, you can only induce it. This is the new commanding point of the international scramble for the intelligent machine of money. Because I know the unilateral and complacent mood, not in time to adjust the direction, the delay of the fighter. The heights we have occupied have been lost. Looking back at the earlier two mathematical schools in the international intelligent data, formal concept analysis and rough set, they studied the sample analysis of the factor space. But they don't know what the distribution of the unknown sample is. Since the factor space is the universal frame of information description, and the data is the carrier of information, the factor space can provide the mother theory for data analysis, and become the theoretical foundation of data science. The formal thesis of the factor space and the two mathematical theories that were originally published in the same year in 1982, this is not just a coincidence, history is calling us to reclaim the lost position!
content, meaning and method of factor space
What are the factors? factor is the quality root of things. For example, male, female is a pair of qualitative sex, sex is the qualitative root of both, sex is a factor. Red, yellow, blue, white, black, ... is a group of things, color is their quality root, color is also a factor. Each quality root is a series of quality, so the gene is higher than the quality of things. It sketchy, if a factor with two qualitative, 10 factors to bring out the comprehensive nature is how much? is 2 of the 10-time side. It is not possible to confuse factors with quality. Quality is a property, why use a new word instead of the existing name of the attribute? The reason is that the term "attribute" appears in the database and is translated from the English word ' Attribute '. In English, there are two different usages of this term in foreign languages. Will in the formal concept analysis with attribute, for example, he cinematographer "biological and water" concept extraction, the list of fish and aquatic plants are ' living in the water ', the dog and beans are ' on the Land of Life '. He ranked ' living in the water ' and ' living on land ' as two different attribute. It can be seen that he uses attribute to refer to the qualitative, not the "biological habitat" of the root. In rough concentration, attribute refers not to the quality but to the quality of the root. For example, when talking about classification by attribute, the toy is classified by color, shape and volume, where attribute refers to color, shape and volume, and they are all qualitative roots. These two different usages confuse the boundary between the qualitative and the qualitative root. Our computer industry colleagues have been aware of this confusion situation, emphasis: ' Color ' and ' red ' yellow ' blue ' can not be confused, if the red, yellow, blue and so called attributes, the color can not be called property, and renamed the property name; If the color is called a property, then, red, yellow, blue and so on can not be called attributes, and renamed property values. We would like to take the first term. Because, the factor is the attribute of the property, it is the name of the homogeneous attribute. In this way, we have two kinds of coordinated, unified coexistence, the equivalent of the same term. It is beneficial to the interdisciplinary and development of disciplines.
Genes are the precursors of biology, and each gene has a chain of hooks, each of which specifies a biological attribute value. Mendel first called the Gene the Factor (Mendelian factor), the factor is the generalized gene. Mendel found the gene, found the key to unlock the life, we emphasized the factors, we can find the opening of everything to describe the key. Gene is the cause of cognition and the basic element of the formation and recognition of things.
What is a factor space? factor space is the coordinate frame of the axis based on the factor (attribute name), and anything can be abstracted into a point of the factor space. It is a universal framework for information description.
There is an analysis and synthesis of the factors between the operation, the mathematical formation of a Boolean algebra. The factor space is mathematically defined as a set family of Boolean algebra as a set of indicators satisfying certain axioms.
Cartesian coordinate system can be regarded as a special factor space, but the dimension of these coordinates of factor space can be changed. At any time, we always have to deal with things in as few dimensions as possible. That is, the main and secondary factors are constantly being transformed. The axis of the factor space is not necessarily a European real axis, as is the case with unstructured things. Factor space theory also has the factor Vine, which is embedded. A point in the factor space that can be magnified into a new factor space. If a set of sample points for a factor space is represented in tabular form, this point can be represented by a concept symbol for the new factor space.
The fundamental purpose of the factor space is to provide a universal framework for information description, and to lay a rigorous mathematical foundation for the science of thought. At present, it is to provide a mathematical basis for data science. To build a huge engineering effect on human-machine cognition.
The relationship between factor space and data science ?
In the existing relational database table, the object column is eliminated, and an information system is a set of sample points in the factor space. Factor space is the platform that hosts its mother.
Factor space The general relationship between the Codd and the establishment of the relational database is further defined as a specific relationship of the attribute configuration which reflects the factors, called the background relation. It determines the whole cognitive information contained in the Matrix, and determines the extraction of concepts and inference. Using the factor space to deal with the relational database, the concept extraction and causality reasoning in the two links than the existing methods, the truth is simple, the algorithm is fast. Because the background relationship determines everything, and the background relation is the combination of all the sample relationships, the table with the same header can be used to stitch rows (objects), especially for distribution, time-sharing operations. The bigger the data, the more ways to get there. According to Professor Zongben's definition of big data algorithm, the factor space is very suitable for establishing a kind of big data algorithm (related to knowledge representation).
what is the core of the factor space?
First of all, people's thinking activities in the final analysis is the concept of division. Life out, the world is a chaotic group, called 0 concept, the connotation is empty. With the increase of knowledge, the forewarned of concept is more fine. From the upper concept to the next concept, the extension is smaller, and the connotation is to add some new attribute descriptions after inheriting the connotation of the upper concept. The decomposition process from upper concept to inferior concept is a cognitive unit of human cognition. The concept of separation is inseparable from factors. Each cognitive unit corresponds to a set of factors called unit factors. This group of elements constitutes a factor space. Called the Cognitive unit space. The cognitive unit space of the factor space directly and completely uses mathematics to describe the human cognition unit.
The cognitive unit space of $u$ is recorded as $ (\{x_f\}_{f\in F} in the extension of the upper concept; U) $, here, there is a group of element factors $f_{1},\cdots,f_n$ integrated into factors $f=f_1\vee \cdots \vee f_n$. Each factor $f_i$ is also defined as a mapping $f_i:u\rightarrow x_{f_i}$, where $x_{f_i}$ is the set of all possible attribute values in $u$ objects under factor $f_i$, called the state space of the factor $f_i$. Factor $f$ is also defined as a mapping $f:u\rightarrow f$, where $x_f$ is the set of all possible attribute values in $u$ objects under the synthetic factor $f$, called the synthetic state space. Remember
$R =\{{x}= (x_1,x_2,\cdots,x_n) \in x_f|\exists u \in u;{ X}=f (U) \} (i.e.\ x_1= f_1 (u), \cdots, x_n= f_n (U)) $,
The background space, called the cognitive unit, is also called the background relationship between the elements of $f_{1},\cdots,f_n\}$. Background relationship is a restriction of the attribute configuration between various factors, it requires that each collocation must be actual, that is, in the domain $u$ There is an object $u$ has this kind of configuration.
The background relation determines the division of the inferior concept. In the synthesis factor $f$ a $x_f$ in the state space of the ${a}=a_1\times \cdots \times a_n$ in $u$ a concept of extension $e=\{u\in u| F (U) \in {a}\}$, its connotation can be described in qualitative language as: "Under the $i$ factor has the attribute value $a_i (i=1,\cdots,n) $". Here, $a _i$ are taken as qualitative language values. Such a super-rectangle must be completely $r$ by the background relation, and can not be expanded, and its decomposition is determined entirely by the background relation.
Element factors, we can decompose the comprehensive factor $f$ into two parts: $F =f\vee g$ set $x=x_f$ and $y=x_g$ respectively are the factor $f, g$ state space, we discuss the causal relationship between them.
An extreme situation is that all configurations are not void, $R =x\times y$, in which case the factors $f, g$ are defined as independent, and there is no meaningful causal inference between independent factors. Causal inference occurs in the constraints of the background.
basic theorem : The background relationship R determines the factors $f, g$ All the constant true reasoning sentences .
Note $f, g$ itself can be a complex factor, $X, y$ can be arbitrarily high-dimensional, the theorem is very effective. It raises the status of background relations. Background relation is the extension of the form background, and the form background presented by would becomes the center of our factor library theory. The basic theorem tells us: Mastering the background relationship, we have mastered all the reasoning knowledge between the factors. The core of factor space reasoning is to determine the shape of $r$! A relational database table (also known as an information system) is a mapping sample of the object-to-factor state space. Remove the object list, that is, the sample privacy, you get a background sample. Background relation is the parent of background sample, and the matrix is obtained by superposition of reliable samples. There is a basic proposition that: the background relation R is equal to the set of the sample background and can be spliced on the line (object). This is in line with the requirements of the big data algorithm, can be distributed, time-sharing, parallel operation, the solution can be combined and splicing.
In this way, the status of the data has changed. The former data is only the quicksand that is analyzed, now it becomes our well-cultivated object. We want to keep the overlay of the sample, and when it represents the matrix, all the reasoning knowledge is generated by it. We are not afraid of it making mistakes, where they make mistakes, and where they stack up, it is a step closer to the real mother. A mature sample, that is, no longer or rarely make mistakes, is very close to the matrix.
From the point of view of pure set theory, the distribution of the background relation is different from the common distribution in probability theory. The borderline is heavier than the density, and the height of the background is determined by the odd man like Yao, who doesn't care how high most people are. Rare objects cannot be overlooked. This also conforms to the feature of big data getting rid of density limitations.
The background relationship is usually convex, both 2.3 meters and 2.1 meters tall people, there are 2.2 meters tall people. Even if it is not, it may be in the future. Under the premise that the mother $r$ has convexity, $R $ can be determined by its vertices, so that all non-vertices can be compressed, which is the core theory and technology of big data processing. Therefore, the background relation R is replaced by the background base $b$. The background base guarantees the storage simplicity of the mature cognition unit. All the reasoning knowledge of a cognitive unit can be compressed into a small numerical matrix.
What's the problem with the factor space study?
The main content of cognitive unit space is:
Secondly, in order to expand cognitive unit into cognitive space, the study of factor space must include the following tasks:
4. Research factors The embedding structure of rattan, forming factors neural network, so that each cognitive unit can connect with each other, to form a cognitive system, large system, large system, to realize the construction of human-machine cognitive body of the grand project. In this integration process, the first three tasks should be put forward and fulfilled, that is, the concept of cross-unit, judgment and reasoning.
The above four is the factor space for the data science must put forward the content. The database referred to by the factor space is called the Factor Library.
Thinking process is different from other material movement process, it has its own characteristics, in order to reflect these characteristics, but also include the following tasks:
5. Factors of weight and factors highlighting the problem
Human brain activity is characterized by factors driven, encountered a problem, how to do? The first is to catch the factor. The contradiction of several major factors cross, people to balance among the factors, it is necessary to weigh the pros and cons, from the weight of light, the formation of weights. There are fixed weights, more variable weights. The weight of the factors becomes a special research topic.
The weight of the factors can be seen as a linear or nonlinear transformation of factors. The practice of pattern recognition often requires such a transformation of factors in order to reveal the factors of real classification. This is called feature extraction. Feature is also a factor, and feature extraction is a process of factor implicit. Factor highlighting is an important topic worthy of study.
6. Potential and field of factor space
Potential optimization is the motive force for the generation and development of things, and also the motivation of brain thinking. There are potential and optimization problems in the space of factors. Solving a linear programming problem is one step to find the optimal solution and ignore the intermediate optimization path and step. In the optimization of factor potential, the final solution is relatively slim and secondary, the first step in front of how to go? What is the path to optimization? is a more realistic and more focused issue.
7. Temporal and spatial factors and dynamic concepts
The temporal and spatial movement of physical molecules is a factor space. For information processing and thinking process, time is the most basic factor that can not be divorced. All things are in motion, the ability of human cognition is not in the still life and in the dynamic recognition. The dynamic curves with time participation are inseparable from the transformation of frequency domain. More math tools to use.
8. Topological structure of the factor space
People have image thinking, not only artists, mathematicians also have, they think that in the study of mathematics, thinking in the image than logical reasoning is also important. What exactly is this image? Psychologists have a preliminary explanation. Professor Ouyang that the factor space should introduce some kind of topological structure to capture the image of thinking. He uses algebraic topology to provide a deep insight into the space of factors.
9. Uncertainty of the factor library
Is the data in the factor library uncertain? This depends on the source and purpose of the data. 1. If used directly in the interpretation. And the judgments made from data to meaning are deterministic, for example, a man with a bearded beard, without suspense, is called definitive interpretation. The problem of concept extraction in most quantitative subjects is mostly deterministic. 2. If used directly in the interpretation. And the judgment from the data to the meaning of the conversion is uncertain, such as the data is ' 27 years old, character aging ' to the ' youth ' or ' middle-aged ' interpretation, then it is not good to judge, with uncertainty, this interpretation is called uncertainty interpretation. The uncertainty when the concept is divided should be attributed to fuzziness. Most of the cognitive units in the qualitative disciplines encountered, many are vague interpretation. Fuzziness is a major feature of human brain thinking, and the use of fuzziness can efficiently transmit information. For the sake of simplicity, we now use classical set theory to establish the theory of factor space. Next, we should use the fuzzy set theory to popularize, from the background related to the concept, the rule extraction all need to blur, in order to adapt to the ambiguity explanation the need. 3. Indirect use in interpretation, acquisition is random, this data is called random data. The probability statistic method must be utilized. It must be admitted that the algorithms which are effective in pattern recognition and classification are still based on the probability statistic method. This is because the cognitive unit recognition method has not really started, the concept of the attribute description is not used, on the fault in isolation rely on a set of variables (not necessarily a group of elements), so that the definitive interpretation is missing or rare. However, even if the cognitive recognition method starts, it still has to use a lot of random data, using probability statistics. The processing of fuzziness can also be transformed into a random set, in the final analysis, also involves the probability statistic method. Therefore, the probability statistic method is still the important pillar of the factor library.
10. Assimilation of data (a unified framework for heterogeneous data processing)
According to Professor Zongben, this is a difficult point in big data processing. Factor space can be a unified framework for heterogeneous data processing. It is not only a framework for describing everything, but it can also accommodate heterogeneous data such as pictures, audio and text. The treatment of them is focused on the word ' interpretation '. According to the requirements of the task, starting from the rough concept, first make a large division, gradually refinement. Encounter obstacles, you can also use the factor Vine, the heterogeneous data symbolized, to it stand a file (sub-factor space) hanging up. Called at any time. Before people have yet to find a unified framework for heterogeneous data, it may be useful to use factor spaces for such invocation and processing.
Two things need to be done to build a huge project for human-machine cognition:
11. Glossary of compilation Factors
Human-machine cognitive body is a coupling of human-machine cognition unit. The key to coupling is to have a dictionary of factors. The dictionary is compiled in the following form:
On the concept name $\rightarrow$ unit factor name $\{f_1,\cdots,f_k; f_{k+1},\cdots,f_n\}$
Among them, $f _1,\cdots,f_k$ is the main factor, $f _{k+1},\cdots,f_n$ is a secondary factor, all sorted by the importance of extracting concept. They will be used as a table header for the factor library.
Factor dictionaries also face the common problems of lexicography. How to achieve clear categories, reduce duplication and avoid conflicts is a problem that needs to be studied.
Another difficulty is that, in the same concept, in different times, places and different groups of the concept of the division if there is a variation, the factor Dictionary of the concept of the description is too rough. At this point, the header needs to be added. When to add? How to add? It's all a matter of trouble.
12. Establishing the Factor library language
The function of human-machine cognitive body is to output intelligent information. Every kind of intelligent information is a question to answer. The form of these questions is nothing more than: what is this? Why is that so? What will be the consequences of this happening? What should I do if I encounter this problem? What is the key? How to control a variable? And so on, they all use concepts, make judgments and inferences, and other thought processes based on judgment and reasoning. The human-machine cognition system requires the direct use of natural language to ask questions. We want to build the language, the first is user-friendly, can grasp, the second is inclusive. This language is not exclusive to any other language and is compatible with it. This is the most convenient language. In this respect, category theory may provide an important tool for translation between different languages.
Third, recent progress in the work
Based on the Institute of Intelligent Science and mathematics, Liaoning University of Engineering and Technology, with the support of the Chinese Academy of Sciences Virtual Economics and Data Science Research Center, we jointly applied for a National Natural Science Fund Director Fund project to further develop the application of factor space in data science. Its progress is as follows:
Four, the construction of human-machine cognition body
What is a human-machine cognitive body? Human-machine cognition is a hardware and software system which is capable of monitoring, organizing, managing and controlling the system with certain purpose, cognitive function, receiving network information and being involved.
Unmanned aerial vehicle (UAV) is a kind of human-machine cognition body, which is a software-driven flying machine to avoid the casualties of pilots. It has a cognitive function to identify ground targets and character features, its flight plan to accept network information adjustment, its combat process needs people's cooperation, it is hardware, but the driving software is the soul.
The supermarket cash register is not a human-machine cognitive body, because it will only collect money and record transactions, there is no cognitive function. But if the function of the cash register is enlarged, the information of printing is increased, and then several basic algorithms of the factor space are put in, the concepts and causal inference rules related to the tight goods and customer fashion are automatically extracted, and then by the Sales Manager or expert to read and control the knowledge, analyze the market factors artificially, combine the network information , improve the operation, serve the people, it becomes a human-machine cognitive body. The supermarket is so, the other is not so? Now, big businessmen in the United States, big bankers are already in the calculation of operating machines, may not have risen to the height of human-machine cognitive body. But the practice is more important than the idea, we must not wait and see, Miss!
Drones are an extreme example of how highly automated they are, with ready-made identification and control techniques already almost enough. The majority of cognitive bodies are less automated, and more are needed to intelligently describe and apply factor spaces. For example, community management, many communities do not have any hardware equipment. This is the most need to establish a recreational community of human-machine service system. The most need to do now is the hardware facilities. Like a supermarket first to have a cash register, first of all, the community's medical, housing, water and electricity, kindergartens, schools, nursing homes, environmental health, cultural and entertainment, neighborhood relations, security and other aspects of the information network set up and linked. Even if there is no intelligence, do not underestimate, with this system, you can use the theory and methods of factor space to expand into the various cognitive units, and then by the cognitive unit coupled to adult cognitive body. Community cadres and residents are human-machine cognition of the builders and participants, factor space is not only a mathematical, but also a methodology, introduced to everyone, encountered problems, on the factors to find the reason, find the way, grasp the main factors and the transformation between factors.
There will be millions of human-machine cognitive bodies. According to the industry, there are various industries of human-machine cognition machine. Target-optimized (e.g., development systems) and factor-balanced (e.g., safety systems) by functional morphology. No matter how it is divided, there are several general rules:
Finally, it should be emphasized that human-machine cognition is self-organized ecosystem. And looking to the future, thousands of human-machine cognitive bodies are about to rapidly emerge, penetrating and affecting all aspects of human life. The world's major powers will struggle for the development of human-machine cognition. This is an objective reality that is not to be shifted by people's will. Can not avoid avoiding, only actively create. If we want to realize our dream of a powerful country, we must focus on the strength of the forces, in the state departments from top to bottom organization and leadership, all walks of life together, from a small cognitive unit to start, from the bottom up to develop a human-machine cognitive system of the great project.
, &NB Sp , &NB Sp , &NB Sp , &NB Sp , &NB Sp , &NB Sp January 8, 2015
The application of the
Factor space theory in big Data--Wang Peizhuang
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
