Database Data Mining Overview

With the continuous development of database technology and the wide application of database management systems, the amount of data stored in the database increases dramatically, hiding behind a large amount of data
More important information, if extracted from the database, will create a lot of potential profits for the company, and this kind of information mining from massive databases
Is called data mining.

Data mining tools can predict future trends and behaviors, so as to better support people's decision-making. For example
Analysis: data mining tools can answer similar questions, such as "which customer is most likely to respond to our company's email promotions, why. Yes
Some data mining tools can also solve some traditional problems that consume a lot of time, because they can quickly browse the entire database and find some experts are not easy
Extremely useful information.

The following describes the basic techniques of data mining.

Data Mining Basics

Data mining technology is the result of long-term research and development of database technology. At first, all kinds of commercial data were stored in the computer database. However
Then, the database can be queried and accessed, and then the database can be traversed in real time. Data mining makes database technology more advanced.
It can not only query and traverse the past data, but also identify the potential links between the past data, so as to promote the transfer of information. Data Mining
Mining technology can be put into use immediately in commercial applications, because the three basic technologies supporting this technology have developed and matured. They are:

Massive Data Collection
Powerful multi-processor computer
Data MiningAlgorithm

Commercial databases are growing at an unprecedented speed, and data warehouses are widely used in various industries.
The requirements can also be met by the mature parallel multi-processor technology. In addition, after more than 10 years of development, data mining algorithms have become
Mature, stable, easy to understand, and operate technology.

From commercial data to commercial information, every step is based on the previous step. See the following table. We can see in the table, fourth
Step-by-step is revolutionary, because from the user's perspective, the database technology at this stage can quickly answer many commercial questions.

Product characteristics of technical product manufacturers supporting commercial problems in the evolutionary stage
Data collection
(1960s) "What is my total income in the past five years ?" Computers, tapes, and disks IBM, CDC provides historic, static data information
Data Access
(1980s) "What is the sales volume of the New England division in last March ?" Relational Database Service (RDBMS), Structured Query Language (SQL), ODBC
Oracle, Sybase, Informix, IBM, and Microsoft provide historical and dynamic data information at the record level
Data warehouse; Decision Support
(1990s) "What is the sales volume of the New England division in last March? What conclusions does Boston draw from this ?" OLAP and multidimensional
Database, data warehouse pilot, comshare, Arbor, Cognos, and microstrategy provide backtracking and dynamic data information at various levels
Data Mining
(Popular) "What will happen to the sales next month in Boston? Why ?" Advanced algorithms, multi-processor computers, massive database pilot,
Lockheed, IBM, SGI, and other startups provide predictive information

Table 1 Evolution of Data Mining.

The core module technology of data mining has undergone decades of development, including mathematical statistics, artificial intelligence, and machine learning. Today, these mature technologies,
Coupled with high-performance relational database engines and extensive data integration, data mining technology has entered a practical stage in the current data warehouse environment.

Scope of Data Mining

The name "Data Mining" comes from a bit similar to mining valuable mineral deposits in mountains. In commercial applications, it is represented in large databases.
Search for valuable business information. Both processes require detailed filtering of massive materials and intelligent and accurate identification of potential values.
In. For databases of a given size, data mining technology can use the following super powers to generate huge business opportunities:

Automatic trend prediction. Data mining can automatically search for potential prediction information in large databases. Traditionally, many experts are required to analyze the problem.
You can quickly and directly find the answer from the data center. A typical example of using data mining for prediction is target marketing. Data mining tools can be rooted in
Find out the customers most likely to respond to future email sales based on a large amount of data from past email sales.

Automatically detects undiscovered modes. Data mining tools scan the entire database and identify hidden patterns, for example, by analyzing retail data
In fact, there are many situations where products that seem unrelated are sold together.

Data mining technology can make existing software and hardware more automated and can be executed on upgraded or newly developed platforms. When data mining tools run
When working on a high-performance parallel processing system, it can analyze a super large database within minutes. This faster processing speed means more users
To make the analysis results more accurate and reliable, and easy to understand.

The database can expand its depth and breadth.

In depth, more columns are allowed. In the past, when conducting complex data analysis, experts were limited to time factors and had to perform calculations on the variables involved.
The number is limited, but variables that are discarded and not involved in the operation may contain some unknown useful information. Now, high-performance data mining
The data mining tool allows you to deeply edit the database, and take into account any variables that may be used in the database. You do not need to select a subset of the variables.
Operation.

In terms of breadth, more rows are allowed. Larger samples reduce the probability of errors and changes, so that users can more accurately export small
But it is an important conclusion.

Recently, a Gartner Group high-level technology survey named Data Mining and artificial intelligence as the five most influential industries in the next three to five years.
Key technologies are at the top, and parallel processing systems and data mining are also listed as the top 10 emerging technologies that will focus on investment in the next five years. According to Gartner's
HPC research shows that "with the rapid development of data capture, transmission and storage technologies, large system users will need to use new technologies to explore prices outside the market.
Value, using a broader parallel processing system to create a new business growth point ."

The most common technologies used in data mining are:

Artificial Neural Network: modeled after the non-linear prediction model of the physiological neural network structure, pattern recognition is performed through learning.

Decision tree: represents the Tree Structure of the decision set.

Genetic algorithms: Based on evolutionary theory, they are optimized using methods such as genetic integration, genetic variation, and natural selection.

Nearest Neighbor Algorithm: A method used to classify each record in a dataset.

Rule deduction: searches for and derives the "if-then" rule in the data in a statistical sense.

Some specialized analysis tools using the above technology have been developed for about ten years, but these tools usually face a small amount of data. Now
These technologies have been directly integrated into many large industrial standard data warehouses and On-line analysis systems.

How does a data mining tool accurately tell you the important information hidden in the depths of the database? How do they make predictions? The answer is modeling. Create
A model is actually a model created when you know the result and applied to a situation you do not know. For example, if you
If you want to find an ancient Spanish sinking ship in the sea, you may first find the time and place to discover these treasures in the past. That
After investigation, you found that most of these sinking ships were found in the Bermuda sea area, and that sea area has a distinctive ocean stream and the route of that era
There are also some features to be searched. Among these many similar features, you abstract and generalize them into a universal model. With this model, you are very hopeful
Discover an unknown treasure from another location with many identical features.

Of course, before the emergence of data mining technology and even computers, this abstract modeling method has been widely used. Modeling and
The previous modeling methods were not very different. The main difference was that the amount of information that computers can process is larger than before. The computer can store known endpoints
A large number of different situations, and then the data mining tool extracts information that can generate models from these large amounts of information. 1. When the model is created
After the establishment, it can be applied to the judgments that are similar but with unknown results. For example, if you are a marketing director of a telecommunications company
If you want to develop some new long-distance phone users, will you scatter advertisements on the street without any purpose? -- Just like searching for treasures at sea with no aim
Sample. In fact, it is much more efficient to use your previous business experience to win customers with a destination than to publicize them without any aim.

As a marketing director, you know a lot about your customers: age, gender, credit history, and usage of long-distance calls. Slave
On the one hand, mastering the information of these customers is actually mastering the same information of many potential users. The problem is that you don't necessarily know about them.
Long-distance call usage (because their long-distance call may be via another telecommunications company ). Now, you focus more on users.
. Through the following table, we can abstract some variables from the database and create a model that can be classified for marketing.
Customer potential
General information
(E.g. Demographic Data) known
Private Information
(E.g. customer transactions) known to be TBD

Table 2. Apply data mining to classified Marketing

Based on the computing model from general information to private information we created, we can obtain the information in the table at the bottom right of table 2. For example, a telecommunications company's
The simplified model can be: 60 thousand of customers with an annual salary of more than 98% US dollars, with a monthly charge of more than 80 US dollars. Based on this model, we can apply the data to push
The company's private information is not yet clear, so that new customer groups can be identified. Small market trial sales data for such a model
It is extremely useful. Because the mining of trial sales data in a small scope can lay a good foundation for classified sales in all markets. Table 3 describes the other
Common application of sample data mining: prediction.
Past, present, and future
Known static information and current plan
Known dynamic information to be determined

Table 3. Apply data mining to Prediction

Architecture of Data Mining

Many existing data mining tools are independent of data warehouses. They need to input and output data independently and perform relatively independent data analysis. Is
To maximize the potential of data mining tools, they must be closely integrated with Data Warehouses like many commercial analysis software. In this way
When the parameters and analysis depth are changed, high integration can greatly simplify the data mining process. Displays the advanced analysis history of a large database.
.

Integrated Data Mining System

The ideal start point for applying data mining technology is to start from a data warehouse where contract information of all customers should be stored and
In addition, there should be relevant data from competitors in the market. Such databases can be databases on various markets: Sybase, Oracle, Redbrick, and
And can optimize the speed and flexibility of the data.

The OLAP server of the online analysis system can apply a very complex end-user business model to a data warehouse. The multi-dimensional structure of the database allows users
Analyze and observe their business operations from different perspectives, such as product classification, Region Classification, or other key perspectives. Data Mining Server
In this case, it is necessary to closely integrate with the on-premise analytics server and data warehouse, so that you can directly track data and help users make business quickly.
Industry decision-making, and users can constantly discover better behavior patterns when updating data, and apply them to future decisions.

The emergence of a data mining system represents the transformation of the basic structure of the conventional decision-making support system. Unlike the query and report languages, only the data query results are fed back
As the end user does, the data mining advanced analysis server directly applies the user's business model to its data warehouse and gives the user a related information
Analysis results. This result is an analytical and Abstract Dynamic View layer, which usually varies according to different user needs. Based on this view, various reports
Tools and visualization tools can present analysis results to users to help users plan what actions they will take.

Tools for generating profits

Many companies have successfully installed data mining tools. Most companies that have adopted this technology earlier are information-intensive companies, such as financial services and
E-Mail Marketing System, but now this technology is ready for various companies, as long as the company has a large database, and has a strong adoption of software technology
Desire to improve company management. However, using data mining technology requires two key factors: large and integrated databases and
Is well-defined business processingProgramIn this way, data mining can be closely applied to company data.

Some successful applications using data mining technology, such as a pharmaceutical company, decide which one by analyzing its recent marketing intensity and sales results
Marketing campaigns have the greatest impact on high value-added physician groups in recent months. This analysis is based on competitors' sales campaign information and local health
Data System. Then the pharmaceutical company can use its office network to deliver the analysis results to the sales representative offices in various regions. Then, the sales representatives can
The sales representatives can make appropriate sales decisions based on the key information transmitted by the Division. In this way, in a rapidly changing and dynamic market, the sales representatives can
Analyze and make the best choice.

Conclusion
A large data warehouse that fully integrates customer, supplier, and market information leads to explosive growth of company information.
This information is often accurately analyzed. In order to make more timely and accurate choices that are conducive to enterprises, it is established in relational databases and on-line analysis.
The technical data mining tools bring us a new turning point. Currently, data mining tools are developing at an unprecedented speed and expand the user base.
In the future, more and more fierce market competition, data mining technology will surely get a quicker response than others and win more business opportunities.

Computer World developer club http://dev-club.esnai.com