Red and black of big data

Photo by Liu Lian/Guangming network profile image

The world is entering the second movement of the IT revolution and globalization-the "cloud era ". Through the Internet, the huge computing and storage functions of data centers cover every corner of the world like clouds. Today, both enterprises and individuals can make convenient and free use of high-performance cloud computing systems beyond the border, and the human life, work, and entrepreneurial environments have suddenly changed.

-Entering the cloud age, Zhou mazhi

This is a dangerous world. After the subprime mortgage crisis in the United States in 2008, it followed the European sovereign debt crisis. The leaders of the two world economies, the United States and Europe, are on the verge of recession. The Japanese economy in Asia is not optimistic. What's terrible is that people cannot see a way out of the crisis. Although the BRICS countries are eye-catching, it is difficult to stand alone when global integration accelerates. The world seems to be shrouded in pessimism. People can't help but ask where they want to be?

In fact, every system and every mode is abstract, but it will consume and degrade like a specific item. Throughout history, these old ways of life and production cannot escape the fate of elimination. Innovation and Development of Science and Technology will always bring about real changes. This means that, in the face of crisis, people need a new development direction and motivation, rather than repairing and supplementing aging parts. As it takes too long to ride a horse across Eurasia, if you are always looking for a better horse or make the current horse run faster, it is only a balance. What humans really need is today's cars, high-speed trains, and airplanes. After the emergence of the Internet, if people only want to talk and chat, there is no need to cross the continent or the ocean, because the network makes the geographic distance completely negligible.

Today, the hottest topic is cloud computing and big data. People's expectations for this change are comparable to an industrial revolution. People often think that science and technology are uncertain and far from themselves, but in fact,The era of big data has quietly penetrated into our daily lives.Fang. The most widely used field of big data may be the consumption field, followed by China Telecom. Telecom service providers are trying to use big data to optimize the use of their capital equipment.

Application Prospects of big data

Big Data has been deeply rooted in the consumption field. Big data is used in this field first, because big data is easy to use and does not need to be controlled, and consumer software is faster than other software. For example, if you buy a book on Amazon, Amazon will recommend another book to you based on the comments of other buyers. In the past few years, consumers and software targeting consumers have led the development of the big data industry, and people's own ideas have been integrated with each other through data. Game companies create a variety of virtual products and use big data to analyze consumers' purchases of virtual products. For example, a game company uses how many virtual umbrellas people buy as a function to determine whether it is raining. It can be seen from this that there is a very interesting connection between the virtual world and the real world.

The use of big data can cover all aspects of life. GE once had an industrial Internet idea about how to connect all parties, how to obtain information from jet engines and automotive engines, and how to ensure optimal operation of these large industrial systems. The use of big data in the industry may be more important. At the same time, the use of this aspect can actually change the world by changing the nature and appearance of the product.

Generally, two aircraft engines can generate 5 TB of data during flight. If the system can analyze the information and pass the information to the pilot before the pilot finds that the engine temperature is abnormal or there are other errors, inform the pilot of the running status of the engine and whether there are other problems, this will further improve flight security. It is very important that all information in the course of Sailing can be informed in a timely manner. It is sometimes able to save lives. This is especially meaningful in today's frequent air crashes. In addition, in the healthcare field,Big Data can combine different features of each person to help create better medical solutions.

Importance of Data Filtering

There is a lot of data, but it is useless. Either there is no value, or it is directly wrong. Big Data technology needs to solve the data cleaning problem. The first thing to use data is to clear data. Before you even extract the data from the data source to the storage location, make sure that the data you entered is valid. Sometimes, you need to filter 5% of the data, and sometimes you even need to ignore 95% of the data, because the data is too complex. People should not store the data they don't need, which is a waste of resources. The current technology can help you divide one thing into multiple steps so that people can remove a lot of data that you don't need before storage and processing.

Many enterprises want to join the big data revolution. Many people think that what they need to do is to use all the data they can obtain. This is actually not economical, because the amount of data is too large. Therefore, we need to clarify our own needs. Like baking a cake, you may want to add all kinds of condiments, but you will not put red chilies in it, because it is irrelevant to baking a cake. Therefore, to solve problems based on big data, you want to remove irrelevant data and focus on valuable data.

Big Data Challenges to computing

There is an interesting thing about big data. The name "Big Data" comes from two aspects. On the one hand, it is because of a large amount of data, that is why we call it big data. On the other hand, it comes from the data disorder. In the past, computers could only process data input in one form. When the format of the data you entered is incorrect, you need to adjust it again.The essence of big data is to be able to process unnecessary data. Since people and enterprises began to process unstructured data, they have developed new technical tools, such as distributed computing. These new technical tools allow people to store data in an unstructured form.

Because of the large amount of data, you cannot always bring a host or browse the data through a supercomputer. What you need to do is to resolve big problems into a series of small problems, and then use a more efficient computing system to solve these small problems one by one. This is "parallel operation ". In fact, if you go to Google's data center and you don't even see a host, you will see a lot of motherboard like your computer. Currently, we are exploring the ability to combine the results after resolving major problems into small ones.

The future of cloud computing

There are many ideas and big plans for using big data, but in order to achieve this, big computing is required first. The key to the problem is how to build an computing system that is much more efficient than a computer created in 1970s? People use cloud computing to achieve big computing. Cloud computing enables infinite processing capabilities even with cheap components. People no longer need to buy a new motherboard because one data center or all computers in multiple data centers can be used on the cloud. How much processing power you want, how much you can get, and suddenly you have to deal with thousands of hard disks and motherboard. When you no longer need these processing capabilities, you can put them back to the original place. The cloud center is constantly running. It is very important to ensure smooth operation, and this is also the research and development direction of new technologies.

If everyone starts to rely on this system, they need to ensure that they have sufficient processing capabilities. For example, in the hot summer, people need to ensure the power supply of all air-conditioning systems in the city. In the event of an accident, people are not allowed to use air conditioners. In computer operation, utility computing is such a "supply ". Public energy companies provide us with power, water and other energy, while cloud computing companies provide us with computing power. In the past, if a company wanted a large software, it would need to purchase tons of servers. Today, people do not need to buy so many servers, because they can rent or buy virtual servers from computing service companies.

Big Data and hackers

Everyone is worried about website hackers and data leakage. Imagine that if cyber hackers attack the Beijing Power Grid, whether it is for prank or terrorist attacks, they can cause the power system to crash in Beijing.

The field of data security is actually a battlefield, but it is not necessarily clear. On this battlefield, there are good people who strive to protect data security, as well as hackers and bad people who attempt to break into the data system for destruction. It's an endless problem. Technology companies must take the lead to make sure they have the technology they possess is better than hackers and software terrorists who want to destroy data. Although there are many ways to break into the computer system, when the system is truly compromised, people must be aware of it. Previously, viruses were the only source of such harm, but unfortunately there are many such sources.

When measuring the risks of Data infringement and the effectiveness of data, the good side prevails in general. All people have to do is ensure that data is protected. E-banking is a classic example. This is actually terrible because people may break into your e-bank and steal your money. But can you leave E-bank now? Now, it is too late for us to consider turning back. You do not want a car accident on the road, but you do not want to give up driving. Therefore, this industry will eventually create a large number of products to ensure data is not infringed, and to ensure security in all aspects.