Academician of Engineering Academy: Technical trends and application prospects of large data

June 5 News, the theme of "large data bandwidth to promote cloud computing applications and innovation," The First China cloud computing Conference at the Beijing National Conference Center held today, NetEase technology as the General Assembly cooperation portal in the live report.

The following is the National Natural Science Foundation, Deputy director of the Chinese Academy of Engineering, Gawain, to do the theme of "Large Data technology trends and application prospects" speech.

Gawain: Ladies and gentlemen, just now the host said that everyone had just finished eating, so I hope the following report can help you digest, at least not hinder everyone to digest.

I report this topic is called "The technology trend and application foreground of multimedia Big Data", this is a proposition composition, maybe I want to do a little sihuo under this proposition composition, probably say three questions: First, repeat, perhaps someone in the morning, perhaps we have seen some of the major trends, Hopefully, this trend will give you a more direct feel for the second question. Second, talk about multimedia large data, but also talk about the specific application of multimedia data, is the smart city. The third is the closing or the outlook.

The era of multimedia data.

Multimedia large data age, I believe that everyone is in it, a most typical feeling, we should know now the whole, whether we call it a cloud, or a big data, in general, the speed of the increase in data is very fast, we estimate from now to 2020 the entire wide range of large data on the web will be to 40Z, Everyone will be 50 times times more than now. This data is large, first of all, a performance is very rich in data, you can describe it as "flood", of course, after more data, the business is more than two, it may bring us a lot of benefits, at the same time to our treatment also brings a lot of trouble.

Specific include: For example, in the retail industry, scientific computing industry, life has a variety of data, of course, in this area of data impact, the volume is greater, but some data impact is not large, but the volume will be very big, so the current storage, processing platform has brought great challenges.

In fact, our lives, our jobs, are changed by Big data:

1. The way the data is produced has now been dramatically changed, since the production of data was previously done by professional groups, professionals, or professional firms, the data are now more of an individual act, an individual, and each person can use the terminals he collects, whether they are photographs, videos, or through his mobile phone, Computers produce a lot of data, so the way data are produced has changed a lot.

2, the traditional way of data has also undergone great changes, in the past, the source of our information was basically either print media, newspapers, or the media, such as television, radio, and so on, and now, especially young people, through the Internet, the Internet has become the main channel of media communication, This change has changed a lot for our society as a whole. As you know, Ma Yun, the video from his chairman was seen in a very short period of time, so I think this is a really great thing, and it also tells us that in the future we do anything to think about information, or that the main way to spread the public opinion is through internet media.

3, social networking changes in the network, before we make friends more is your life circle, such as classmates, neighbors, relatives, now more through the Internet this virtual environment.

4, data storage habits change has occurred, previously all want to save a copy, whether it's a photo, whether it's your file, you want to be on your computer, or you want to engrave it on your floppy disk, and put it on the shelf, this idea has changed, of course, unless you're doing some secrecy work, Or a little older than the habit is a different story, most people put it online, in the cloud to store.

5, the Social security system changes, has not really realized, after realization will have a very big change to our life. I will spend more time talking about a problem, now the entire social security, the whole city has a lot of sensors, cameras and so on, through which they can make society more stable, or the case will be solved quickly, this system will be a very big impact on the current security system. For example, we often examples, including the original Zhou Kehua of the case, is the monitoring system has made a considerable contribution, of course, this contribution is ultimately relying on people to solve the problem, at that time in order to find a more clear, positive photos, or to find some of his personal characteristics, There were more than 2000 policemen who spent the last month watching the videos repeatedly, finally finding some photos to solve the case, or to escape. Of course, there are unsuccessful cases, such as the spring in March in Changchun, someone stole a car, the car has children, and finally killed the child, this is the perpetrator finally surrendered to find clues. The Boston Marathon bombing earlier this year was also a relatively short time to solve the case, this case is in fact also the United States police centralized a lot of police to get the entire video data can be repeatedly seen, almost every video has been seen more than 400 times, and finally found some clear pictures to help chase away.

These cases tell us that the big data age is here, of course, in the big data we need to know what is the largest quantity, this curve you can see, we have 2012 of years of image and video data has accounted for more than 80%, this year, image and video data in the entire large proportion of data has to be close to 90%, such a proportion, What is your core challenge in real big data, of course, how to deal with data, how to excavate some of the value of the law, is our first to do. How to deal with the biggest data is actually the biggest challenge.

So we multimedia large data, special and image, video related transmission, storage, processing, application is four problems. Why is the problem of effective storage difficult, in fact, now on the street by the camera is not always photographed things to save, some will save three months, some will save one months, and some will save one weeks will be covered, this data will be lost forever, why this? The cost is too high because it cannot be saved. How to save effectively for large data is a big problem. How to deal with, we all think that the problem is very simple, there are a variety of intelligent processing systems, image analysis system, in fact, those are re-study or to do demonstrations, such as urban large-scale systems or relatively few.

Second, the challenge of large multimedia data in smart cities. What is the challenge? Because the wisdom of the city itself, this concept is a very good concept, the so-called intelligent city is also an ecosystem, the ecosystem in order to achieve, for example, the city's peace, health, livable, convenient transportation and so on, so you need to build a complete information system, This information system is actually composed of video sensor, IoT system and network, and the whole decision system is constructed into a complete system, which is called Intelligent City. In this system there are many subsystems, including the visual security control system, emergency linkage subsystem, Digital city management system, etc., such as security air defense system through each card, electronic police, some monitoring, technology and other subsystems to achieve security control. For emergency linkage, the function of emergency linkage can be initiated after the discovery of natural disasters and public affairs. The city management includes what we often say about car positioning, geographic information, authentication, object identification, digital communication and so on, linking these systems together is a complete information system. The monitoring of this system is critical, of course, not video, camera, which includes a variety of sensors, video is the main sensor, these sensors are distributed in every corner of the city, in every corner of the access to the information is through a network of the information can be transmitted, of course, these transmissions, The network is a group of a variety of groups, which are related to residential, and office-related, and transport-related and so on. These groups of data through the network will be structured into a so-called perceptual network, if you are the main camera is the visual perception network, the visual perception network will be sent to the Intelligent Analysis Center, and finally to the entire intelligent city decision-making. There are two very big problems in this decision:

1, can not save, just have said that the data up to three months, some may be one months, one weeks will be covered up, the data will be removed forever, it is a pity.

2, find unhappiness, if there is one thing to hope that the system quickly find out instead of finding unhappiness.

First of all, we look at this problem, because the current data flow, continuously pouring into the system, storage system unless you are very rich, constantly increase storage equipment, but now does not say which people continue to increase storage equipment, a budget is stored for a period of time to forget, Or take the information inside and put it in another place. It's easy to get in here, if there is a very good efficient storage technology, or the original image and video coding compression can save storage space, the original can save three months, if you can increase its compression efficiency of one-fold can save six months, or the original system cost to lose half. This thing in the technical field has been done for more than 30 years, probably from the beginning of the 90, someone proposed, just started not for video surveillance, but for radio and television, digital TV, such as the first generation of coding standards, earlier the use of VCD, DVC standards, Did a period of time we feel that the compression is not enough and began to do the second generation, feel that is not enough to start the third generation, now the system is heavily used H.264, we call the second generation of standard technology, is now doing some of the third generation of things.

How high is the third generation, the second generation, the first generation of coding? The first generation of coding standards can compress the original video data to 1/75, the second generation than the first generation of efficiency increased by one times, to the original video compression to 1/150, now doing the third generation is probably able to compress the data to 1/300, presumably, After the video is compressed, it is stored there, and if there is a way to find a coded compression technology, it can improve its coding efficiency. Why have 150:1, 300:1, and even the future to repair 600:1 compression ability, because we make video when there are a lot of redundancy, as long as you have a good algorithm to remove this redundancy, so the towel is to do video compression to do, why can this? Because there are all kinds of redundancy, such as the same thing each frame has been photographed many times, if each frame is to be expressed again must be very wasteful, can the later copy of the same handcuffs come on. There is also called coding redundancy, the actual theory can be analyzed, the current algorithm is far from the real theoretical limit, but could not find a better algorithm to approximate the theoretical limit, so now tens of thousands of engineers and scientists to find efficient coding algorithm, hoping to improve a little bit. So we have a lot of space. The basic frame is the frame that this picture gives. Generally speaking, the use of orthogonal transformation, motion prediction, business and so on to make the coding efficiency a little bit higher.

AVS Video Standard Framework is also a big idea, are the same, the world several major different technology groups adopt the idea is relatively similar, why China to do this thing? In addition to improving efficiency, there are other reasons, patents, intellectual property rights and other reasons, in order to let China's own enterprises, or Chinese enterprises to overseas development time, not to be bullied, so China has to have such a thing, this set of things from the pure technology, effect and so on.

After some time in China, in China do not necessarily are Chinese, of course, AVS are foreign enterprises, the international more well-known large cooperative video coding related enterprises have sent representatives to join the AVS work Group, recently made a thing into an IEEE standard, called the IEEE 1857 Standard, For the Internet video codec group, this group was formally established in February 2012, after a very complete process, the first video coding standards yesterday has just been printed, approved in March this year, after three months of preparation time just finished printing yesterday. IEEE 1857 the entire processing process, since the establishment of last February, the first meeting in April about 7 meetings of this year, March 15, and finally complete the text.

In this text, the whole editor goes through a very long process, so doing a standard technology is very good, the process must be step-by-step to go back and forth many times, now this IEEE 1857 standard has a variety of different application-oriented parts, we call profile or group, It's not the same as other standards, or it's a feature where IEEE 1857 has special support for video surveillance.

What kind of special support? Is the first time to add the background modeling technology to the entire processing process, added to the loop, this is a very difficult thing, this figure shows that when you do not do background modeling your processing efficiency is not so high, as your modeling is getting better, your coding efficiency will be higher. So here's the Blue line, it's actually the whole efficiency, the more efficient your code rate, for example, now give you 2 trillion bandwidth, the same quality of things you will account for why the bandwidth is low, so the efficiency is higher, this is a very good idea. How did the background modeling work out? This is a very intuitive picture, when it gives you a video, you want to see the food section is still very complex, through a mapping change in another space, you will find that the transverse is almost the same, but someone, there is a car moving a little bit of change, we have based on this change to build a background model, Use it to improve coding efficiency, if from a technical point of view, we have some objects, I can model this object, put the original did not have to do with some points, and finally found a very clean background, all of which are not background things, according to these things I know what new and what is new, The efficiency of the coding becomes very high, this also includes a variety of different weather conditions, time, fog, rainy days can be the corresponding modeling, the model through the changes in parameters to cover, it is easy to detect the object out, detection of our analysis behind will be very helpful. And the model can be constantly updated. The standard was published yesterday. Of course, this version, a version of the end, there are various departments in the step-by-step, this standard than the existing standard coding efficiency of one-fold. It's about saving.

Find unhappiness, so far analysis and coding are done separately, separate to do because it is a completely different system, if you are technically understandable, but to do a separate one of the biggest problem, in the code can not be analyzed in the analysis of the time can not be encoded, is generally the first code from the collection end sent back to the end of the analysis , analysis and then untie it for analysis, so that their precious time lost. This is still the most important question, in order to pursue the coding efficiency you may lose to your recognition rate, we give an example: Of course, IEEE 1857 can solve this problem well, for example, the red box is interested in, detected immediately in the description text inside this description, can be started later, including object detection, Object tracking, behavior analysis, behavior tracking, a complete loop can flow out, and can include GPS information are included. Object detection, the face how to do, in fact, now have the technology to find this.

In particular, there is a video on the left, and someone is walking up and down in this video, which is the best face in the movie. I do not know now is not know, now is the computer to do this thing, we can a model, the model has 6 main parameters, through the 6 parameters include, I see its resolution, brightness, posture , sharpness, noise level, gray level, the 6 parameters can be combined to find a clearest face image. Of course, the same idea can detect cars, testing people and so on.

In order to do this, this year to organize a national postgraduate Smart City competition, is now being organized by the Ministry of Education Degree Research Center, the Intelligent City Industry Alliance, China Association for Science and Technology and so on, the Secretariat is now located in Beihang, now has some specific plans.

As an epilogue, the big data itself this matter, whether you agree with it or not, and in large data images and video data volume is particularly large, in the image and video how to save and find Fast, this is two very big technical challenges, we should be in these two technical challenges to work hard, Make big data do not have no way to it, can have the way to deal with it, treat it as a best application scenario of big data of intelligent city, also hope that in a few years this meeting has better similar results to show to everybody. Thank you!

(Responsible editor: The good of the Legacy)