Big Data puzzle for Boston Marathon bombing

In the aftermath of the Boston Marathon bombing, the new technology in the IT industry and the business world-big data, can reflect their value, after the analysis of massive data to identify the suspect? How do government law enforcement departments avoid touching civil liberties and citizen privacy red lines in the process of data acquisition and analysis of crime detection and prevention? even, How to prevent the occurrence of crime through large data technology? These are the lessons that big data professionals and governments and public security departments need to learn from the Boston Marathon bombing process.

According to the Los Angeles Times report on this week's bombing, the FBI has collected about 10TB of data near the crime scene after the Boston Marathon bombing. According to Time magazine, the data include phone records from mobile base stations, surveillance footage from nearby stores, gas stations and newsstands, as well as images and video footage provided by volunteers. (This raises the question of whether a number of criminal investigative efforts can be wrapped up to protect citizens ' privacy rights.) )

At present, the FBI collected 10TB of data and "big data" of the "Volume" some gaps, but the impact of data analysis results is not only the size of the data, and several other important factors are data sources and quality. Unlike in the case of a massive bank transaction that identifies money laundering transactions in underground banks, it is much more difficult in the Boston bombings to lock down suspects or find sources of bombs. According to a blog from data analysis expert Jeff Jonas, regular data sources often fail to provide any information that will help find bombs.

Jeff Jonas points out that when conventional data sources are inadequate, it is necessary to pay attention to the following points when detecting or even predicting a crime requires adding new observational space (i.e. data sources):

1. The integration of external data sources into the analysis is an art that requires a full understanding of the data-flow mechanism inside and outside the firewall, as well as related legal and policy issues.

2. The order to increase the data source is in the first and the outside. Of course, can also increase the old data source collection range. (Editor's note, in the case of the Boston bombing, increasing the range means not just collecting video surveillance and moving base station data along the marathon)

3. The key to catching the bad guys is that some of the data sources (temporarily) are not known to criminals.

4. Make good use of social media.

5. Prioritizing data.

For the FBI and the Boston police, expanding the source of data and viewing space does not just mean increasing the analytical power of monitoring and monitoring data, but more importantly, increasing the ability to gather and integrate social media and mass data. At a seminar this week, IBM I2 chief expert Haibo The case of "Skynet project" as an example:

1. From the report to the next morning the crowd reported why did not find the car?

2. The camera accuracy is insufficient, still cannot process the video record?

3. The card after the report can not receive timely notification and interception?

4. If the offender was with whom and whether he could obtain evidence, could it be proved that the person who surrendered himself was a murderer?

The Skynet project's drop chain shows that increasing the collection of old data sources may not solve the problem. Data quality, visualization and intelligent, unstructured data rapid and even real-time analysis, prediction are "police data" need to face a major issue.

As the first large data prediction and analysis pilot unit in the U.S. Police force, the Santa Cruz Police Department predicts crime sites and times through large city data, analyzes historical cases, discovers crime trends and crime patterns, finds common ground and relevance, and can even predict crime by analyzing urban data sources and social network data. Data analysis, which used to take days, weeks or even months, was completed within hours of the latest large data analysis system for police Hadoop, which greatly accelerated the efficiency of police handling. Reference reading: Police 2.0: Crime prevention with large data

While the FBI and Boston Police issued a statement yesterday condemning and denying reports from several media, including CNN and Reuters, that "locked and arrested suspects" (Editor's note: The headlines of these big media are likely to come from a similar Reddit civil investigation team), But social media are far more efficient at collecting pictures and videos than the FBI's 800 phone calls (but there are many legal risks and moral controversies). If we can design explicit police social media norms and information sharing mechanism beforehand, we can effectively enlarge the source of police data analysis, improve the efficiency of criminal investigation, and reduce and avoid the spread of rumors.

(Responsible editor: The good of the Legacy)