Hadoop: Opportunities for security vulnerabilities, good prospects

Hadoop, as a large, hyped data tool, is just beginning to design and not to process credit card numbers, but to create Web pages for search engines, so the security issue is not a priority. For this reason, many companies are skeptical of Hadoop. For now, several Hadoop distributors, including Cloudera and Intel, are implementing or developing security plans.

Patents and Patches

Zettaset is a company that provides security features for the Hadoop release, and its chairman and CEO Jim Vogt says many companies have become more interested in Hadoop technology this year, but a large part of it has not moved on because of security issues, When it comes to measuring the viability of a technology in an enterprise or a broader market, security issues must be considered. ”

According to Vogt, Zettaset has patented methods for managing and controlling cryptographic key technologies distributed across multiple servers in a Hadoop cluster. In order to achieve security on Hadoop, minimizing performance degradation, Zettaset will launch a system to implement priority data storage in the cluster next year. If a block of data is frequently accessed, it is faster to put this part of the data on SSD than on the hard drive. Of course, by pointing out existing deficiencies that make users aware that their infrastructure is unsafe, the immediate beneficiaries must be zettaset and other companies providing security services.

Charles Zedlewski, vice president of Cloudera products, believes that security issues are mainly designed in four aspects:

Authentication: How to verify that you are a user of a system.

Authorization: Controls the information that the user can read and the actions that are available to users of a particular data.

Audit: Audits can provide documentation support to meet regulatory requirements to see if there is a violation.

Encryption: Provides more protection for data security.

At present, native Apache Hadoop provides some of these features in MapReduce, HBase, Hive, and other Hadoop programs. Hadoop, for example, has a rigorous authentication mechanism. Zedlewski that what we need to improve from a customer perspective is to make it easier to install and configure.

"Encryption is another matter," Zedlewski said. Data can be encrypted over the course of a network transfer, a feature that was implemented two years ago. Involving the encryption of ' static ' data, some companies use off-the-shelf cryptographic libraries, such as those provided by security providers such as Gazzang and Vormetric. "Cloudera is considering adding encryption to its products, so customers don't have to look for other security service providers." Cloudera, as the leader in the Hadoop market, is commendable.

Zedlewski that Hadoop is not mature enough in terms of authorization. Cloudera wants customers to be able to determine the authorization granularity of a table on their own. For example, a table of 10,000 credit card numbers, if you have permission to view part of the data, then based on the table granularity authorization mechanism, you do not have access to the table's permissions, and based on the record granularity authorization mechanism, you can see 50 specific range of data. In other words, fine-grained authorization mechanisms enable more employees to gain access.

Rhino Project

About 3 months ago, Intel, which had just joined the Hadoop camp, listed the security features that were expected to be implemented in Hadoop under the Rhino project.

In the area of authentication, implement a new internal system that does not rely on external sources, while providing better single sign-on capabilities. Licensing mechanisms can span many Hadoop applications, from batch mapreduce to HBase databases. These features are added to Intel's Hadoop release and can be added as patches in other distributions.

Knox Project

Several engineers from Hortonworks have been active in an incubator project called Knox this year. Shaun Connolly, vice president of corporate strategy at Hortonworks, explains that the project is like constructing a large virtual enclosure around a server in the Hadoop cluster, with only one security gateway for available Hadoop services.

Jack Norris, chief marketing officer at MapR, said: "MapR is trying to add encryption key management capabilities, including" static "data. Just as CLOUDERA,MAPR wants to make security issues easier, especially in the process of data encryption and authentication. ”

"Edit Recommendation"

Hadoop:cloudera acquisition Myrrix "Big Learning" white Elephant: Developers must have the Hadoop tool data analysis ≠hadoop+nosql, improve the existing technology of 10 shortcuts to win financial----- Analysis of HP Flow MFP Security Characteristics "responsible editor: Xiao Yun TEL: (010) 68476606"