End-to-end: IBM Large Data Solution resolution

As can be seen from IBM's large data platform framework and application solutions, large data platforms include 4 components: information integration and governance component, core processing platform for large data (including Biginsights platform, streaming computing platform, data Warehouse based on the framework of open source Apache Hadoop) , contextual search, four parts), accelerators, and high-level applications that include visualization and discovery, application development, and system management.

Mr. Lu Weihuan, general manager of information management software at IBM Software group Greater China, said that, in addition to the traditionally mentioned large data (Volume), diversity (produced), speed (velocity), the authenticity of data (veracity) will become increasingly important in future large data applications. "Social data, enterprise content, transaction and application data, which transcend traditional data sources, require effective information governance to ensure their authenticity and security." "IBM in addition to the traditional data warehouse and data information management and audit, but also from different sources of information for the authenticity of large data audit and effective control, this is IBM in the industry is particularly strong than other vendors important dimensions." ”

It is learned that the components to achieve information integration and governance are guardium, its data governance component has three main features: first, its master data management enables the management of duplicate data from different data sources; second, each product has security management; third, it is managed through an integrated platform. Currently, Guardium is able to manage software data including DB2, Netezza, Oracle, Sybase, Informix, SQL Server, SharePoint, Teradata, MySQL, and so on.

On top of this is IBM's Biginsights platform, which is based on the framework of open source Apache Hadoop, and adds capabilities including management capabilities, workflow, security management, and integration into the unique and leading data analysis of IBM Research labs, Machine learning technology and text data analysis and mining. IBM says all of these enhancements are designed to better enable the program to be used for complex, massive data analysis. "The Hadoop platform does not have the appropriate administrative tools and the ability to summarize different data." Lu Weihuan said, "IBM borrowed from the experience of the past decades in the database field, the management of the database has been transplanted to a large data management platform, so that the Hadoop platform for the availability, manageability, security has increased a lot. According to incomplete statistics, IBM has added at least 100 features to the Hadoop platform.

Not only that, Biginsights supports the most popular x86 platform at the moment, but also supports a powerful power platform. "With Linux systems optimized for the power platform, Biginsights can run well on power system." "This allows large data solutions to fully enjoy the power system's performance while at the same time distributed processing capabilities," Lu Weihuan said. "IBM's Biginsights program is very open, not only to support standard Hadoop, but also to support some of the mainstream Hadoop distributions, such as Cloudera Hadoop," added Jian, senior advisor to IBM's banking solution for the Greater China Software division. This means that customers can move smoothly from third parties to the IBM Enterprise Hadoop platform. "As a contrast, Oracle's large data solution explicitly requires the application of the Hadoop version optimized by Oracle Company."

However, "biginsights is not a replacement for the Data Warehouse, it is a supplement and extension of the traditional data warehouse, the overall formation of a broader internet-level mass data warehouse." "said Mr. Jian.