Star ring release supports spark and HADOOP2 large data software

The 2013 China Hadoop Summit Forum, following the October end of the Hadoop technology, the largest company in the United States Cloudera Company announced and Databricks cooperation, providing the Apache Spark Computing framework of technical support, the local large data platform software company Star-ring information technology ( Shanghai) Co., Ltd. (hereinafter referred to as "star-ring Technology") pioneered a large data platform in the country to launch the product of the Transwarp data Hub, the integration of Apache Spark and Apache Hadoop 2.0, providing comprehensive localization of technical support.

"With the development of mobile internet and interconnected devices, enterprise data has multiplied every year, so Hadoop technology has received wide attention from the outset," says CTO Sun Yuanhao, co-founder of Star-ring technology. Large enterprises and government agencies have tried to use Hadoop to solve a large number of complex data storage and query problems, while looking for a better cost-effective combination of hardware and software. Apache Hadoop, as a distributed computing framework and storage engine, is an excellent match for this requirement. Hadoop can be said to have the "operating system" or "data Hub" conditions for future enterprise data. At the same time, many of the original application of SQL development and complex data set of the enterprise also put forward a greater challenge to Hadoop and its surrounding software.

Large data platform software needs to simultaneously support massive data storage and high speed analysis capability

The four characteristics of large data-massive data scale (volume), fast data flow and Dynamic Data system (velocity), diverse data types (produced) and huge data value (value).

At present, the combination of HDFs and map reduce solves the problem of massive data scale and various data types, but there is a great space for the flexible support of application in the case of high speed data query demand and different data scale scenarios. As the next Generation Map/reduce computing framework, the Apache Spark takes full advantage of memory acceleration, and solves the problem of interactive query and iterative machine learning efficiently. The Transwarp Data Hub, with support for Apache Spark and Apache Hadoop2.0, uses optimized memory processing technology to 10x/100x the level of performance of the Hadoop platform.

Support for multiple scenarios and data pressures

At the same time, through the integration and optimization of the new framework technology, TRANSWARP data hub to expand the application of Hadoop and the ability to handle different data pressure, suitable for processing from GB to PB-level data. Currently available for off-line analysis, statistics/mining, online storage and OLAP systems, and online memory based high-speed analytics.

"The widespread use of Apache Hadoop also requires integration with traditional data-processing tools to provide convenience to solution providers, application developers, and to create a more complete system that benefits the user." Sun Yuanhao also said: "So we are developing the Transwarp data hub at the same time, we seek and more data processing and analysis tool manufacturers to cooperate and integrate." At present, and revolution R,informatica,tableau has been achieved docking.

"The Transwarp Data Hub has been tested and used in a number of key customer areas, with more than 10 software and hardware partners, and we hope to get more customers through our technological innovations." ”

Full cooperation with the domestic ISV, the program business

In the market, star-ring technology to launch a partner support program, sincerely hope that with the industry's ISVs, system integrators, Hadoop service providers to carry out in-depth, win, long-term cooperation. Hope that the products for the partners to bring new large data opportunities, new customers, and rich long-term returns. The launch of the Local Partnership program, proposed 9 specific to the partner's support program.

IT168 's corporate chief editor, Hong Jun, chaired the conference and the guest interaction at the scene, with star-ring technology's strategic partner Revolution R Asia Pacific general manager Laurence Liew, databricks start-up staff, Spark core members Reynold Xin, Bdas Tachyon author Li Haoyuan, Henderson General manager Shen Zhenyu, chief architect of Informatica Greater China but he gathered together to share their views on the opportunities and challenges of the big data age and the future development of the industry.

Speaking of cooperation with star-ring technology, databricks start-up staff, Spark Core member Sing (Reynold Xin), said: "Star ring focus on spark technology development, is the AMP lab and databricks in the domestic technical partners, Together, we will contribute to the development of spark technology. ”