Another fire in the integrated Gemfire,hadoop biosphere

Ivotal strives to integrate and improve the large data technologies bought by its parent company to meet the challenges of large data enterprise applications.

Pivotal is a new company that is split and reorganized by VMware's Cloud foundry and EMC's Greenplum, and aims to bring a restructured, rigorously tested Hadoop to the enterprise, which has put Apache Hadoop 2.2 Technology is incorporated into its new pivotal HD 2.0 version, while also integrating the memory database GemFire XD in this release.

GemFire uses cloud computing platform virtualization technology to centralize the memory of several X86 servers and form a pool of memory resources up to dozens of TB, loading all data into memory for memory calculations. The calculation process itself does not require a read-write disk, but periodically writes data synchronously or asynchronously to disk. GemFire stores multiple data in a distributed cluster, any machine fails, backup data on other machines, no worries about data loss, and disk data as a backup.

Pivotal hopes to use stable technology to provide a data solution for large companies, and to establish periodic positive feedback mechanisms for the data generated by large companies, such as mobile phone operators who can use the large data and analytics capabilities of Pivotal HD and GemFire HD to determine the fastest call routing, If the call is degraded or failed, the feedback is returned for processing so that the problem can be resolved in a timely manner. In a perfect large data environment, timely apology to the user can also leave a good impact on the customer.

Benefiting from the Apache Hadoop 2.2 update, Pivotal HD 2.0 now supports NFS and snapshot processing, which means that enterprise customers can roll back when problems occur.

Pivotal HD 2.0 also has improved database performance, enhanced HAWQ SQL query performance, its database engine is based on Greenplum database and made some improvements. HAWQ can now apply the algorithms in more than 50 databases in the Madlib machine learning Library, and the database engine now supports automated translations based on R, Python, and Java language queries and applications, so HAWQ can handle business logic well and control the process well with SQL.

Pivotal may not be the first commercial company to benefit from the Apache update, but it is the first company to claim the most rigorous testing of open source code. Michael Cucchi, senior director of the pivotal product marketing department, told us: "We're using the Apache release and hardening it, doing our own QA and regression tests, and trying to implement rollback operations on 1000-node clusters, Our tests were done on a large scale cluster. ”

The new version of Pivotal HD 2.0 integrates GemFire HD for the needs of real-time analysis, and also incorporates Graphlab graph analysis technology, which also adds an improved HAWQ SQL query engine to cope with the huge workload.

This is nothing.--pivotal is simply consolidating the technologies that are available in a series of acquisitions by EMC and VMware, enabling enterprise customers to easily and truly value the use of Hadoop on a large scale. For example, the HAWQ SQL query tool came from EMC2010 's acquisition of Green Plum in the year, GemFire from VMware's acquisition of Gemstone in the same year.

Cucchi told us: "Initially pivotal HD software is supported on bare metal, can also be run in VMware environments, and can be bundled with hardware as a device, but in the future if users want to run pivotal HD on AWS or other public cloud. ”

This is indeed an excellent opportunity, which allows pivotal to even contend with the new Hadoop powers such as IBM and Cloudera and Hortonworks, in other words, in the future these companies will face such strong rivals as pivotal.

Original link: Pivotal spiffs up Hadoop and GemFire to meet enterprise the big data challenges