Apache Streaming

Apache Pig, a high-level query language for large-scale data processing, works with Hadoop to achieve a multiplier effect when processing large amounts of data, up to N times less than it is to write large-scale data processing programs in languages ​​such as Java and C ++ The same effect of the code is also small N times. Apache Pig provides a higher level of abstraction for processing large datasets, implementing a set of shell scripts for the mapreduce algorithm (framework) that handle SQL-like data-processing scripting languages ​​in Pig ...

Spark is a memory-based, open-source cluster computing system designed for faster data analysis. Spark was developed using Scala by Matei, AMP Labs, University of California, Berkeley. The core part of the code is only 63 Scala files, which is very lightweight. Spark provides an open source clustered computing environment similar to Hadoop, but Spark performs better on some workloads based on memory and iteratively optimized designs. & nbs ...

During the year, we saw that many vendors focused mainly on integrating Hadoop or NOSQL data processing engines and improving basic data storage. The most successful thing about Hadoop is that it uses MapReduce. MapReduce is a programming model for processing Super large datasets and generating related execution, MapReduce's core idea is to draw lessons from the function is the programming language and the character of the vector into language. Today includes Microsoft, IBM, Oracle, Cloudera, mapr ...

1. 　　 　　Download Helix.server, the open source version of the real server, the RM media streaming server, without any restrictions. Support The RTSP agreement, support Rm,mp3 and other formats, even if it is commercial, can cope with. You can download it here. 2. Installation decompression: #tar-zxvf helix_dna_server_10.1.tar.gz directory helix_dna_server_10.1 under current path ...

Hadoop is a large data distributed system infrastructure developed by the Apache Foundation, the earliest version of which was the 2003 original Yahoo! Doug cutting is based on Google's published academic paper. Users can easily develop and run applications that process massive amounts of data in Hadoop without knowing the underlying details of the distribution. The features of low cost, high reliability, high scalability, high efficiency and high fault tolerance make Hadoop the most popular large data analysis system, yet its HDFs and mapred ...

The authors observed that http://www.aliyun.com/zixun/aggregation/14417.html ">apache Spark recently issued some unusual events databricks will provide $ 　　14M USD supports Spark,cloudera decision to support Spark,spark is considered a big issue in the field of large data. The beautiful first impressions of the author think that they have been used with Scala's API (spark).

Hadoop Here's my notes about introduction and some hints for Hadoop based open source projects. Hopenhagen it ' s useful to you. Management Tool ambari:a web-based Tool for provisioning, managing, and Mon ...

Star Ring Technology's core development team participated in the deployment of the country's earliest Hadoop cluster, team leader Sun Yuanhao in the world's leading software development field has many years of experience, during Intel's work has been promoted to the Data Center Software Division Asia Pacific CTO. In recent years, the team has studied large data and Hadoop enterprise-class products, and in telecommunications, finance, transportation, government and other areas of the landing applications have extensive experience, is China's large data core technology enterprise application pioneers and practitioners. Transwarp Data Hub (referred to as TDH) is the most cases of domestic landing ...

The Big data field of the 2014, Apache Spark (hereinafter referred to as Spark) is undoubtedly the most attention. Spark, from the hand of the family of Berkeley Amplab, at present by the commercial company Databricks escort. Spark has become one of ASF's most active projects since March 2014, and has received extensive support in the industry-the spark 1.2 release in December 2014 contains more than 1000 contributor contributions from 172-bit TLP ...

(1) The Apache Hadoop version introduces Apache's Open source project development process:--Trunk Branch: New features are developed on the backbone branch (trunk); 　　-Unique branch of feature: Many new features are poorly stabilized or imperfect, and the branch is merged into the backbone branch after the unique specificity of these branches is perfect; --candidate Branch: Split regularly from the backbone branch, General candidate Branch release, the branch will stop updating new features, if the candidate branch has b ...