hadoop2.6.0 rollup: New features latest compilation 32-bit, 64-bit installation, source package, API download and deployment documentation

Related content:

hadoop2.5.2 rollup: New features newly compiled 32-bit, 64-bit installation, source package, API, eclipse plugin download
Hadoop2.5 Eclipse plugin maker, connect to cluster video, and hadoop-eclipse-plugin-2.5.0 plugin download
hadoop2.5.1 Summary: Latest compilation 32-bit, 64-bit installation, source package, API download and new features, etc.

Beginner's Guide: Introduction to Hadoop website and how to download Hadoop (2.4) Versions and view Hadoop API introduction


From zero teaches you how to compile hadoop2.4 in a Linux environment (Ubuntu 12.04)






First, the download area

Hadoop-2.6.0-src.tar.gz "maven Package"
Link: http://pan.baidu.com/s/1gdEJVSf Password: Ixet


hadoop--2.6.0.tar.gz "official website download"
Link: Http://pan.baidu.com/s/1kTLxwZP Password: 9gp0

Hadoop-2.6.0-64.tar.gz
Link: Http://pan.baidu.com/s/1ntn3voH Password:

Content hidden in this post ISN2

Hadoop2.6-api
Link: http://pan.baidu.com/s/1pJLpmOz Password: 7PMA

Hadoop2.6-eclipse Plug-in
Because the Hadoop eclipse plugin is related to the Eclipse version, it's best to compile it yourself
Hadoop2.5 Eclipse plugin maker, connect to cluster video, and hadoop-eclipse-plugin-2.5.0 plugin download


II. Deployment Documentation

build hadoop2.6.0 ha and yarn ha

Hadoop 2.6.0 Single-node-pseudo-distributed mode installation





third, Apache Hadoop 2.6.0 new features


Apache Hadoop 2.6.0 released, the new stable version, the release frequency and quality is getting higher, and added a lot of things, from the size of the installation package can be seen, directly increased the 50m,30% have wood.
Here's a look at what good things are in 2.6.0.
Common:
1. Hadoop key Management Server (KMS) is a key Management Server written based on the Hadoopkeyprovider API. He provides a client and a server component that uses the rest API communication between the client and server based on the HTTP protocol. The client is an Keyprovider implementation that interacts with KMS using the KMS HTTP REST API. KMS and its client have built-in security mechanisms that support HTTP SPNEGO Kerberos authentication and HTTPS secure transport. KMS is a Java Web application that runs on a pre-configured Tomcat server bundled with the Hadoop release.
2. Tracing
HDFS-5274 added the ability to track requests via HDFs, which uses an open source library, Htrace. We can look at the Htrace, the function is very powerful, Cloudera open source out.
Hdfs:
1. The Transparent Encryption,hdfs implements a transparent, end-to-end encryption method. Once encrypted is configured, the process of reading data from HDFs to decrypt and write data encryption is transparent to the user application code strip. The encryption process is end-to-end, which means that the data can only be decrypted by the client encrypted. HDFs is never stored, nor does it access unencrypted data and data encryption keys. This satisfies the two typical requirements of the encryption process: At-rest encryption (static encryption, that is, data persistence on a medium like a hard disk), In-transit encryption (in-transit encryption, for example, when data is transmitted over a network).
2. Storage ssd&& Memory. Archivalstorage (archive memory) is the separation of computing power from the growing capacity of storage. High-density, low-cost storage, but less compute-capable nodes become available, and cold storage can be done in the cluster. Adding more nodes as cold storage can improve the storage capacity of the cluster, independent of the computing power of the cluster.
Mapreduce
This section is mainly about bug fixes and improvements. Two new additions have been added, as described in 2.5.2. Here's a quick look.
1. Resourcemanger Restart
2. Allow am to send historical event information to timeline server.
YARN
1. NodeManager Restart: This feature enables NodeManager to restart without losing the container of the activity running in the node.
2. The Docker Container Executor:dockercontainer Executor (DCE) allows yarn nodemanager to start yarn Container in Docker Container. Users can specify the image of the Docker they want to use to run yarn container. These container provide a customizable software environment where the user's code can be run and isolated from the environment in which the NodeManager runs. These container that run user code can contain the specific libraries that the application needs, and they can have different versions of Perl,python or even Java than the NodeManager. In fact, these container can run different versions of Linux than the NodeManager OS. Although yarn container must define all the environments and libraries required to run the job, all of the things in NodeManager are not shared.
Docer provides yarn with both consistent and isolated modes, consistent mode in which all yarn container will have the same software environment, in isolation mode, regardless of what is installed on the physical machine without interference.


Fourpublish Apache Hadoop 2.6.0 Highlights-heterogeneous storage and hadoop2.7.0 Outlook

Publish Apache Hadoop 2.6.0
--heterogeneous storage, long-running service and rolling upgrade support

I am pleased to announce that the Apache Hadoop community has released the Apache 2.6.0:http://markmail.org/message/gv75qf3orlimn6kt!

In particular, we are pleased with the three major films in this release: heterogeneous storage using SSDs and memory tiers in HDFs, support for long running in yarn services and rolling upgrades, upgrade your cluster software, and then restart upgraded nodes without shutting down the cluster or losing the work in progress. Yarn as its architecture center, Hadoop is constantly attracting new engines to run in the data platform as an organization that wants to efficiently store data in a single repository and interact with it in different ways at the same time.

Thank you very much for all the contributors and the people who have worked with this release, there are nearly 900 Jira issues solved in four ways:
Hadoop General: 231 Jira Problem Solving
Hadoop's hdfs:305 a jira problem solving
Hadoop's yarn:290 a jira problem solving
Hadoop's MapReduce: 70 Jira Problem Solving

The highlights of Apache Hadoop2.6.0

Here are some details about the most important features. For a complete list of features, improvements, and bug fixes, see Release Notes: Http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/releasenotes.html.

Enhanced HDFs support for heterogeneous storage tiers

Administrators can store data to these different tiers of storage in a qualified datanode across disk storage tiers, as well as APIs that the application can take advantage of. This means that administrators can optimize their applications by using Hadoop to run:
• Increase read/write latency on SSD storage tiers
• Memory storage layer for fast read/write applications (such as Spark, Tez, etc.) that are either temporary data or fail
• Archive storage tiers for increased storage efficiency.

Support for long-running services in yarn

Apache Hadoop2.6.0 includes enhancements to the core Apache Hadoop yarn platform to make long-lived services (such as Apache Storm,apache Samza,apache Kafka or Apache HBase), Can run in yarn and take full advantage of its benefits of fault tolerance, security and ease of maintenance.

Apache Hadoop originally architected to support batch processing of data. However, some applications are "always online" and ready to process input data. For example, Apache Storm must be prepared to process data streams in real time at any time of the day, any day of the year.

With Hadoop2.6.0, clusters can now take advantage of the same infrastructure arrangements to execute and manage multiple workloads for all deadlines. Long-lived services such as Storm and hbase can coexist peacefully together at a specific point in time (such as Apache Hive or Apache Pig) for ad hoc working applications.

Rolling upgrade works in yarn, leaving a reboot

The new work, maintenance Restart feature allows the application to keep its complete and ongoing state in the face of a node failure or reboot. Yarn can now provide scrolling with minimal quality of service degradation used to run the application's upgrade support. The progress of the application work node that has completed or is in progress remains unchanged during the restart process, without having to restart all tasks from the beginning.

Outlook Apache Hadoop2.7 version

The main driving force for the next version of Apachehadoop is the jdk7+ that we now require to use JDK7 (hadoop-10530:https://issues.apache.org/jira/browse/ HADOOP-10530) Apachehadoop forward, also supports JDK8 as a runtime (hadoop-11090:https://issues.apache.org/jira/browse/hadoop-11090).

Other important activities undertaken in the Apachehadoop community are:
• Erasure code support in HDFS-hdfs-7285:https://issues.apache.org/jira/browse/hdfs-7285
• Resources that support disk YARN scheduling and isolation-yarn-2139:https://issues.apache.org/jira/browse/yarn-2139
• Container Resource delegation extended YARN resource Management-yarn-1488:https://issues.apache.org/jira/browse/yarn-1488

As always, you can follow along with the development of Wiki:http://wiki.apache.org/hadoop/roadmap Apache Hadoop by tracking the roadmap.

Related content:

hadoop2.5.2 rollup: New features newly compiled 32-bit, 64-bit installation, source package, API, eclipse plugin download
Hadoop2.5 Eclipse plugin maker, connect to cluster video, and hadoop-eclipse-plugin-2.5.0 plugin download
hadoop2.5.1 Summary: Latest compilation 32-bit, 64-bit installation, source package, API download and new features, etc.

Beginner's Guide: Introduction to Hadoop website and how to download Hadoop (2.4) Versions and view Hadoop API introduction


From zero teaches you how to compile hadoop2.4 in a Linux environment (Ubuntu 12.04)

hadoop2.6.0 rollup: New features latest compilation 32-bit, 64-bit installation, source package, API download and deployment documentation