Newest member of Apace Twill:hadoop Club

Twill, formerly known as Weave, has now become one of the new members of the http://www.aliyun.com/zixun/aggregation/14417.html ">apache incubator Project, It is designed to simplify the operation of applications in Yarn/hadoop.

The fact that Hadoop is now a compelling technology solution is almost no doubt. The success of the project has been further expanded with the release of its 2.0 release, with the newly introduced yarn able to help almost all applications-not just the commonly used MapReduce job classes associated with Hadoop-to be posted on the Hadoop cluster in a simpler way.

But the problem remains, one of the most troubling of all is that Hadoop began to become hard to use after yarn. The fact that an application running through yarn must communicate or handle errors across platforms in its own unique way makes managers feel a bit headache.

As a major participant in the Hadoop camp, technicians from continuuity think it is necessary to launch their own set of libraries--a set called weave--designed to make it much easier to yarn and run applications. After the initial goals were successful, they decided to redouble their efforts to make weave a larger and more industry-influential project, in view of this, weave was presented to Apache incubator as one of the nurturing projects to continue to develop healthily. Since then, Weave has had his own new life and started to mount a new peak as an Apache project-the protagonist of our today's article, Apache twill.

When communicating with Continuuity CEO John Gray, ask them why they took the initiative to make the decision to hand over their project to Apache. The answer is simple: community.

"When we first launched the weave," he told me, "it was originally promoted through GitHub." However, in the GitHub, the results of the consumption of open source is very strong, but a relatively small contribution. This is not a site created by the community; it is ideal for a shared platform for results, but it is not suitable as a foundation for community building. "By contrast, Apache's approach is more in line with our expected process, with a software that creates a clear center for the user community."

"Open source options are diverse, but we want twill to be truly adopted, expanded and ultimately adapted to the real needs of the community." ”

For now, most of these requirements revolve around applications written by Java, but now the project has ushered in a new vision that will allow twill the ability to directly support other languages running in the JVM in the future. Python, for example, is a highly likely candidate language-but Gray thinks it is best for some members of the community to do so, such as those working for Python companies.

But the most common idea about twill is to further extend the scope of application-oriented applications. "The main users of Hadoop and yarn are still mapreduce," Gray said. "But twill can help a lot of open source applications that have not been developed through MapReduce. "This, in turn, will help to push for a shift in yarn and shape the future direction of the framework."