Hadoop for. NET Developers

Hadoop for. NET Developers (i): Understanding Hadoop

Over the years, big data has become an exciting source of analysis for the industry. For the purpose of this blog series, I will loosely define the term refers to the expansion of data sources from the data core business system-the scope of traditional business intelligence-including new (either historically overlooked or new available) data sources.

This is a rough simplification of the big data, but it's inherent to the challenge of working with these new data platforms that have been pushed through the new data platform. When our attention is on Hadoop, the broadest recognition of these new data platforms, big data such a limited definition seems to suffice.

Hadoop is an elastic, distributed, modeless data processing platform that is ideal for you to have a large number of data sets, single-record containing a small number of values, such as log files, because it provides a low input for data access solutions. It is also a good platform for complex analysis and interpretation of complex data, such as XML or JSON documents, image files, etc., and/or may be subject to variable interpretations such as customer tweets (in JSON files).

In addition, when you need massive scalability beyond what can be achieved with traditional relational database platforms, Hadoop is a great platform. Having said that, I did not find this final solution to be applicable to many of my customers (though it is applicable to some). For my work and for the vast majority of people, the flexibility and economics of Hadoop are often the most compelling reasons to explore this platform.

Hadoop for. NET Developers (ii): Infrastructure

Hadoop is an implementation of a set of interrelated project components. The core component is MapReduce, which is used to process job execution, and a storage layer, typically implemented as a Hadoop Distributed File System (HDFS). For the purposes of this article, we will assume that HDFs is in use.

The components of Hadoop are implemented through a series of servers called data (or compute) nodes. These nodes are where the data is stored and processed.

The name of the node server retains the data node in the environment, its data is stored on which node on the track, and the node that provides the data is a single entity. This singular representation is called a cluster. If you are familiar with the RDBMS implementation terminology cluster, be aware that there is not necessarily any other resource between the shared storage or the node. The Hadoop cluster is purely logical.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Hadoop for. NET Developers