Tips for businesses using Hadoop to handle large data

As a model of large data technology, Hadoop has always blessed and cursed the enterprise that uses large data. Hadoop is powerful, but very complex, which makes many companies prefer to wait for something easier to come out and launch big data projects. The wait is over. Hadoop is making steady progress, with significant ease-of-use enhancements from vendors such as Hortonworks and Cloudera, which have reduced the learning curve of Hadoop by half. Companies are increasingly embracing large data and Hadoop to migrate from basic ETL workloads to advanced data analysis.

But what more people don't know is that the trick that companies use Hadoop to handle big data is to start small.

The key to the use of Hadoop small start big data trip smaller? This seems to be a word that is not related to Hadoop. But it fully conforms to the reality of big data. We tend to talk about the advantages of Hadoop at PB and ZB level data, but most companies do not have a PB-scale problem. At the very least, they are not sure how to manage this level of problem.

Instead, a survey by Newvantage, a big data consultancy, shows that companies first focus on mastering new types of unstructured data. Gartner confirms this, saying: "Many organizations find large data diversity more challenging than gross or real-time." ”

As a result, smart Hadoop vendors are revising their strategy to help companies grow from small deployments and from there. Shaun Connolly, vice president of Hortonworks Enterprise Strategy, said in an interview with reporters:

"We've seen repeatable adoption patterns, starting with a focus on a new data type, and building or enhancing targeted applications around new data types." These new applications are typically driven by a line of business and start with data from one of the following new types: social media, click Stream, server log, sensor and machine data, geo-location data and files (text, video, audio, etc.).

"The eventual deployment of more applications and new data types leads to a broader modern data architecture." But successful customers start releasing value from specific types of data, then flushing them and repeating their journey from there. "For proving the value of Hadoop, it's a great way to start small, measurable projects that don't force businesses to swallow the entire elephant in the early stages." This is a clever strategy, so that powerful technology can be easily adopted.

As a result, Hadoop is becoming the "elephant in the room" that people really want to talk about. While more people are talking about big data, there are far fewer organizations that actually launch important big data projects, and Gartner stresses that only 8% of companies have actually deployed large data projects, although 64% of companies say they intend to. These companies value the real growth of the Hadoop Big Data Project, the achievable business values, not the hype of Hadoop.

In fact, most large data projects today tend to focus on incremental improvements to existing use cases, such as better understanding customer needs, making processes more efficient, further reducing costs, or better detecting risks. For all the talk about dramatically changing the business of an enterprise, most of the big data and the resulting deployment of most hadoop, the focus is on gradual improvement, not a radical change in the project.

That makes sense. The enterprise first uses Hadoop to implement projects that can be implemented in small steps, then master the technology and then bigger.

In 2014, we'll see that Hadoop is being used faster. Hortonworks's Connolly and Cloudera's Mike Olson all saw their business grow rapidly in 2013, and the last two quarters were growing faster. This acceleration reflects their improved marketing information, and it has been around how businesses can more easily derive value from Hadoop, while also demonstrating that the barriers to getting value from Hadoop have been lowered.