Technical home: The big Data age can affect your 7 business trends

We have seen many of these cases where companies are willing to share their achievements in the use of large data. In any paradigm shift in the IT Industry (Paradigmshift), a specific theme attracts a lot of attention from news media, investors and innovators, a shift that requires strong commercial price support. This typical case is: customer service, distributed computing, and service-oriented architecture and language, such as JAVA.

We have also seen a beneficial ecosystem of emergence, rapid praise or expansion capabilities of the core support technology, in large data cases, large data ecosystems have rapidly focused on a group of technology providers.

So what trends can I see in large data ecosystems?

On Hadoop for SQL extensibility and consistency

A large number of technology companies are trying to build a no-sql technology to provide solutions for big data such as Hadoop. However, the depth and breadth of SQL language support varies, but it is possible to use these benefits with SQL professional analysts to manipulate large data in SQL language.

(Note: Since the current large data store is not based on relational databases, the traditional way of manipulating data through SQL is not directly available, for example: data stored in Hadoop cannot be queried directly through SQL.) Thus, the traditional SQL language needs to be intermediate to operate, for example: Hive in Hadoop is equivalent to converting SQL to MapReduce to read and manipulate data on Hadoop. )

Unified support for structured, unstructured and semi-structured data

The growth of unstructured data at any time, IDC forecasts the number of data, most of which will be stored in unstructured form, and will grow by 40% to 50% a day. By 2020, the total amount of data will reach 40ZB. Unstructured data comes mainly from: Mail, forums, blogs, social networks, POS system and machine generated data. To capture and analyze these large data volumes, innovators must expand their large data solutions, not just one.

Optimizing Search

Finding out the real search needs of users before they can be seen from a huge amount of data is almost impossible before they look like a needle in a haystack. But at any time, more and more large data solutions are integrated into the retrieval support. In this respect the leader is: Lucidworks,ibm,oracle (its acquisition endeca) Autonomyandmarklogic. Where Lucidworks combines an open-source heap of Lucene and Solr,hadoop,mahout and NLP.

ETL Extension and Support

Many people think that the first use of Hadoop security is for ETL because of its batch functionality. However, if you see all the infrastructure of a complex Hadoop platform based on an ETL solution, you can use other pure-informatica,talend,syncsort,cloveretl ETL tools to solve it. Over the years these companies have struggled to build the most value combination of ETL solutions, and now more we call it: data consolidation solutions.

Pure ETL providers are trying to provide solutions for large data. These support is easy to include: ETL, and include elt that translates from inside Hadoop to Hadoop. This allows the company to use a build environment that uses pure ETL solutions and the powerful features of Hadoop itself. With the development of time, these pure ELT companies have supported large data solutions ranging from: Newsql and NoSQL.

In addition, I expect many large data solutions companies to embed support for ETL and ELT, just as many traditional database vendors have been embedding or acquiring ETL solutions.

Large Data movement stabilized

In my previous article, Hadoop, which uses Apache as an open source framework, has been used in a batch-oriented, distributed environment, especially in the context of analysis. At any time, companies are beginning to focus on how to dominate and use large amounts of data resources for real-time decision making, and we anticipate significant help for the ' Big Data movement ' impact and growth. This "landing" represents the real-time flow of information used to handle large streams of data in various industries: Capital markets, healthcare 7, energy and social media.

Increase data mining and analysis techniques

Industry segments in large data areas know the need to expand data analysis and statistical capabilities on their platforms. In addition to the general analysis function also adds the very data mining function. Teradataaste includes many analytic functions, including support statistics, text mining, image, emotion analysis, etc. Other companies, such as Ibmnetezza, have added support for R, which supports all types of r packages, such as parallel algorithm packages and matrix-related packages. In the future, we can see that large data solutions will continue to increase this functionality in large numbers.

Profiting from the R language

There is no doubt that the R language will be the more popular open source statistics language. Revolutionanalytics Company in the development of "industrial" use of the R version, the performance is significantly enhanced and to meet the characteristics of other enterprises. In a bit more, they've developed r expansion packs that can be applied to Hadoop and Puredata. The university also offers a wide range of R-language courses that allow more students to have the ability to use R, as well as their ability to deal with complex statistical analysis. It can be foreseen that R will be included in many large data solutions, and will significantly improve the language for better performance.

With the development of large data ecosystem, the related industry must be accompanied by its development. In today's competitive environment, companies that implement data-driven strategies will gain a competitive advantage.