"Bdtc Sneak peek" HU: Large data storage time for space exchange

December 2014 12-14th, hosted by the China Computer Society (CCF), CCF large data Expert committee, the Chinese Academy of Sciences and CSDN co-organizer of the 2014 China Large Data Technology conference (DA data Marvell Conference 2014,BDTC 2014 will be opened at Crowne Plaza Hotel, New Yunnan, Beijing. The General Assembly lasts three days to promote the development of large data technology in industry applications. To set up a "large data infrastructure", "large Data ecosystem", "large Data Technology", "large Data Application", "large data internet finance technology", "intelligent information processing" and many other theme forums and industry summits. Sponsored by the China Computer Society, CCF large data committee of experts, Nanjing University with the co-organizer of the "2014 second CCF large data academic conference" will also be convened, and the technical conference to share the theme of the report.

The Conference will invite top experts and front-line practitioners in nearly 100 foreign data technology fields to discuss the latest development of OSS, YARN, Spark, Tez, HBase, Kafka, oceanbase, etc., Nosql/newsql, memory calculation, The development trend of flow calculation and graph computing technology, OpenStack ecosystem for large data computing needs, and large data visualization, machine learning/depth learning, business intelligence, data analysis, the latest industry applications, sharing the actual production system of technical characteristics and practical experience.

NetEase ntse/tnt engine leader Hu will be in "Big Data Technology" sub-forum, bring "NetEase Database Compress Technology" speech. Efficiently compressing the data in the database is an important way to deal with the rapid growth of the data volume. At present, the industry's mainstream database products provide compression, however, there is still room for improvement in the actual effect. NetEase in the field of database research and development has many years of practical experience, this sharing will be combined with the data storage engine, the introduction of NetEase in the database data compression technology.

Before the meeting, Csdn and Hu had a simple communication. Hu that data compression technology can obviously reduce the pressure of data storage, is a time for space transactions, in which way to choose between the two, and how to improve the efficient compression management is a need to carefully consider the issue.

Hu, manager of NetEase NTSE/TNT engine

2012 joined NetEase, NetEase ntse/tnt engine leader, currently focused on the research and development of NetEase database engine and distributed database middleware, keen on a variety of database technology.

CSDN: What position does data compression technology occupy in large data technology? What are the difficulties in the practice of data compression?

HU: The Big Data Age presents a new challenge to the old computer technology of data storage, which requires a large amount of storage equipment to support the PT,EB data. While the cost of storage, which is booming in storage hardware, has dropped dramatically today, in a single system, internal and external savings are still one of the most significant costs of a database system. Data compression technology obviously reduces the pressure of data storage. Database compression technology is actually a time for space transactions, in which way to choose between the two, and how to improve the efficient compression management is a need to carefully consider the issue.

CSDN: What large data technologies have you used in your company? What are some of the areas where you are satisfied with these technologies and where are you dissatisfied?

Hu: HBase, Hive, NoSQL, and so on, scalability is very good, but in the ease of use and in some common scenarios, performance can be improved.

CSDN: According to your understanding, at present similar enterprise, in the data aspect, encounters the biggest difficulty is what?

HU: Data is the most important wealth of enterprises, but also the most fundamental safeguard, so the data should be based on stable, efficient and mature technology to build a set of data platform program. Consolidate and manage all the data, the different departments, different applications of the Times to the management and operational dimensions of the great challenges. In the selection of data storage and management tools, we should actively embrace the open source community and work with other developers to create a sustainable development path.

CSDN: What are some of the technologies you are looking at and studying in the Big data field?

Hu: Phoenix is very interesting to me, it provides hbase similar to the use of relational databases, support JDBC, and in the near future, such as multiple table connection, level Two index, and so on. Relational mode is a very friendly way for a database, Phoenix let HBase have NoSQL extensibility and RDBMS of ease of use, optimistic that it will have better development.

CSDN: Please talk about the topic you are about to share at this conference.

HU: Data compression is the most products in the database field of the practice is to use a mature compression technology to the data file fragmentation block compression. The result is to obtain a still acceptable compression effect, but in the flexibility and compression ratio can continue to improve the space, netease in the process of the storage engine is also hoping to be in the existing product compression effect can be further. In the area of data compression, our engineers combine the storage database, the traditional row-level database compression technology, and based on their innovation, and achieved good results. We will share with you the questions we have thought about in the design selection process and the challenges we have encountered, hoping to inspire and help the audience.

CSDN: Which listeners should know these topics best? What topics can you share to help your audience solve problems?

HU: Engineers involved in the development of data storage systems may be interested in my sharing, and I hope this sharing will provide some new ideas for how these engineers will do data compression when they develop data storage systems in the future.

The National large Data Innovation Project selection activity is now in full swing, details click here.

The 2014 China Large Data Technology Conference (Marvell conference 2014,BDTC 2014) will be held at Crowne Plaza Hotel, New Yunnan, December 12, 2014 14th. Heritage since 2008, after seven precipitation, "China's large Data technology conference" is currently the most influential, the largest large-scale data field technology event. At this session, you will not only be able to learn about Apache Hadoop submitter uma maheswara Rao G (a member of the project Management Committee), Yi Liu, and members of the Apache Hadoop and Tez Project Management Committee Bikas Saha and other shares of the general large data open source project of the latest achievements and development trends, but also from Tencent, Ali, Cloudera, LinkedIn, NetEase and other institutions of the dozens of dry goods to share. There are a few discount tickets for the current ticket purchase.

Free Subscribe to the "CSDN large data" micro-letter public number, real-time understanding of the latest big data progress!

CSDN large data, focus on large data information, technology and experience sharing and discussion, to provide Hadoop, Spark, Impala, Storm, HBase, MongoDB, SOLR, machine learning, intelligent algorithms and other related large data views, large data technology, large data platform, large data practice , large data industry information and other services.