Big data technology is a technology to quickly obtain valuable information from various types of data.A large number of new technologies have emerged in the field of big data, and they have become powerful weapons for
big data collection, storage, processing, and presentation.
First. Big data access
1. Big data access
Existing data access, real-time data access, file data access, message record data access, text data access, picture data access, video data access
2. Big data access technology
Kafka, ActiveMQ, ZeroMQ, Flume, Sqoop, Socket (Mina, Netty), ftp / sftp
Second, big data storage
1. Big data storage
Structured data storage, semi-structured data storage, unstructured data storage
2. Big data storage technology
Hdfs, Hbase, Hive, S3, Kudu, MongoDB, Neo4J, Redis, Alluxio (Tachyon), Lucene, Solr, ElasticSearch
Third. Data analysis and mining
1. Big data analysis and mining
Offline analysis, quasi-real-time analysis, real-time analysis, picture recognition, voice recognition, machine learning
2. Big data analysis and mining technology
MapReduce, Hive, Pig, Spark, Flink, Impala, Kylin, Tez, Akka, Storm, S4, Mahout, MLlib
Fourth, big data sharing exchange
1. Big data sharing and exchange
Data access, data cleaning, conversion, desensitization, declassification, data asset management, data export
2. Big data sharing and exchange technology
Kafka, ActiveMQ, ZeroMQ, Dubbo, Socket (Mina, Netty), ftp / sftp, RestFul, Web Service
Fifth. Big data display
1. Big data display
Graphical display (scatter chart, line chart, histogram, map, pie chart, radar chart, K line chart, box line chart, heat map, relationship chart, rectangular tree chart, parallel coordinates, Sankey chart, funnel chart, Instrument panel), text display;
2. Big data display technology
Echarts, Tableau
The country plans to develop a big data industry development strategy. All industries need big data technology to support exponential data incremental services. More and more companies are gradually transforming into big data. The market demand for big data is exploding.
