[Big Data paper note] overview of big data Technology Research

Basic concepts of big data:

1. Generation of big data

A. Scientific Research

B. Iot applications

C. Generation of massive network information

2. Proposal of the big data Concept

3. 4 V features of big data

A. Volume (Large capacity): massive data volume and data integrity

B. Variety (multiple types): internal associations must be discovered between massive and diverse types of data.

C. velocity (FAST): faster to meet real-time requirements

D. Value (low value density): converts information into knowledge

4. Big data application fields

A. Business

B. Finance

C. Healthcare

D. Manufacturing

Big Data Processing Process

 

1. Data collection

2. Data Processing and Integration (filtering)

3. Data Analysis (CORE)

4. Data Interpretation (data visualization)

Key big data technologies

1. Cloud computing and mapreduce

Cloud computing:

A. Service IAAs

B. Platform as a service PAAs

C. software as a service (SAAS)

Mapreduce:

 

2. Distributed File System

GFS: the master-slave architecture (master-slave) is used to achieve high-speed storage of massive data through data blocks, append updates, and other methods.

3. Distributed Parallel Database

Bigtable:

Nosql:

4. Open-Source implementation platform hadoop

5. Big Data Visualization

 

Challenges brought by big data:

1. Security and Privacy of big data

2. Big data integration and management

A. Data Storage

B. Data cleansing

3. IT technology architecture of big data

A. Big Data Analysis Technology

B. Data Fusion

C. Big Data energy consumption problems

4. Ecological Problems of big data

[Big Data paper note] overview of big data Technology Research