Design of the column family database

1. Design Essentials related to the column family database

• The column family database should be a sparse and multidimensional map beforehand
• In a column family database, the columns owned by each row of data can be different
• Columns of a column family database can be added dynamically
• The column family database does not need to perform a connection operation, and the data model needs to be normalized

2. Recommendations to be followed when designing data tables

• To use normalization to replace connections
• Use a column with no value
• Store data in both column names and column values
• Model A single entity with a single row of data
• Do not say a large number of operations are assigned to a small number of servers when designing row keys
• Maintain an appropriate number of column value versions
• Do not use complex data structures in column values

3. How do I avoid connection operations when using the column family database?

In the column family database, many-to-many relationships are represented by a normalized data model.

4. Why should entities be modeled as a single row of data?

Entities can represent specific things, or they can be expressed as abstract concepts. The column family database uses data rows to model the entity.

A data row should correspond to an entity. Data rows are distinguished by row key.

and the column family database does not need to perform the connection operation, the data model needs to normalize processing.

The column family database does not provide the same transaction control mechanism as the relational database level.

In general, writes to data rows are atomic operations. If you update more than one column in a table, the columns are all updated, or one cannot be updated.

5. What is a hot spot (hotspotting), and why should you avoid this phenomenon?

Distributed systems allow us to use many servers to solve problems, but if you put a lot of tasks on a few servers, it will make

Generating hotspots (hotspotting) in distributed systems can lead to the failure of other servers to be fully utilized and the benefits of distributed systems.

6. What are the drawbacks of using complex data structures as column values?

Using different columns to store different attributes makes it easier to use the properties of the column family database on these attributes.

Placing attributes in different columns also helps you to divide the column family according to requirements, and to use indexing to optimize performance.

Of course, using complex data structures does not have these advantages.

7. Say three scenarios in which you should not use secondary indexes.

1. There are few cardinality values in the column, and indexes do not help improve performance and sometimes even damage performance

2. Too much technology in the column

3. When many data rows are not used for a column, it is not significant to have a secondary index for that column

8. What are the two categories of statistics? What are their respective uses?

Statistics is a branch of mathematics that studies how to describe large datasets (overall) and how to make inferences from data.

It is divided into descriptive statistics and two kinds of predictive statistics .

Descriptive statistics help us understand the composition of the data and make it easier for us to compare a dataset to other data.

Predictive statistics studies how to make predictions based on data.

9. What are the two types of machine learning? What are their respective uses?

The method of machine learning involves many other disciplines, such as computer science, artificial intelligence, statistics, and linear algebra.

There are many services behind the machine learning technology to support, such as according to past purchase behavior to recommend products to customers, detect network fraud, machine translation and so on.

Including

1. Unsupervised learning helps us to explore huge datasets

2. Supervised learning enables programs to learn from sample data

What is the difference between Spark and MapReduce?

MapReduce needs to write large amounts of data to the disk, and Spark consumes a lot of memory.

MapReduce uses a relatively fixed computational model, first map to reduce, and the computational model used by Spark is more general.

Design of the column family database