Data Archiving and Backup

Two kinds of strategies
Archiving and backup for a system, there are two different strategies that play two different, complementary functions. Backups are used for high-speed replication and recovery to reduce the impact of failures, personnel errors, or disasters, archiving to effectively manage data, retention of data, and long-term access and retrieval.

Archiving Requirements
Archive data range: Primarily DB data and file data. Call data (call data, service record, recording data, also consider mail, fax, IVR message file and business attachment), business data (business records, business documents, etc.), customer information is not archived;

Archiving mode: Automatic archiving or manual operation of the archive.

Data life: How long the business data is retained for archiving. Consider the size of the data volume.

How data is accessed and retrieved after archiving:

Archiving data cleanup: The archived data that needs to be purged is data that has no reserved value. Delete the freed resources directly ...

Method of Thinking
1, the Business table and historical data design into different tables to store data. Only business data that is being processed is retained in the business table. After the processing is completed, the historical data is formed and automatically enters the historical data table.

Analysis: The idea is to separate the data being processed by the system from the historical data that needs to be queried. can guarantee the performance of the system running. However, at present, the C6 scheme is not considered in the design.

2, using the previous ideas, still divided into business table and historical data table, but not every transaction after the completion of the automatic formation of historical data, but in the business table for a period of time (depending on the amount of data and actual demand), the system is regularly archived or manually in the interface operation to archive, the business data transfer to the historical data table.

Analysis: Data being processed and user-focused data in the short term remain in the business table, periodically archiving historical data.

Both of these ideas need to consider procedures to achieve archiving operations, and provide an archive data query interface. This is not difficult to achieve. How files are archived.

3, directly open two sets of systems (completely consistent), two sets of database instances. One for the business system and the other for the filing system. Periodically transfer the previous set of data directly to the latter set. When you need to view archived data, direct access to the archiving system.

Analysis: It's quite simple. But feel not friendly enough. For example, some common data is not required to be archived, such as customer data, code parameter class data tables and other public resources. But it must be in both systems.

Determine the design

1, the business table needs to have a corresponding history table;
2, the archive provides interface operation, by the operation user decides to file according to the time or according to the data classification.
3, after the archive of data to provide export or clear function.
4, file data does not provide archiving, such as: fax, mail, message and other documents. But provides cleanup functionality. After archiving, the file remains, but the file is purged when the archive data clears.
5, the Business table and the history of the separate, do not consider the business operation of the historical data query. means the operation of business data and historical data is also separate. A query interface that provides historical data separately.
6, do not provide the reverse operation of the archive.
7, Backup: Using the operating system scheduling (cron or Scheduled Tasks) +export backup strategy. Specific scenarios are issued by the DBA.