Store a large number of XML documents in relational databases

When saving XML documents in relational databases, you must have encountered many problems due to method reasons, especially for a large number of documents. This articleArticleThe specific problems related to a large number of XML documents will be raised, and then the methods to solve these problems will be provided.

Potential problems with a large number of documents

What makes an XML document very huge? One of the major signs of an XML document is that it encountered problems when it was started. Now let's discuss the potential issues.

The first is about raw space. The problem is related to the frequency and space. It is not that your documents are larger than the normal state, but that you use them frequently. You must have a good size Data Partition to store these documents (or databases ).

Second, running the database may produce errors. Sometimes XML documents are divided into multiple parts and stored in discontinuous tables that represent the structure of XML documents. However, XML documents are usually stored in blob, text, or varchar fields.

In addition to the lack of design, you may encounter problems on the interface for communicating with the database. For example, the vast majority of SQL executions limit data to 4,096 bytes-this means that when SQL is used to send 10 KB documents to the database, you will be blocked. In database storageProgramSimilar restrictions also exist. If you are not aware of these limitations and your documents are out of scope, you will encounter these problems.

Database Design

Developers and database administrators must have a clear understanding of the operations to be performed in the XML document so that they can correctly design the database. Understanding how databases will be used is equally important. Is your XML document a simple continuous storage mechanism? Or is it stored in a non-continuous state?

If you store the entire document in a whole block, you have multiple options. For non-continuous data, you should design a table that represents the XML structure and store the data separately in an independent table. The second step has the advantages of being Indexable and queryable, and you do not need to perform operations on the entire document.

Blob (Binary Large Object) is used to store large binary data. For example, encrypted files are often used in BLOB fields. Blob is very suitable for storing XML files. The disadvantage of the Blob field is that it is not Indexable and queryable, and it cannot exchange a large amount of data with BLOB through standard SQL.

Split

One of the solutions to SQL restrictions is to split XML documents into 4 kb or small blocks. These small pieces are stored in the table, and then linked using the ID field.

When you store these documents, you should split them into small pieces and place them in the table. When you return a document, you select all the blocks with a specific ID and then set them in the appropriate order. Although it is troublesome, this method is very useful for multiple databases.

Bundling and blob access

The problem with large BLOB data is that you cannot use SQL to store and obtain data. In recent years, every database merchant has its own storage and large object acquisition methods.

To store a large number of XML documents in a blob domain, you must use (Binding). Bundling is to extract data from the programCodeThe process of interacting with the domains in the database. By using this method, other functions of the database may be affected when the database is changed.

Summary

Storing a large number of XML documents in relational databases can cause many problems. The first step to solve these problems is to have a deep understanding of the parameters in the XML document. Then correctly design the appropriate solution. The solution can include a series of structure tables, which divide XML documents into small pieces, or bind them directly to the Blob field in the database.