Text Database-a wonderful future in the. Net industry

Text storage, which is a field that I seldom implement in the past.

 

He will talk about the text database. From the previous article: Analysis of QBlog technology principles of the autumn color Garden: Performance Optimization: read/write splitting and Text Database (18th), I saw that it was out of application, so I had a few impulse.

 

Of course, from the above, we can see that a simple text-based read/write application is not very close to the database, and the realm is not enough.

 

In my imagination, the "Text Database" should be as simple as possible:

 

1: storage structure [table structure]

2: If you have a primary key ID, instead of a GUID, you have to use the auto-increment ID ).

3: add, modify, and delete data.

4: it is better to query, sort, and pagination. The grouping may require a higher level.

5: it is better to have concurrency control.

 

Looking back, the current application of QBlog in the autumn garden is:

 

1: less text storage content: one text is not stored in a table, but basically one row (occasionally multiple rows ).

2: Based on, it can only be said to be a text reading application, and it is not related to the "Database.

 

Therefore, the above applications cannot reach the realm of text database applications.

 

Of course, with text-based applications, it is natural to naturally have a little interest and impulse to the expanded "Text Database" of the text.

 

So I searched for "Text Database" on the Internet and found that the. net field was almost absent, but it was a big difference in the php field. Why?

 

Why? It is estimated that the following content is used:

 

Basic advantages of using text as a database:

 

1: simple text operations are faster than databases:

I have to explain that I can only say this: for disks that are stored by everyone, direct storage is certainly faster than storing disks after Database Rules bypass certain storage structures. Of course, the more complex the database will be, that's hard to say.

2: Easy access:

You don't need to worry about ADO. NET. You can do it with System. IO. File!

 

Basic disadvantages of using text as a database:

 

1: The data volume is not suitable for big data.

2: It seems that concurrency is not good.

3: deletion and modification are not easy to operate

4: there are many things to deal with to transform into a "Text Database:

A: What about the auto-increment ID?

B: What is the query result?

C: What is the sorting order?

 

Basic application scenarios of text databases:

 

1: small applications, mainly text databases:

A single (table) text data volume is small: less than 0.1 million pieces of data, less than 10 MB (one-time loading into the memory, it becomes a memory database, the speed is great ).

2: large applications, supplemented by text:

By spreading some databases to scattered texts, the pressure on the primary database is reduced.

3: Medium-sized applications, mainly text databases:

This is hard to say, not good, and requires strong technical support.

 

 

In order to give full play to the initiative of "Text Database", I spent some time researching and thinking about it. I will share my experience with you below:

 

1: Storage Structure

 

I got a prompt from the text application of QBlog in the autumn garden. It's nice to save it as json. Isn't it popular? I can read it directly to the front end. What do I like!

So the technical point here is: How to output json and read json parsing.

Here: In the open-source CYQ. Data V3.0 framework (download), there is a JsonHelper that can generate Json Data and parse the json string into a key-value correspondence. If you are interested, you can go to research.

Therefore, the basic read and fetch operations are solved.

 

2: Data insertion

 

Use: System. IO. File. AppendAllText to easily add a line of json to the end of the text.

 

3: update and delete

 

These two operations are almost the same. I think of two methods:

Method 1: simple [This is actually quite good, so from the perspective of text database application scenarios, the basic requirements are not too high]

Output The json of the entire table and rewrite the text again.

Method 2: balanced [This is more performance considerations, but it is a bit too much for text databases, because if it is too complicated, why not look for other databases, isn't it easy to use text?]

This is quite painful. Here we also provide some personal ideas:

1: When determining the table structure, you must specify the length of each field, so that you can determine the total maximum length of a row.

2: When writing a row of data, the total length is not long enough, and the end is blank (as if it is usually written to \ 0 ).

3: During update or deletion, locate the start write position based on the (ID-1) * total length of the row and then rewrite the line, if the row is deleted, the row is empty (\ 0 ).

In fact, the space is changed for time, and the text size does not change during data deletion. Is it a bit like access?

 

Method 2 above the needle. Here is the sample code:

 

FileStream fs = File. Open (File path );
Fs. Seek (locate the start position for writing, SeekOrigin. Current );
Fs. Write (... Write content ...);
Fs. Close ();

 

4: How can I find the auto-increment ID?

 

It is easier to add the ID. The value of the first field + 1 in the last line of the text is read. Of course, it is only applicable to the number type, and then global cache, read ++ each time.

 

The following is a reference sample code:

 

/// <Summary>
/// The next auto-increment ID
/// </Summary>
Private int NextID
{
Get
{
Lock (lockNextIDobj)
{
If (maxID> 0)
{
MaxID ++;
}
Else if (DataType. GetGroupID (Table. Columns [0]. SqlType) = 1) // The auto-increment ID is only valid for int
{
If (Table. Rows. Count> 0)
{
Int lastIndex = _ Table. Rows. Count-1;
Do
{
If (lastIndex> = 0)
{
If (_ Table. Rows [lastIndex] [0]. IsNull)
{
LastIndex --;
}
Else
{
MaxID = Convert. ToInt32 (_ Table. Rows [lastIndex] [0]. Value) + 1;
}
}
Else
{
MaxID = 1;
}
}
While (maxID = 0 );

}
Else
{
MaxID = 1;
}
}
Else
{
Throw new Exception ("Increment id only use for int type ");
}
}
Return maxID;
}
}

 

5: Query

 

In fact, this is very easy to do. After the json Parsing is restored to the array list, the array has a FindAll method. You can search for the tutorial and study it. For the array query, there are still many articles in the garden.

 

6: how to sort

 

This is also easy to do. After the json Parsing is restored to the array list, the array has an Sort method. You can also search for the tutorial.

 

7: how to control single-process concurrency

 

In fact, you can do it by adding the lock.

 

8: how to control multi-process concurrency

 

When multiple exe programs are recycled or enabled in the IIS application pool, multi-process operations may occur to the text database at the same time. I have been thinking about this for a long time. How can I control it?

Finally, I figured out that when the process is preparing to change the text, it reads the last modification time of the text and compares it to achieve a relative control.

 

Summary:

 

I personally think that after the above problems are solved, the basic simple text database is also formed. Of course, you can continue to pursue it.

However, text databases are complicated and unnecessary. After all, text databases are mainly simple.

If NoSql becomes popular, why not let the text database also show off in the. net field and grow into a wonderful effect in the. Net field!