Further packaging of MongoDB official drivers

 

Recently, there was a demand for high real-time data requirements. I have searched for some memory databases before. I first excluded the paid products and then excluded some embedded products, two products are left behind:
1: MySQL memory engine;
2: MongoDB, a document database based on memory file ing;

For the above two products, the MySQL memory engine has the following Disadvantages:
1: when the data volume is large, the advantage of the InnoDB engine is not obvious. When the data volume is small, the performance of the B-tree index structure is acceptable.
2: The Memory engine is very difficult for persistence. If a large amount of data operations are completed in the memory engine, we need to make data persistence, whether it is the replication and subscription method orProgramMethods All have relatively large difficulties.

3: The Memory engine occupies a large amount of data space, especially the default hash structure. The same data volume is six times that of SQL Server, and the B-tree structure is not compared.

Reason for choosing MongoDB:
1: data persistence;
2: Data insertion performance is superior, especially when the non-return value mode is adopted. Theoretically, it is an asynchronous operation;
3: Because documents are stored in K-value mode, you do not need to consider the query performance of a single record;

4: For horizontal scaling, it is very easy to back up Master/Slave Data.

Problems encountered during MongoDB usage:
1: Because the stored content is completely determined by the client, some field update operations, such as set COUNT = count + 1 in SQL, have a concurrent update problem, when the two threads need to add 1 to the Count field at the same time, the Count content queried by both threads is 1. After thread a completes adding 1, the database has actually changed to 2, at this time, thread B adds 1 to the count, which is also 2. The original content is 3 and will eventually become 2. The main reason is that the field is not updated on the server, and the content of the field is completely determined by the client, that is, the update will not be performed on the server variable basis like the SQL server.

Solution: Add a custom lock ID to the record and try to add a lock to the record. If the lock is successfully applied, query the record. If the lock ID proves to be the current thread, update the record, otherwise, the lock is tried cyclically. There is a possibility of deadlock. Therefore, when locking a tag, you can add a lock time. When the lock time is exceeded, the lock is automatically unlocked to avoid deadlocks.

2: There are two main data insertion modes: one is not to return the processing result, the other is to return the processing result, if we do not care about the returned result, the maximum insert performance can be achieved. For some business scenarios that focus on processing results, the return result method is required.

3: it is best to customize the primary key without using the default _ id value. We process a student record and the student record is unique. For example, if we use the default _ id value, there will also be a concurrency problem during data insertion: two threads simultaneously process student A. At that time, when the query found that there was no student a record in the database, the data was inserted at that time, because the default _ id value is automatically generated, similar to guid or auto-incrementing ID, the common point is to generate a non-duplicate key, and the student a record is inserted twice, if we use the student ID as the primary key, we can avoid this situation. When we try to insert data for the second time, the system will throw the primary key repetition exception.

Why do I need to re-encapsulate the MongoDB driver?
In fact, both the samus driver and the official driver have their own swing functions. The samus driver encapsulates data operations in LINQ, that is, when we operate the list, we can use the same syntax as that of LINQ to reduce our learning costs. The official driver is characteristic of returning values for data processing. Our requirement is that the return value of the operation and the LINQ encapsulation are required. At that time, I foundEasymongo written by ZhaoBut it basically makes a small part of the trade-off. The trade-off is as follows:

1: removed the following logic. The original intention is to put most operations in the same connection for processing, but it does not meet our needs. We provide a one-time execution method for external users.

Insertonsubmit, updateonsubmit, deleteonsubmit
Public   Void Submitchanges ()
{
Using ( This . Database. server. requeststart ( This . Database ))
{
This . Deleteentities ();
This . Updateentites ();
This . Insertentities ();
}
}

 

2: The insert, update, and delete statements mentioned above require a return value. There are two modes for insert: one is to return a value and the other is not required. Especially for updates and deletions, it is very necessary for us to know the processing results. If we don't care about the results, I think it may be some scenarios with low data accuracy requirements.

Public Einsertstatus insertonsubmit (tentity entity)
{
If (Entity =   Null ) Throw   New Argumentnullexception ();
Return   This . Insertentities (entity, esafemode. False );
}
Public Einsertstatus insertonsubmit (tentity entity, esafemode safemode)
{
If (Entity =   Null ) Throw   New Argumentnullexception ();
Return   This . Insertentities (entity, safemode );
}
Public   Void Insertbatchonsubmit (list < Tentity > Entity)
{
If (Entity =   Null ) Throw   New Argumentnullexception ();
This . Insertbatchentities (entity );
}
Public   Bool Updateonsubmit (tentity entity)
{
If (Entity =   Null ) Throw   New Argumentnullexception ();
Return   This . Updateentites (entity );
}
Public   Bool Deleteonsubmit (tentity entity)
{
If (Entity =   Null ) Throw   New Argumentnullexception ();

Return This. Deleteentities (entity );
}

3: Modify the query. When ing an object to a mongo document, you need to provide a map for the custom object. This map is mainly used to identify which is the primary key, which fields can be recognized by LINQ? For example, to query by student ID, add this field to map. For example:

Public   Class Aggresponcompanyrecruitstudentinfomap: entitymap < Aggresponcompanyrecruitstudentinfo >
{
Public Aggresponcompanyrecruitstudentinfomap ()
{
Collection ( " Aggresponcompanyrecruitstudentinfo " );
Property (n => N. organizationid). Identity ();
Property (n => N. locktime );
Property (n => N. lockfiled );

}
}

 

However, for queries in source code, if no query expression is provided, only the fields mentioned in map are returned by default, and other object fields are not processed. Therefore, the modification is as follows: that is, the complete document is returned without a query expression.

Fieldsdocument fieldsdoc =   Null ;
If ( Null   ! = Selector)
{
Fieldsdoc = Mapper. getfields (selector );
}

 

4: remove the original complex update logic, and delete state tracking that is useless to us. What we need is simple and direct, and overly complex logic is not conducive to development.

A simplified structure chart of mymongo is shared with you:

Note: you are welcome to talk about MongoDB. If something is wrong in this article, I hope to criticize and correct it and make common progress.