Under category e.net Series 4-Index

This article will introduce the issue of index concurrency control, so as to end the discussion on index creation in e.net.

1. allow any number of concurrent read operations, that is, any number of users can query the same index at the same time.

2. any number of read operations are allowed when the index is being modified. that is, even if the index is being optimized, adding or deleting a document also allows you to query the index. (It's so cool .)

3. Only one operation can be performed to modify the index at a time. That is, only indexwriter or indexreader can open the same index at a time. You cannot open one index at the same time.

Lucene provides several read/write operations on indexes. add documents to indexes, delete documents from indexes, optimize indexes, and merge segments. these are the methods for writing indexes. the index content is read during query.

The index concurrency issue is a very important issue, and it is easily overlooked by beginners of Lucene. When the index is damaged, orProgramWhen an exception occurs suddenly, beginners often do not know that it is caused by their own misoperations.

Let's take a look at how Lucene handles the concurrent control of index files.

First, remember the three principles:

1. allow any number of concurrent read operations, that is, any number of users can query the same index at the same time.

2. any number of read operations are allowed when the index is being modified. that is, even if the index is being optimized, adding or deleting a document also allows you to query the index. (It's so cool .)

3. Only one operation can be performed to modify the index at a time. That is, only indexwriter or indexreader can open the same index at a time. You cannot open one index at the same time.

The first criterion is easy to understand. The second criterion shows that Lucene supports concurrent operations well. the third criterion is also normal, but it should be noted that the third criterion only indicates that indexwriter and indexreader cannot coexist, and there is no objection to using the same indexwriter in multithreading to modify the index. this function is often used, so do not think it is not allowed. however, you need to control the concurrency at this time to avoid conflicts.

(Note: As mentioned in the previous series, indexreader does not read the index, but deletes the objects used by docuemnt from the index)

Let's take a look at the following table for the embodiment of these three principles when actually using the Lucene API:

The table lists the primary read/write operations on indexes. the blank space indicates the operation on the X axis and the operation on the Y axis allows concurrency.

X indicates that the operation on the X axis and the operation on the Y axis cannot be performed simultaneously.

For example, when you add a document to an index, You cannot delete the document from the index at the same time.

In fact, the above table is the embodiment of the first three principles. the add optimize merge operation is performed by indexwriter. the delete operation is completed through indexreader. therefore, the blank space in the table is the embodiment of the first and second principles, while the X (Conflict) is the concrete manifestation of the third principle.

 

To make full use of Lucene APIs without understanding concurrency control, Lucene provides a file-based lock mechanism to ensure that index files are not damaged.

When you modify the index, for example, when adding or deleting a document, the *** write. lock file, and when you read information from the segment or merge the segments, the *** commit will be generated. lock file. by default, these files are stored in the temporary folder of the system. in short, write. the lock file has a long time, that is, the lock time for modifying the index is long, and the commit. the lock time is usually very short. the details are shown in the table below.

If the index exists on the server, when many clients want to access it, they naturally want to see the lock files of other users. In this case, it is not good to put the lock files in the temporary folder of the system. in this case, you can change the location of the lock file through the configuration file.

For example, in an Asp.net application, you can use the web. config file as follows to achieve your goal.

<Configuration>
<Deleetask>
<Add key = "Lucene. net. lockdir" value = "C: yourdir"/>
</Appsettings>
</Configuration>

Not only that, in some cases, for example, when your index file is stored in a CD-ROM, there is no way to modify the index, there is no so-called concurrency conflict, in this case, you can even cancel the lock file mechanism. also through the configuration file.

<Configuration>
<Deleetask>
<Add key = "disablelucenelocks" value = "true"/>
</Appsettings>
</Configuration>

Do not use this function unless otherwise, your index files will not be protected.

The following example shows the embodiment of the lock mechanism.

Using system;
Using system. IO;
Using Lucene. net. analysis;
Using Lucene. net. index;
Using Lucene. net. store;
Using nunit. Framework;
Using directory = Lucene. net. Store. Directory;

[Testfixture]
Public class locktest
{
Private directory dir;

[Setup]
Public void Init ()
{
String indexdir = "Index ";
Dir = fsdirectory. getdirectory (indexdir, true );
}

[test]
[expectedexception (typeof (ioexception)]
Public void writelock ()
{< br> indexwriter writer1 = NULL;
indexwriter writer2 = NULL;
try
{< br> writer1 = new indexwriter (Dir, new simpleanalyzer (), true);
writer2 = new indexwriter (Dir, new simpleanalyzer (), true );

}< br> catch (ioexception e)
{< br> console. out. writeline (E. stacktrace);
}< br> finally
{< br> writer1.close ();
assert. isnull (writer2);
}< BR >}

[Test]
Public void commitlock ()
{
Indexreader reader1 = NULL;
Indexreader reader2 = NULL;
Try
{
Indexwriter writer = new indexwriter (Dir, new simpleanalyzer (),
True );
Writer. Close ();
Reader1 = indexreader. Open (DIR );
Reader2 = indexreader. Open (DIR );
}
Finally
{
Reader1.close ();
Reader2.close ();
}
}
}

However, it is disappointing that the exception that should be received in Lucene (Java) is not caught in dotlucene (1.4.3. then I asked about it on the dotlucene forum, and there are no answers yet. this is also the helplessness of open-source projects.