Processing of Lucene index files larger than 2 GB

Is there a way to limit the size of an index?

This question is sometimes brought up because of the 2 GB file size limit of some 32-bit operating systems.

This is a slightly modified answer from Doug cutting:

The easiest thing is to useIndexwriter. setmaxmergedocs ().

If, for instance, you hit the 2 GB limit at 8 m quota ents setMaxmergedocsTo 7 m. That will keep Lucene from trying to merge an index that won't fit in your filesystem. It will actually adjust tively round this down to the next lower powerIndex. mergefactor.

So with the defaultMergefactorSet to 10 andMaxmergedocsSet to 7 m Lucene will generate a series of 1 m document indexes, since merging 10 of these wowould exceed the maximum.

A slightly more complex solution:

You cocould further minimize the number of segments if, when you 've added 7 m users ENTs, optimize the index and start a new index. Then useMultisearcherTo search the indexes.

An even more complex and optimal solution:

Write a versionFsdirectoryThat, when a file exceeds 2 GB, creates a subdirectory and represents the file as a series of files.

In Linux, you can use ulimit-F to change the maximum supported file size.
Multiple indexes can be retrieved at the same time.
The problem has been solved. You can access the index file in XP. Front-end display interface needs to be improved.