Sphinx Full-Text Search engine Usage Guide: Indexer program configuration options

8.3.1 Mem_limit

Index procedure memory usage limit. Optional option, default 32M.

This is a mandatory memory limit that indexer does not exceed. Can be in bytes, kilobytes (with K), or megabytes (with a suffix of M). See examples. This limit is automatically increased when a value that is too small causes the I/O buffer to be below 8KB, and the minimum value depends on the size of the data to be indexed. If the buffer is below 256KB, a warning is generated.

The maximum possible limit is 2047M. Too low values affect indexing speed, but 256M to http://www.aliyun.com/zixun/aggregation/12560.html ">1024m should be sufficient for most datasets, if not all." Setting this value too high may cause the SQL Server connection to timeout. During the document collection phase, sometimes a portion of the memory buffer is sorted, and communication with the database is paused, and the database server may time out. This can be resolved by increasing the SQL Server-side timeout or reducing mem_limit.

Example：

Mem_limit = 256M

# mem_limit = 262144K # Mahouve, but in KB

# mem_limit = 268435456 # mahouve, but in bytes

8.3.2 Max_iops

The maximum number of I/O operations per second, which restricts I/O operations. Optional option, defaults to 0 (unrestricted). Options related to I/O throttling. It limits the maximum number of I/O operations (read or write) per second. A value of 0 means unrestricted.

Indexer can cause bursts of intensive disk I/O during indexing, so it is necessary to limit its disk activity (leaving some resources for other programs running on the same machine, such as SEARCHD). I/O throttling is used to achieve these functions. It works by forcing a guaranteed delay between indexer continuous disk I/O operations. Modern SATA drives can perform up to 70-100 + I/O operations per second (mainly limited by the head seek time). Limiting index I/O to one of the above values reduces the search performance caused by the index.

Example：

Max_iops = 40

8.3.3 Max_iosize

The maximum allowable I/O operation size, in bytes, for I/O throttling. Optional option, default to 0 (unrestricted).

Options related to I/O throttling. It restricts the single maximum size of indexer file I/O operations. A value of 0 represents unrestricted. Read-write operations that exceed the limit are divided into small operations and counted as multiple times by Max_iops. When writing this article, all I/O operations are limited to 256KB (the default internal buffer size), so max_iosize values greater than 256KB have no effect.

Example：

Max_iosize = 1048576