Dex-MongoDB index optimization tool

Dex is an open-source MongoDB optimization tool. It analyzes query logs and current database indexes and proposes efficient index optimization policies to administrators.

Quick Start Installation

pip install dex

Start monitoring

dex -f mongodb.log mongodb://localhost

In the monitoring process, dex outputs recommendation results through stderr.

{"index": "{'simpleIndexedField': 1, 'simpleUnindexedFieldThree': 1}","namespace": "dex_test.test_collection""shellCommand": "db.test_collection.ensureIndex(  {'simpleIndexedField': 1, 'simpleUnindexedFieldThree': 1}, {'background': true})"}

Some statistics will be output.

Total lines read: 7Understood query lines: 7Unique recommendations: 5Lines impacted by recommendations: 5

We can see that there is a shellCommand field in the output result, which is the index addition statement. If you think dex is recommended, you can directly copy this script inMongoDBThe index is added. It is quite convenient.

In addition to the information name output to stderr during running, the recommendation information is packaged into a large JSON object and output once in stdout.

Working Principle

DexThe following three steps are performed during running.

1. Parse query
2. Use an existing index to determine the current query
3. If you find that the index is incorrect, we recommend the appropriate index.

Step 1: parse query

Dex parses the query into the following categories:

• EQUIV-common query by value, for example: {a: 1}
• SORT-sort operation, such as. sort ({a: 1 })
• RANGE-RANGE query, for example: Specifically: '$ ne', '$ gt', '$ lt', '$ gte', '$ lte', '$ in ', '$ nin',' $ all', '$ not'
• UNSUPPORTED
  • Combined Query, such as: $ and, $ or, $ nor
  • Nested queries except RANGE

Step 2: determine the current index

There are two criteria to find the index required for the query.

• Coverage (none, partial, full)-Coverage indicates the index, which contains three values in brackets. None indicates that no index is overwritten. Full indicates that all fields in the query can be indexed. Partial indicates the situation between none and full.
• Order (ideal or not)-Order is used to determine whether the Order of the index is ideal. The ideal index sequence is:
  Equivalence ○ Sort ○ Range
  It is worth noting that only geographic location indexes are analyzed, but no suggestions for improvement are provided.

Step 3: recommend appropriate Indexes

Through the above two steps, we can have an understanding of the possible use of indexes in a query. Dex generates an Optimal Index for this query. If this index does not exist and the query condition does not include the UNSUPPORTED mentioned above, Dex will make corresponding index optimization suggestions.

Dex will introduce many new features in the future, such as using system. profile logs for recommendation and supporting geographic indexes. For more information, see the project homepage.

Dex details: click here
Dex: click here