Clustered and nonclustered indexes for SQL Server

Microsoft's SQL Server provides two types of indexes: Clustered indexes (clustered index, also called clustered indexes, clustered indexes), and nonclustered indexes (nonclustered index, also called nonclustered indexes, non-clustered indexes) ...
(a) Understanding the index structure in a comprehensible sense

In fact, you can interpret an index as a special kind of directory. Microsoft's SQL Server provides two types of indexes: Clustered indexes (clustered index, also called clustered indexes, clustered indexes), and nonclustered indexes (nonclustered index, also called nonclustered indexes, non-clustered indexes). Let's take a look at the differences between clustered and nonclustered indexes, for example:

In fact, the body of our Chinese dictionary is itself a clustered index. For example, we have to check the word "Ann", it will be very natural to open the first few pages of the dictionary, because "ann" Pinyin is "an", and alphabetical order of Chinese characters in the dictionary is the English letter "a" beginning and "Z", then the word "Ann" naturally ranked in the front of the dictionary. If you have turned over all the parts that begin with "a" and still cannot find the word, then it means that you do not have the word in your dictionary, and if you look up the word "Zhang", you will also turn your dictionary into the last part, because the pinyin of "Zhang" is "Zhang". That is, the body part of the dictionary is itself a directory, and you do not need to look up other directories to find what you need to find.

We refer to this body of content itself as a directory of certain rules, called a "clustered index."

If you know a word, you can quickly find the word from the code. But you may also encounter the words you do not know, do not understand its pronunciation, at this time, you can not follow the method to find the word you want to check, and need to go to the "radicals" to find the word you are looking for, and then according to the page number after the word directly to a page to find the word you are looking for. But the sort of words you find in combination with the "radicals" and "gept" is not really the sort method of the body, for example, you check the word "Zhang", we can see in the Gept table after the Radicals "Zhang" page number is 672 pages, gept table "Zhang" above is "Chi" word, but the page number is 63 pages, "Zhang" below is "crossbow "Word, page is 390 pages. Obviously, these words are not really in the "Zhang" the word of the upper and lower side, now you see the continuous "Chi, Zhang, crossbow" three words is actually their order in the nonclustered index, is the dictionary body of words in the non-clustered index mapping. We can find the words you need in this way, but it takes two procedures to find the results in the catalog and then turn to the page numbers you need.

We put this kind of directory purely as a directory, the body is purely the sort of body is called "nonclustered index".

From the example above, we can understand what is "clustered index" and "nonclustered index".

Further, we can easily understand that each table can have only one clustered index, because the catalog can only be sorted in one way.

(ii) When to use clustered or nonclustered indexes

The following table summarizes when to use clustered or nonclustered indexes (it is important).

Action description using a clustered index with a nonclustered index
Foreign key columns should be
The primary key column should be
Columns are often sorted by grouping (order by) should be
Returning data in a range should not be
A small number of different values should not be
A large number of different values should not be
Columns that are frequently updated should not be
Frequently modifying index columns should not be
One or very few different values should not be

In fact, we can understand the above table through examples of the previous clustered index and the definition of a nonclustered index. For example, to return data in a range. For example, if you have a table with a time column and you have the aggregate index in that column, you will be very fast when you query the entire data from January 1, 2004 to October 1, 2004, because the body of your dictionary is sorted by date, A clustered index only needs to find the beginning and end data in all the data to be retrieved, rather than a nonclustered index, you must first look up the page number for each item in the table of contents, and then find the specific content based on the page number.

(c) The misunderstanding of the use of indexes with the actual practice

The purpose of the theory is to apply. Although we have just listed when clustered or nonclustered indexes should be used, in practice the above rules are easily overlooked or cannot be analyzed in the light of the actual situation. Below we will be based on the actual problems encountered in the practice of the index used in the misunderstanding, so that you can master the method of index establishment.

1, the primary key is the clustered index

The idea, I think, is an extreme mistake, a waste of a clustered index. Although SQL Server defaults to establishing a clustered index on the primary key.

In general, we will create an ID column in each table to differentiate each piece of data, and this ID column is automatically incremented, and the stride size is typically 1. This is true of the column GID in our example of office automation. At this point, if we set this column as the primary key, SQL Server will think of this Lieme as a clustered index. The benefit is that your data can be physically sorted in the database by ID, but I don't think it makes much sense.

Obviously, the advantage of a clustered index is obvious, and there can be only one rule for a clustered index in each table, which makes the clustered index more valuable.

From the definition of the clustered index we've talked about, we can see that the biggest benefit of using a clustered index is the ability to quickly narrow the query based on query requirements and avoid full table scans. In practice, because the ID number is automatically generated, we do not know the ID number of each record, so it is difficult to use the ID number to query. This makes the ID number the primary key as a clustered index a waste of resources. Second, a field that has a different ID number as a clustered index does not conform to the "Aggregate index should not be established" rule for a "large number of different values"; Of course, this situation is only for the user to modify the record content, especially when the index entry is negative, but for the query speed does not affect.

In the office automation system, whether it is the System home page display needs the user to sign the document, the meeting or the user carries on the file query and so on any circumstance to carry on the data inquiry to be inseparable from the field is "the date" and the user's own "user name".

Typically, the home page of office automation displays files or meetings that each user has not yet signed up for. Although our where statement can only limit the current user has not yet signed the case, but if your system has been established for a long time, and the amount of data is very large, then every time each user opens the first page of a full table scan, this does not make sense, the vast majority of users 1 months ago files have been browsed , it only increases the cost of the database. In fact, we can allow users to open the system first page, the database only query the user for nearly 3 months not to read the file, through the "date" this field to limit the table scan, improve query speed. If your office automation system has been established for 2 years, then your homepage display speed will theoretically be 8 times times faster than the original speed.

The word "theoretically" is mentioned here because if your clustered index is still blindly built on the primary key of the ID, your query speed is not so high, even if you set the index (non-aggregated index) on the "Date" field. Let's take a look at the speed performance of various queries in the case of 10 million data volumes (data in 3 months is 250,000):

(1) The clustered index is established only on the primary key, and the time period is not divided:

Select Gid,fariqi,neibuyonghu,title from Tgongwen

Spents: 128470 milliseconds (i.e.: 128 seconds)

(2) Set up a clustered index on the primary key and a nonclustered index on Fariq:

Select Gid,fariqi,neibuyonghu,title from Tgongwen

where Fariqi> DateAdd (Day,-90,getdate ())

Spents: 53763 milliseconds (54 seconds)

(3) Set up the aggregation index on the date column (Fariqi):

Select Gid,fariqi,neibuyonghu,title from Tgongwen

where Fariqi> DateAdd (Day,-90,getdate ())

Spents: 2423 milliseconds (2 seconds)

Although each statement extracts 250,000 data, the differences in the various cases are enormous, especially when the clustered index is set in the Date column. In fact, if your database really has 10 million capacity, the primary key is built on the ID column, as in the 1th and 2 cases above, the performance on the Web page is timed out and cannot be displayed at all. This is also one of the most important factors that I discard the ID column as a clustered index.

The method for the above speed is: Before each SELECT statement, add:

DECLARE @d datetime

Set @d=getdate ()

and add it after the SELECT statement:

Select [Statement execution takes time (milliseconds)]=datediff (Ms,@d,getdate ())

2, as long as the index can significantly improve the query speed

In fact, we can see that in the example above, the 2nd and 3 statements are identical, and the indexed fields are the same; only the non-aggregated indexes that were established on the Fariqi field, the latter set up in the aggregate index on this field, but the query speed is vastly different. Therefore, not simply indexing on any field can improve query speed.

From the statement in the table, we can see that there are 5,003 different records for the Fariqi field in the table with 10 million data. It is more appropriate to establish an aggregate index on this field. In reality, we send a few documents every day, these documents are issued in the same date, which is fully in line with the requirements of the establishment of a clustered index: "Neither the vast majority of the same, but not only a very few of the same" rule. As a result, it is important for us to build an "appropriate" aggregate index to improve query speed.

3. Add all fields that need to increase query speed to the clustered index to improve query speed

As already mentioned above: in the data query can not be separated from the field is the "date" and the user's own "user name." Since both of these fields are so important, we can merge them together to create a composite index (compound index).

Many people think that as long as you add any field to the clustered index, you can improve the query speed, and some people are puzzled: if the composite clustered index field is queried separately, then the query speed will be slow? With this problem, let's look at the following query speed (the result set is 250,000 data): ( Date column Fariqi first row in the starting column of the composite clustered index, the user name Neibuyonghu in the latter column)

(1) Select Gid,fariqi,neibuyonghu,title from Tgongwen

where fariqi> ' 2004-5-5 '

Query speed: 2513 ms

(2) Select Gid,fariqi,neibuyonghu,title from Tgongwen

where fariqi> ' 2004-5-5 ' and neibuyonghu= ' office '

Query speed: 2516 ms

(3) Select Gid,fariqi,neibuyonghu,title from Tgongwen

where neibuyonghu= ' office '

Query speed: 60280 ms

From the above experiment, we can see that if only the starting column of the clustered index is used as the query condition and the query speed of all columns used in the composite clustered index is almost the same, it is even faster than the full composite index column (in the case of the number of query result sets); This index has no effect if only the non-starting column of the composite clustered index is used as the query condition. Of course, the query speed of statements 1, 2 is the same as the number of entries queried, if all the columns of the composite index are used, and the query results are small, so that will form an "index overlay", thus the performance can be achieved optimally. Also, keep in mind that no matter if you use other columns of the aggregated index frequently, the leading columns must be the most frequently used columns.

(iv) Summary of index use experience not available on other books

1. Using aggregate index is faster than primary key with not aggregate index

Here is the instance statement: (all extracts 250,000 data)

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-9-16 '

Usage Time: 3326 ms

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen where gid<=250000

Usage Time: 4470 ms

Here, the aggregate index is nearly 1/4 faster than the primary key speed that is not an aggregated index.

2, using the aggregate index than the general primary key for the order by when the speed, especially in the case of small data volume

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen ORDER by Fariqi

Spents: 12936

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen ORDER by GID

Spents: 18843

Here, it is 3/10 faster to use the aggregate index than the general primary key for order by. In fact, if the amount of data is very small, it is much faster to use the clustered index as the rank sequence than the non-clustered index, and if the data volume is large, such as more than 100,000, the speed difference between the two is not obvious.

3. Using the time period within the aggregated index, the search time is scaled down by the percentage of the data in the data table, regardless of how many of the aggregated indexes are used

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi> ' 2004-1-1 '

Spents: 6343 milliseconds (extract 1 million)

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi> ' 2004-6-6 '

Spents: 3170 milliseconds (extract 500,000)

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-9-16 '

Time: 3326 milliseconds (identical to the result of the previous sentence.) If the number of acquisitions is the same, then the greater than and equals sign are the same)

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi> ' 2004-1-1 ' and fariqi< ' 2004-6-6 '

Spents: 3280 milliseconds

4. The date column will not slow down the query speed because there is a minute or seconds input

In the following example, there are 1 million data, 500,000 data after January 1, 2004, but only two different dates, the date is accurate to the day, before the data 500,000, there are 5,000 different dates, the date is accurate to the second.

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi> ' 2004-1-1 ' ORDER by Fariqi

Spents: 6390 milliseconds

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi< ' 2004-1-1 ' ORDER by Fariqi

Spents: 6453 milliseconds

(v) Other considerations

"The water can carry the boat, also overturn it", the index is the same. Indexes can help improve retrieval performance, but too many or improper indexes can cause system inefficiencies. Too many indexes can even cause index fragmentation.

An index is one of the most efficient ways to get data from a database. 95% of database performance problems can be solved by indexing technology.

1. Do not index small tables that are commonly used

Do not set any keys for small data tables, even if they are frequently inserted and deleted. Index maintenance on these insert and delete operations can be more time consuming than scanning the table space.

2. Do not select a social Security number (SSN) or identification number (ID) as the key

Never use SSN or ID as the key to the database. In addition to privacy reasons, SSN or ID needs to be entered manually. Never use a manually entered key as the primary key, because once you enter the error, the only thing you can do is delete the entire record and start from scratch.

3. Do not use the user's key

When determining what field to use as the table's key, be careful about the fields that the user will edit. In general, do not select User-editable fields as keys. Doing so will force you to take the following two measures:

4. Do not index memo/notes fields and do not index large text fields (many characters)

Doing so will make your index occupy a lot of database space

5. Using a system-generated primary key

If you are always designing a database with a system-generated key as the primary key, you actually control the database's index integrity. This allows the database and non-manual mechanisms to effectively control access to each row in the stored data.

There is also an advantage to using a system-generated key as a primary key: it is easy to find a logical flaw when you have a consistent key structure.

Second, improve SQL statements

Many people don't know how SQL statements are executed in SQL Server, and they worry that the SQL statements they write will be misunderstood by SQL Server. Like what:

SELECT * FROM table1 where name= ' Zhangsan ' and TID > 10000

and execution:

SELECT * FROM table1 where TID > 10000 and name= ' Zhangsan '

Some people do not know whether the execution efficiency of the above two statements is the same, because if it is simple from the statement successively, the two statements are not the same, if the TID is an aggregate index, then the next sentence only from the table of 10,000 records after the row; And the previous sentence to look at the whole table to see a few name= ' Zhangsan ', and then based on the constraints of the conditions tid>10000 to put forward the query results.

In fact, such worries are unnecessary. There is a query analysis optimizer in SQL Server that calculates the search criteria in the WHERE clause and determines which index narrows the search space for table scans, which means that it can be automatically optimized.

Although the query optimizer can automate query optimization based on the WHERE clause, it is still necessary to understand how the query optimizer works, if not, and sometimes the query optimizer does not query quickly as you intended.

During the query analysis phase, the query optimizer looks at each stage of the query and decides whether it is useful to limit the amount of data that needs to be scanned. If a stage can be used as a scanning parameter (SARG), then it is called an optimization, and the index can be used to quickly obtain the required data.

Sarg definition: Used to limit the search to an operation, because it usually refers to a specific match, a worthy range of matching or more than two conditions and connection. The form is as follows:

Column name operators < constants or Variables >

Or

< constants or variables > operator column names

Column names can appear on one side of the operator, while constants or variables appear on the other side of the operator. Such as:

Name= ' Zhang San '

Price >5000

5000< Price

Name= ' Zhang San ' and price >5000

If an expression does not meet the form of sarg, it cannot limit the scope of the search, which means that SQL Server must determine for each row whether it satisfies all the conditions in the WHERE clause. So an index is useless for an expression that does not satisfy the Sarg form.

After the introduction of Sarg, we will summarize the experience of using SARG and the conclusions of certain materials encountered in practice:

1. Whether a like statement belongs to Sarg depends on the type of wildcard you are using

such as: Name like ' Zhang% ', which belongs to Sarg

And: Name like '% Zhang ', does not belong to Sarg.

The reason is that the wildcard% is opened in the string so that the index is unusable.

2, or will cause a full table scan

such as: Name= ' Zhang San ' and price >5000 symbol SARG,

And: Name= ' Zhang San ' or price >5000 is not in line with Sarg.

Using or causes a full table scan.

3. Non-operator, function-induced statements that do not satisfy the Sarg form

The most typical case of a statement that does not satisfy the Sarg form is a statement that includes non-operators, such as not,! =, <>,!<,!>, not-EXISTS, not-in, not-like, and also functions. Here are a few examples that do not satisfy the Sarg form:

ABS (Price) <5000

Name like '% three '

Some expressions, such as:

WHERE Price *2>5000

SQL Server will also assume that Sarg,sql server will convert this type to:

WHERE Price >2500/2

However, we do not recommend this, because sometimes SQL Server does not guarantee that this conversion is completely equivalent to the original expression.

4, in the role of equivalent and OR

Statement:

Select * FROM table1 where tid in (2,3)

And

Select * FROM table1 where tid=2 or tid=3

Is the same, it will cause a full table scan, and if there is an index on the TID, its index will be invalidated.

5, try to use less

6, exists and in execution efficiency is the same

Much of the data shows that exists is more efficient than in, and should be used instead of not exists as much as possible. But in fact, I experimented with it and found that both the implementation efficiency is the same, both in front and without. Because of the subquery involved, we experimented with the pubs database that comes with SQL Server. We can open the statistics I/O State of SQL Server before running.

(1) Select Title,price from the titles where title_id in

(select title_id from sales where qty>30)

The result of this sentence is:

Table ' Sales '. Scan Count 18, logic read 56 times, physical read 0 times, pre-read 0 times.

Table ' titles '. Scan count 1, logic read 2 times, physical read 0 times, pre-read 0 times.

(2) Select Title,price from titles where exists

(SELECT * from sales where sales.title_id=titles.title_id and qty>30)

The result of the second sentence is:

Table ' Sales '. Scan Count 18, logic read 56 times, physical read 0 times, pre-read 0 times.

Table ' titles '. Scan count 1, logic read 2 times, physical read 0 times, pre-read 0 times.

From this we can see that the efficiency of execution is the same with exists and in.

7, like execution efficiency with the function charindex () and the preceding wildcard character%

Earlier, we talked about the fact that if you precede the like with a wildcard, it will cause a full table scan, so its execution is inefficient. However, some data show that the use of function charindex () instead of like speed will have a large increase, after I tried to find that this explanation is also wrong:

Select Gid,title,fariqi,reader from Tgongwen

where CHARINDEX (' Forensic detachment ', reader) >0 and fariqi> ' 2004-5-5 '

Spents: 7 seconds, plus: Scan count 4, logic read 7,155 times, physical read 0 times, pre-read 0 times.

Select Gid,title,fariqi,reader from Tgongwen

Where reader like '% ' + ' forensic detachment ' + '% ' and fariqi> ' 2004-5-5 '

Spents: 7 seconds, plus: Scan count 4, logic read 7,155 times, physical read 0 times, pre-read 0 times.

8, the Union is not absolutely more efficient than or execution

We've talked about using or in the WHERE clause to cause a full table scan, generally, the data I've seen is recommended to use Union instead of or. It turns out that this argument is applicable to most of them.

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-9-16 ' or gid>9990000

Spents: 68 seconds. Scan count 1, logic read 404,008 times, physical read 283 times, pre-read 392,163 times.

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-9-16 '

Union

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen where gid>9990000

Spents: 9 seconds. Scan Count 8, logic read 67,489 times, physical read 216 times, pre-read 7,499 times.

It seems that the Union in general is more efficient than using or.

But after the experiment, I found that if the query column on or both sides is the same, then the Union and with or the execution speed is much worse, although here the Union scan is the index, and or scan the full table.

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-9-16 ' or fariqi= ' 2004-2-5 '

Spents: 6423 milliseconds. Scan count 2, logic read 14,726 times, physical read 1 times, pre-read 7,176 times.

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-9-16 '

Union

Select Gid,fariqi,neibuyonghu,reader,title from Tgongwen

where fariqi= ' 2004-2-5 '

Spents: 11640 milliseconds. Scan Count 8, logic read 14,806 times, physical read 108 times, pre-read 1144 times.

9, the field extraction to follow the "how much, how much" principle, avoid "select *"

Let's do an experiment:

Select top 10000 gid,fariqi,reader,title from Tgongwen ORDER by gid Desc

Spents: 4673 milliseconds

Select top 10000 gid,fariqi,title from Tgongwen ORDER by gid Desc

Spents: 1376 milliseconds

Select top 10000 Gid,fariqi from Tgongwen ORDER by gid Desc

Spents: 80 milliseconds

As a result, each time we extract a single field, the data extraction speed will be correspondingly improved. The speed of ascension depends on the size of the field you discard.

10, COUNT (*) is not slower than count (field)

Some of the information says that using * will count all columns, which is obviously less efficient than a world listing. This argument is in fact unfounded. Let's see:

Select COUNT (*) from Tgongwen

Spents: 1500 milliseconds

Select COUNT (GID) from Tgongwen

Spents: 1483 milliseconds

Select COUNT (Fariqi) from Tgongwen

Spents: 3140 milliseconds

Select COUNT (title) from Tgongwen

Spents: 52050 milliseconds

As can be seen from the above, if the speed of count (*) and COUNT (primary key) is equivalent, and count (*) is faster than any other field except the primary key, and the longer the field, the faster the rollup. I think, if you use COUNT (*), SQL Server may automatically find the smallest field to summarize. Of course, if you write the count (primary key) directly, it will come more directly.

11, order by clustered index column to sort the most efficient

Let's see: (GID is the primary key, Fariqi is the aggregate index column)

Select top 10000 gid,fariqi,reader,title from Tgongwen

Spents: 196 milliseconds. Scan count 1, logic read 289 times, physical read 1 times, pre-read 1527 times.

Select top 10000 gid,fariqi,reader,title from Tgongwen ORDER by GID ASC

Spents: 4720 milliseconds. Scan count 1, logic read 41,956 times, physical read 0 times, pre-read 1287 times.

Select top 10000 gid,fariqi,reader,title from Tgongwen ORDER by gid Desc

Spents: 4736 milliseconds. Scan count 1, logic read 55,350 times, physical read 10 times, pre-read 775 times.

Select top 10000 gid,fariqi,reader,title from Tgongwen ORDER by Fariqi ASC

Spents: 173 milliseconds. Scan count 1, logic read 290 times, physical read 0 times, pre-read 0 times.

Select top 10000 gid,fariqi,reader,title from Tgongwen ORDER BY Fariqi Desc

Spents: 156 milliseconds. Scan count 1, logic read 289 times, physical read 0 times, pre-read 0 times.

As we can see from the above, the speed of unordered and the number of logical reads are equivalent to the "ORDER by clustered index column", but these are much faster than the "ORDER by nonclustered index column" query speed.

At the same time, in order to sort by a field, whether it is a positive or reverse order, the speed is basically equivalent.

12. Efficient Top

In fact, when querying and extracting very large data sets, the biggest factor that affects database response time is not the data lookup, but the physical i/0 operation. Such as:

Select Top Ten * FROM (

Select top 10000 gid,fariqi,title from Tgongwen

where neibuyonghu= ' office ' ORDER by gid Desc) as a

ORDER BY GID ASC

This statement, in theory, the execution time of the whole statement should be longer than the execution time of the clause, but the opposite is true. Because the clause executes after 10,000 records are returned, and the entire statement returns only 10 statements, the most important factor that affects the database response time is physical I/O operations. One of the most effective ways to limit physical I/O operations here is to use the top keyword. The top keyword is a system-optimized word in SQL Server that extracts previous or previous percentage data. Through the application of the author in practice, it is found that top is very useful, and the efficiency is very high. But this word is not in another large database Oracle, which is not a pity, although it can be solved in Oracle with other methods (such as: RowNumber). In a later discussion about the "paging display stored procedure for TENS data", we will use the keyword top.

So far, we've discussed how to quickly query the data you need from a large-capacity database. Of course, we introduce these methods are "soft" method, in practice, we also have to consider a variety of "hard" factors, such as: Network performance, server performance, operating system performance, and even network cards, switches and so on.

Clustered and nonclustered indexes for SQL Server