On rownum and limiting results

By Tom Kyte

Http://www.oracle.com/technetwork/issue-archive/2006/06-sep/o56asktom-086197.html

Our monitoring ist explains how rownum works and how to make it work for you.

This issue's ask Tom column is a little different from the typical column. I receive into questions about how to perform top-N and pagination queries in Oracle Database, so I decided to provide an excerpt from the book published tive oracle by design (Oracle Press, 2003) in hopes of answering a lot of these questions with this one column. note that the content here has been modified from the original to fit the space and format.

Limiting result sets

Rownum is a magic column in Oracle Database that gets contains people into trouble. when you learn what it is and how it works, however, it can be very useful. I use it for two main things:

• To perform top-N processing. This is similar to using the limit clause, available in some other databases.

• To paginate through a query, typically in a stateless environment such as the web. I use this technique on the asktom.oracle.com web site.

I'll take a look at each of these uses after I review how rownum works.

How rownum works

Rownum is a pseudo column (not a real column) that is available in a query. rownum will be assigned the numbers 1, 2, 3, 4 ,... n, where N is the number of rows in the Set rownum is used. A rownum value is not assigned permanently to a row (this is a common misconception ). A row in a table does not have a number; you cannot ask for Row 5 from a table-there is no such thing.

Also confusing to assign people is when a rownum value is actually assigned. A rownum value is assigned to a row after it passes the predicate phase of the query but before the query does any sorting or aggregation. also, a rownum value is incremented only after it is assigned, which is why the following query will never return a row:

 
Select * from t where rownum> 1;
 

Because rownum> 1 is not true for the first row, rownum does not advance to 2. Hence, no rownum value ever gets to be greater than 1. Consider a query with this structure:

 
Select..., rownum from t where <WHERE clause> group by <columns> having  
 
Think of it as being processed in this order:
 

 
1. The from/WHERE clause goes first.
2. rownum is assigned and incremented to each output row from the from/WHERE clause.
 

3. Select is applied.
 

 
4. Group by is applied.
 

 
5. Having is applied.
 

 
6. Order by is applied.
 

 
That is why a query in the following form is almost certainly an error:
 

 
Select * from EMP where rownum <= 5 order by Sal DESC;
 
 
The intention was most likely to get the five highest-paid people-a top-N query. what the query will return is five random records (the first five the query happens to hit), sorted by salary. the procedural pseudo docode for this query is as follows:
 

 
Rownum = 1for X in (select * from EMP) loop exit when not (rownum <= 5) Output record to temp rownum = rownum + 1end loopsort temp
 
It gets the first five records and then sorts them. A query with where rownum = 5 or where rownum> 5 doesn't make sense. this is because a rownum value is assigned to a row during the predicate evaluation and gets incremented only after a row passes the WHERE clause.
 

 
Here is the correct version of this query:
 

 
Select * from (select * from EMP order by Sal DESC) Where rownum <= 5;
 
 
This version will sort EMP by salary descending and then return the first five records it encounters (the top-five records ). as you'll see in the top-N discussion coming up shortly, oracle Database doesn't really sort the entire result set-it is smarter than that-but conceptually that is what takes place.
 
Top-
 N Query Processing with rownum
In a top-N query, you are generally interested in taking some complex query, sorting it, And then retrieving just the first n rows (the top N rows ). rownum has a top-N optimization that facilitates this type of query. you can use rownum to avoid a massive sort of large sets. i'll discuss how it does this conceptually and then look at an example.
 

 
Suppose you have a query in this form:
 

 
Select... from... where... order by columns;
 
 
Assume that this query returns a lot of data: thousands, hundreds of thousands, or more rows. however, you are interested only in the Top N-say the top 10 or top 100. there are two ways to approach this:
 


 
 
 
  
 
   
Have the client application run that query and fetch just the first n rows.
 
   
Use that query as an inline view, and use rownum to limit the results, as in select * from (your_query_here) Where rownum <= n.
 
  
 

 
The second approach is by far superior to the first, for two reasons. the lesser of the two reasons is that it requires less work by the client, because the database takes care of limiting the result set. the more important reason is the special processing the database can do to give you just the top N rows. using the top-N query means that you have given the database extra information. you have told it, "I'm interested only in getting n rows; I'll never consider the rest. "Now, that doesn' t sound too earth-shattering until you think about sorting-How sorts work and what the server wocould need to do. let's walk through the two approaches with a sample query:
 

 
Select * from t order by unindexed_column;
 
Now, assume that T is a big table, with more than one million records, and each record is "fat"-say, 100 or more bytes. also assume that unindexed_column is, as its name implies, a column that is not indexed. and assume that you are interested in getting just the first 10 rows. oracle Database wocould do the following:
 

 
1. Run a full-Table scan on t.
2. Sort t by unindexed_column. This is a full sort.
 

 
3. Presumably run out of sort Area Memory and need to swap temporary extents to disk.
 

 
4. Merge the temporary extents back to get the first 10 records when they are requested.
 

 
5. Clean up (release) the temporary extents as you are finished with them.
 

 
Now, that is a lot of I/O. Oracle database has most likely copied the entire table into temp and written it out, just to get the first 10 rows.
 

Next, let's look at what oracle database can do conceptually with a top-N query:
 

 
Select * from (select * from t order by unindexed_column) Where rownum <: N;
 
 
In this case, Oracle database will take these steps:
 

 
1. Run a full-Table scan on T, as before (you cannot avoid this step ).
2. In an array of: n elements (presumably in memory this time), sort only: N rows.
 

 
The first n rows will populate this array of rows in sorted order. when the n + 1 row is fetched, it will be compared to the last row in the array. if it wocould go into slot n + 1 in the array, it gets thrown out. otherwise, it is added to this array and sorted and one of the existing rows is discarded. your sort area holds n rows maximum, so instead of sorting one million rows, you sort n rows.
 

This seemingly small detail of using an array concept and sorting just n rows can lead to huge gains in performance and resource usage. it takes a lot less RAM to sort 10 rows than it does to sort one million rows (not to mention temp space usage ).
 

 
Using the following table t, you can see that although both approaches get the same results, they use radically different amounts of resources:
 

 
Create Table tasselect dbms_random.value (100000) ID, rpad ('*', 40, '*') data from dualconnect by level <=; begindbms_stats.gather_table_stats (user, 'T '); end;/now enable tracing, viaexec dbms_monitor.session_trace_enable (waits => true );
 
 
And then run your top-N query with rownum:
 

 
Select * from (select * from t order by ID) Where rownum <= 10;
 
 
And finally run a "do-it-yourself" query that fetches just the first 10 records:
 

Declarecursor C isselect * from t order by ID; l_rec C % rowtype; begin open C; for I in 1 .. 10 loop fetch C into l_rec; exit when C % notfound; end loop; close C; end ;/
 
 
After executing this query, you can use tkprof to format the resulting trace file and review what happened. first examine the top-N query, as shown in Listing 1.
 

 
Code listing 1:Top-NQuery using rownum
 

Select * from (select * from t order by ID) where rownum <= 10 call Count CPU elapsed disk query current rows -------- ------- -------- parse 1 0.00 0.00 0 0 0 0 execute 1 0.00 0.00 0 0 0 0 fetch 2 0.04 0.04 0 949 0 10 -------- ------- -------- ------ total 4 0.04 0.04 0 949 0 10 rows row Source Operation ----------------- limit 10 count stopkey (Cr = 949 Pr = 0 PW = 0 time = 46997 us) 10 view (Cr = 949 Pr = 0 PW = 0 time = 46979 US) 10 sort order by stopkey (Cr = 949 Pr = 0 PW = 0 time = 46961 us) 100000 table access full T (Cr = 949 Pr = 0 PW = 0 time = 400066 us)
The query read the entire table (because it had to), but by using the sort order by stopkey step, it was able to limit its use of temporary space to just 10 rows. note the final row source operation line-it shows that the query did 949 logical I/OS in total (Cr = 949 ), saved med no physical reads or writes (Pr = 0 and PW = 0), and took 400066 millionths of a second (0.04 seconds ). compare that with the do-it-yourself approach shown in Listing 2.
 

 
Code listing 2:Do-it-yourself query without rownum
 

Select * from t order by idcall count CPU elapsed disk query current rows -------- ------- -------- parse 1 0.00 0.00 0 0 0 0 execute 2 0.00 0.00 0 0 0 0 fetch 10 0.35 0.40 155 949 6 10 -------- ------- -------- ------ total 13 0.36 0.40 155 949 6 10 rows row Source Operation ----------------- limit 10 sort order by (Cr = 949 Pr = 155 pw = 891 time = 401610 us) 100000 table access full T (Cr = 949 Pr = 0 PW = 0 time = 400060 US) elapsed times include waiting for the following events: event waited on times ------------------------------ ------------ direct path write temp 33 direct path read temp 5
As you can see, this result is very different. notably, the elapsed/CPU times are significantly higher, and the final row source operation lines provide insight into why this is. you had to perform a sort to disk, which you can see with the PW = 891 (physical writes ). your query completed MED some direct path reads and writes-the sort of 100,000 records (instead of just the 10 we are ultimately interested in) took place on disk-adding considerably to the runtime/resource usage of your query.
 
Pagination with rownum
 
My all-time-favorite use of rownum is pagination. In this case, I use rownum to get rows nthrough m of a result set. The general form is as follows:
 

 
Select * from (select/* + first_rows (n) */. *, rownum rnum from (your_query_goes_here, with order by) A where rownum <=: max_row_to_fetch) Where rnum >=: min_row_to_fetch; where
 

 

 
  
 
   
 
 first_rows (n) tells the optimizer, "Hey, I'm interested in getting the first rows, and I'll get n of them as fast as possible. "
 
 
   
 
: max_row_to_fetch is set to the last row of the result set to fetch-if you wanted rows 50 to 60 of the result set, you wowould set this to 60. 
 
 
   
 
: min_row_to_fetch is set to the first row of the result set to fetch, so to get rows 50 to 60, you wocould set this to 50. 
 
 
  

 
 the concept behind this scenario is that an end user with a Web browser has done a search and is waiting for the results. it is imperative to return the first result page (and second page, and so on) as fast as possible. if you look at that query closely, you'll notice that it inates a top-N query (get the first: max_row_to_fetch rows from your query) and hence benefits from the top-N query optimization I just described. further, it returns over the network to the client only the specific rows of interest-It removes any leading rows from the result set that are not of interest. 

One important thing about using this pagination query is that the order by statement shocould order by something unique. if what you are ordering by is not unique, you should add something to the end of the order by to make it so. if you sort 100 records by salary, for example, and they all have the same salary value, then specifying rows 20 to 25 does not really have any meaning. in order to see this, use a small table with lots of duplicated ID values:
 

 
SQL> Create Table T 2 as 3 select Mod (level, 5) ID, trunc (dbms_random.value (1,100) data 4 from dual 5 connect by level <= 10000; Table created.
 
 
And then query rows 148 to 150 and 151 after sorting by the ID column:
 

 SQL> select * 2 from 3 (select. *, rownum rnum 4 from 5 (select ID, data 6 from T 7 order by ID) A 8 where rownum <= 150 9) 10 where rnum> = 148; ID data rnum ------- ---------- ----------- 0 38 1480 64 1490 53 150sql> SQL> select * 2 from 3 (select. *, rownum rnum 4 from 5 (select ID, data 6 from T 7 order by ID) A 8 where rownum <= 151 9) 10 where rnum> = 148; ID data rnum ------- ---------- ----------- 0 59 1480 38 1490 64 1500 53 151 
 note in this case that one time for row 148, The result returned data = 38, and that the next time, the result returned data = 59. both queries are returning exactly the right answer, given what you 've requested: sort the data by ID, throw out the first 147 rows, and return the next 3 or 4 rows. both of them do that, But because Id has so far duplicate values, the query cannot do it deterministically-the same sort order is not assured from run to run of the query. in order to correct this, you need to add something unique to the order. in this case, just use rowid: 

 SQL> select * 2 from 3 (select. *, rownum rnum 4 from 5 (select ID, data 6 from T 7 order by ID, rowid) A 8 where rownum <= 150 9) 10 where rnum> = 148; ID data rnum ------- ---------- ----------- 0 45 1480 99 1490 41 150sql> SQL> select * 2 from 3 (select. *, rownum rnum 4 from 5 (select ID, data 6 from T 7 order by ID, rowid) A 8 where rownum <= 151 9) 10 where rnum> = 148; ID data rnum ------- ---------- ----------- 0 45 1480 99 1490 41 1500 45 151 
Now the query is very deterministic. rowid is unique within a table, so if you use order by ID and then within ID you use order by rowid, the rows will have a definite, deterministic order and the pagination query will deterministically return the rows as expected.
 

 
Next steps
 

 
Ask Tom 
Oracle Vice President Tom Kyte answers your most difficult technology questions. Highlights from that forum appear in this column.
 

 
Asktom.oracle.com
 

 
Read more Tom
 
Expert Oracle database architecture: 9i and 10g programming techniques and Solutions
 
Proactive oracle by design
 
Rownum Wrap-Up
 
I'll hazard a guess that you and your other readers now have a newfound respect for rownum and understand these aspects:
 


 

 
  
 
   
How rownum is assigned, so you can write bug-free queries that use it
 
   
How it affects the processing of your query, so you can use it to paginate a query on the Web
 
   
How it can reduce the work completed MED by your query, so that top-N queries that once consumed a lot of temp space now use none and return results much faster
 
  
 

 
 
Tom KyteHas worked for Oracle since 1993. The he is a vice president in the Oracle public sector group and the authorExpert Oracle database architecture: 9i and 10g programming techniques and Solutions(Apress 2005) andProactive oracle by design(Oracle Press, 2003), among other books.