SharePoint handles large folders and lists (note)

Restrictions on large list queries

 

The default query threshold for Microsoft SharePoint Foundation 2010 and Microsoft SharePoint Server 2010 applications for 5,000 projects. Any custom code that depends on the query result set that may exceed the maximum value is not executed as expected. If the list contains more than 5,000 items and these items contain fields not indexed by the query condition, queries to these lists will also fail because these queries must scan all rows in the list. You can follow the steps listed below to view and add this restriction or allow the object model to replace this restriction:

View and increase this threshold or allow the object model to override this threshold

1. Under application management on the Administration Center website, Click Manage web applications ".

2. Click general settings, and then click resource limits ".

3. View and update the threshold or allow the object model to override the limit.

 

Process folders and lists

The following suggestions are used to solve performance problems when processing large folders and lists. Based on the Steve peschka White Paper, we recommend that you process large lists in Office Sharepoint Server 2007 (this link may point to an English page) test results reported in. These recommendations also apply to Microsoft SharePoint Server 2010. For other instructions on using the spquery and portalsitemapprovider classes (which are dedicated to SharePoint Server 2010), see compile valid code in Sharepoint Server.

When processing folders and lists:

• Do not use splist. items.

  Splist. Items Selects all items in all subfolders, including all fields in the list. Use the following alternatives for each use case.

  • Add Project

    Instead of calling splist. Items. Add, use splist. additem.

  • Retrieve all items in the list

    Instead of using splist. Items, use splist. getitems (spquery query ). Apply the filter as needed and specify only the required fields to make the query more efficient. If the list contains more than 2,000 projects, the list will be paginated Based on the increments of up to 2,000 projects. The following code example shows how to paging a large list.

    Good coding practices

    Use splist. getitems to retrieve a project

     

    SPQuery query = new SPQuery();SPListItemCollection spListItems ; string lastItemIdOnPage = null; // Page position.int itemCount = 2000 while (itemCount == 2000){    // Include only the fields you will use.    query.ViewFields = "<FieldRef Name=\"ID\"/><FieldRef Name=\"ContentTypeId\"/>";       query.RowLimit = 2000; // Only select the top 2000.    // Include items in a subfolder (if necessary).    query.ViewAttributes = "Scope=\"Recursive\"";    StringBuilder sb = new StringBuilder();    // To make the query order by ID and stop scanning the table, specify the OrderBy override attribute.    sb.Append("<OrderBy Override=\"TRUE\"><FieldRef Name=\"ID\"/></OrderBy>");    //.. Append more text as necessary ..    query.Query = sb.ToString();    // Get 2,000 more items.     SPListItemCollectionPosition pos = new SPListItemCollectionPosition(lastItemIdOnPage);    query.ListItemCollectionPosition = pos; //Page info.    spListItems = spList.GetItems(query);    lastItemIdOnPage = spListItems.ListItemCollectionPosition.PagingInfo;    // Code to enumerate the spListItems.    // If itemCount <2000, finish the enumeration.    itemCount = spListItems.Count;}

    The following example shows how to enumerate a large list and pagination it.

     

    SPWeb oWebsite = SPContext.Current.Web;SPList oList = oWebsite.Lists["Announcements"];SPQuery oQuery = new SPQuery();oQuery.RowLimit = 10;int intIndex = 1;do{    Response.Write("<BR>Page: " + intIndex + "<BR>");    SPListItemCollection collListItems = oList.GetItems(oQuery);    foreach (SPListItem oListItem in collListItems)    {        Response.Write(SPEncode.HtmlEncode(oListItem["Title"].ToString()) +"<BR>");    }    oQuery.ListItemCollectionPosition = collListItems.ListItemCollectionPosition;    intIndex++;} while (oQuery.ListItemCollectionPosition != null);

  • Get project by identifier

    Instead of splist. Items. getitembyid, use splist. getitembyid (int id, string field1, Params string [] fields ). Specifies the project identifier and required fields.

• Do not enumerate the entire splist. items set or spfolder. Files set.

  Some methods and attributes are listed in the left column of Table 1. If these methods and attributes are used, the entire splist. items set will be enumerated, resulting in low performance and restrictions on large lists. Use the performance replacement items listed in the right column.

  Table 1. enumerate alternative items of splist. Items

  Methods and attributes with poor performance	Alternative with good performance
  Splist. Items. Count	Splist. itemcount
  Splist. Items. xmldataschema	Create a spquery object to retrieve only the required items.
  Splist. Items. numberoffields	Create a spquery object (specify viewfields) to retrieve only the required items.
  Splist. items [system. guid]	Splist. getitembyuniqueid (system. guid)
  Splist. items [system. int32]	Splist. getitembyid (system. int32)
  Splist. Items. getitembyid (system. int32)	Splist. getitembyid (system. int32)
  Splist. Items. reorderitems (system. boolean [], system. int32 [], system. int32)	Use spquery to perform paging queries and re-order projects on each page.
  Splist. Items. listitemcollectionposition	Contentiterator. processlistitems (splist, contentiterator. itemprocessor, contentiterator. itemprocessorerrorcallout) (Microsoft SharePoint Server 2010 only)
  Splist. Items. listitemcollectionposition	Contentiterator. processlistitems (splist, contentiterator. itemprocessor, contentiterator. itemprocessorerrorcallout) (Sharepoint Server 2010 only)

  Note
  Using the splist. itemcount attribute is a recommended method for retrieving the number of items in the list. However, optimizing this attribute for performance may cause side effects, so this attribute occasionally returns unexpected results. For example, if you need a precise number of projects, use getitems (spquery query) with poor performance, as shown in the preceding code example.

• Use the guid or URL of the list as the key to obtain references to the list.

  You can use the guid or display name of the list as the index to retrieve the splist object from the spweb. Lists attribute. Using spweb. Lists [guid] And spweb. getlist (strurl) is always better than using spweb. Lists [strdisplayname. It is best to use guid because it is unique and permanent and only needs to be searched in a single database. The name indexer retrieves the names of all lists on the website and then compares them with strings. If you have a list URL without a guid, you can use the getlist method in spweb to search for the guid of the list in the content database and then retrieve the list.

• Do not enumerate the entire spfolder. Files set.

  The column on the left of Table 2 lists the methods and attributes that expand the spfolder. Files set and cause low performance and restrictions on large lists. Use an alternative with good performance in the right column.

  Table 2. Replacement of spfolders. Files

  Methods and attributes with poor performance	Alternative with good performance
  Spfolder. Files. Count	Spfolder. itemcount
  Spfolder. Files. getenumerator ()	Contentiterator. processfilesinfolder (spfolder, system. Boolean, contentiterator. fileprocessor, contentiterator. fileprocessorerrorcallout) (Sharepoint Server 2010 only)
  Spfolder. Files [system. String]	Contentiterator. getfileinfolder (spfolder, system. String) or spfolder. parentweb. GetFile (spurlutility. combineurl (spfolder. url, system. String) (Sharepoint Server only 2010)
  Spfolder. Files [system. int32]	Do not use. Switch to contentiterator. processfilesinfolder and count the number of items during the iteration. (Sharepoint Server 2010 only)