Maximum File Size for crawling search services

February 16th, 2007

Goto comments leave a comment

By default, search services can crawl and filter a file with a size of up to 16 megabytes (MB ). it will always crawl the first 16 MB of a file. after this limit is reached, SharePoint Portal Server enters a warning in the Gatherer log "the file reached the maximum download limit. check that the full text of the document can be meaningfully crawler."

To increase the limit of 16 MB, you must add in the Registry new entry maxdownloadsize. To do this, follow these steps:

1. Start Registry Editor (regedit.exe ).
2. Locate the following key in the registry:
3. HKEY_LOCAL_MACHINE \ SOFTWARE \ Microsoft \ Office Server \ 12.0 \ Search \ Global \ gathering Manager
4. Open edit-New-DWORD Value. Name it maxdownloadsize.
5. Double-click, change the value to decimal, and type the maximum size (in MB) for files that the Gatherer downloads.
6. Restart the server.
7. Start Full crawl.

Note: increasing the file size may cause a timeout exception because the crawler can timeout if the file takes too long to crawl/index (because of its size ). to increase timeout value, follow these steps... For the full post, check out maximum file size for crawling

You can also review searcu usage data by enable search query logging by following these steps:

1. On the home page, onSite actionsMenu, pointSite settings, And then clickModify all site settings.
2. On the site settings page, underSite collection administration, ClickSite collection usage reports.
3. On the site collection usage summary page, on the quick launch, clickSearch queriesOrSearch results.

Additional resources:

• Crawl time factors
• Moss 2007's search feature can support up to 50 million documents in a single index
• Review search usage data