A sitemap index is an XML file that lists the locations of multiple individual sitemaps, so that a search engine can discover all of them from a single address. In crawling, it is the standard way for large or complex websites to stay within the size limits of the sitemap protocol while still giving crawlers a complete inventory of their URLs.
A sitemap index contains no page URLs itself. It points to child sitemaps, and each child sitemap lists the actual pages, images or videos. Search engines such as Google and Bing read the index, fetch the child files it references, and use the listed URLs as discovery hints alongside links found during normal crawling.
How a sitemap index is structured
A sitemap index follows the same Sitemaps protocol as regular XML sitemaps but uses a different root element. The file opens with a sitemapindex root that declares the protocol namespace, and inside it each child file is described by a sitemap entry containing a required loc value and an optional lastmod date.
The loc value must be the full, absolute URL of the child sitemap, including the protocol. The lastmod value tells crawlers when the child file last changed, in W3C date format, and helps them decide which child sitemaps to fetch again first.
| Element | Required | Purpose |
|---|---|---|
| sitemapindex | Yes | Root element that wraps all entries |
| sitemap | Yes, one per child file | Container for one child sitemap |
| loc | Yes | Absolute URL of the child sitemap |
| lastmod | No | Date the child sitemap last changed |
Child sitemaps may be compressed with gzip, and the index can reference compressed and uncompressed files alike. The index itself may also be gzipped, as long as it is served with a correct content type and the size limits apply to the uncompressed data.
Limits that make a sitemap index necessary
The protocol caps each sitemap at 50,000 URLs and 50 MB of uncompressed data. A site with more URLs than that, or with very long URLs and many image entries, must split its inventory across several sitemaps and tie them together with an index.
| File type | Maximum entries | Maximum uncompressed size |
|---|---|---|
| Individual sitemap | 50,000 URLs | 50 MB |
| Sitemap index | 50,000 child sitemaps | 50 MB |
| Nested index | Not supported | Not applicable |
A sitemap index cannot reference another sitemap index. The hierarchy has exactly two levels, which in theory still allows billions of URLs per index. Very large sites that need more simply submit several index files, which Google Search Console accepts up to a per-site limit of several hundred.
Sitemap index compared with related crawling tools
A sitemap index is one of several tools that influence how crawlers discover a site. It works together with an XML sitemap, the robots.txt file and internal linking, but each has a distinct role, summarized in the table below.
| Concept | What it does | Relation to a sitemap index |
|---|---|---|
| XML sitemap | Lists page, image or video URLs | Child file referenced by the index |
| Robots.txt | Controls crawler access and can name sitemaps | Can point crawlers to the index |
| Internal linking | Lets crawlers discover pages through links | Primary discovery path the index supplements |
| Crawl budget | Limits how much a crawler fetches | Index helps prioritize but does not raise it |
A sitemap index does not force indexing and does not increase crawl budget. It only makes discovery more reliable and helps crawlers spot changed sections faster, which matters most for large catalogs, news archives and sites with deep pagination where some pages have few internal links.
Organizing child sitemaps in practice
A well-designed sitemap index groups URLs into meaningful child sitemaps, not into arbitrary chunks. Splitting by content type or site section turns the index into a diagnostic tool, because Search Console reports submitted and indexed counts per child file.
- By content type: separate sitemaps for products, categories, articles, images and videos.
- By site section or language: one child file per country site or language version.
- By date: monthly or yearly sitemaps for news and blog archives, so older files rarely change.
- By update frequency: frequently changing URLs in small files with accurate lastmod values.
The index is usually submitted in Google Search Console and Bing Webmaster Tools, and announced with a Sitemap: line in robots.txt so that other crawlers can find it. Child sitemaps should sit on the same host as the index, or the owner must prove control of both hosts, for example through cross-site submission declared in robots.txt.
Common sitemap index errors
Most sitemap index problems are simple technical errors that stop crawlers from reading child files. Others are quality problems, where the files are valid but list URLs that should not be crawled at all.
| Error | Likely cause | Fix |
|---|---|---|
| Child sitemap unreachable | Wrong URL, redirect or 404 response | Use final absolute URLs that return 200 |
| Nested index rejected | Index lists another index | Reference only regular sitemaps |
| Lastmod ignored | Dates change on every generation | Update lastmod only on real changes |
| Low indexed share | Child files list noindexed or redirected URLs | List only canonical, indexable URLs |
| File too large | Limits exceeded after growth | Split child files automatically |
Search engines learn to distrust inaccurate lastmod values. If every child sitemap claims a fresh date on every request, the signal loses its value, so generators should derive dates from actual content changes.
Understanding it simply
A sitemap is a list of pages that a website wants search engines to know about. When a site has too many pages for one list, it makes several lists. A sitemap index is the list of those lists.
It works like the table of contents at the front of a multi-volume encyclopedia. The table of contents does not contain the articles, but it tells you which volume to open. A search engine reads the index first, then opens each smaller sitemap it points to.
Applied in practice
An online hardware store has about 180,000 product pages, 4,000 category pages and a blog with 2,500 articles. A single sitemap cannot hold that many URLs, and the team also wants to know which part of the site has indexing problems.
The developers generate four product sitemaps of up to 50,000 URLs each, one category sitemap and one blog sitemap, all listed in a single sitemap index submitted in Search Console and referenced in robots.txt. Within a few weeks, the per-file report shows that one product sitemap has a much lower indexed share. Investigation reveals that it contains discontinued items redirecting to categories, and removing them improves the overall indexing ratio.