Crawl depth is the number of link hops a web crawler must follow from a starting point, usually the homepage, to reach a given page. In crawling and SEO, a page linked directly from the homepage has a depth of 1, a page linked from that page has a depth of 2, and so on. The metric is also called click depth when it is measured from a user’s perspective.
Crawl depth matters because shallow pages are discovered sooner, recrawled more often and usually receive more internal link equity. Deep pages are not penalized as such, but they tend to be found later, refreshed less often and treated as less important, which can delay indexing of new or updated content.
How crawl depth is measured
Crawl depth is calculated as the shortest path through internal links from the start URL to the target URL. Site audit crawlers perform a breadth-first traversal: they fetch the homepage, record every linked URL as depth 1, then fetch those pages and assign newly found URLs depth 2. A page reachable by several routes always takes the lowest depth among them.
Only links that a crawler can follow count. Links injected after user interaction, links in blocked resources and links marked nofollow may not be followed, so the depth seen by a search engine can be greater than the depth a visitor perceives. Pages reachable only through an XML sitemap have no link depth at all and are often reported as orphan pages.
| Depth from homepage | Typical page types | Usual crawl behavior |
|---|---|---|
| 0-1 | Homepage, main categories, key services | Crawled most frequently |
| 2-3 | Subcategories, important articles, top products | Crawled regularly |
| 4-5 | Long-tail products, older posts | Crawled less often |
| 6 or more | Deep pagination, archive pages, filter results | Crawled rarely or late |
A common rule of thumb is to keep important pages within three clicks of the homepage. It is a heuristic, not a search engine rule: large catalogs may legitimately use more levels as long as priority pages stay shallow.
Why crawl depth affects discovery and ranking
Search engine crawlers schedule URLs by perceived importance and change frequency. Link structure is one of the strongest hints of importance: a page linked from the homepage or main navigation is treated as central, while a page buried behind several paginated lists looks peripheral. Depth therefore influences how quickly new content enters the index.
Depth also correlates with internal link equity. Each hop distributes authority across many outgoing links, so pages far from strong pages receive less of it. The effect is indirect: depth itself is not a ranking factor, but it reflects how much internal support a page receives.
| Concept | What it measures | Relation to crawl depth |
|---|---|---|
| Crawl depth | Link hops from the start URL | The structural distance itself |
| Crawl budget | How many URLs an engine will fetch | Deep, bloated structures waste it |
| Crawl rate | Requests per second to the server | Limits how fast depth is traversed |
| Orphan page | Page without internal links | Has no measurable depth |
| Crawler trap | Infinite URL spaces | Creates artificial, unlimited depth |
The relationship works in both directions. Search engines discover important pages faster when they are shallow, and site owners signal importance by deliberately placing pages near the top of the structure. A sudden increase in average depth after a redesign is therefore an early warning sign that priority content has lost internal support.
What causes excessive crawl depth
Excessive depth usually comes from site architecture rather than site size. A site with a million pages can keep key pages shallow, while a small blog can push its posts deep through long chronological pagination. The table lists common causes.
| Cause | Effect on depth | Typical fix |
|---|---|---|
| Long linear pagination | Each page adds one hop | Link to more pages per list, add category hubs |
| Narrow navigation | Few entry points to sections | Expand menus and hub pages |
| Missing contextual links | Content reachable only via lists | Link related articles and products |
| Faceted navigation | Endless filter combinations | Restrict crawlable filters |
| Redirect chains | Extra hops before content | Link directly to final URLs |
JavaScript can also add hidden depth. When category lists load more items only after scrolling or clicking a button, crawlers may see only the first batch of links, and everything beyond it becomes reachable solely through deeper routes. Rendering a crawlable paginated fallback with plain anchor links keeps those items within reach.
How to reduce and monitor crawl depth
The goal is a flat but meaningful architecture: important pages close to the homepage, related pages linked to each other, and no artificial layers. Practical measures include the following.
- Hub and category pages that link to the most valuable children directly.
- Contextual internal links inside articles and product descriptions.
- Related item and breadcrumb modules, which add short alternative paths.
- Pagination with jump links to distant pages instead of only next and previous.
- Regular site crawls that report depth distribution and flag pages deeper than the target.
Log file analysis completes the picture. Comparing depth with actual crawler visits shows whether deep pages are really neglected. If Googlebot or Bingbot rarely request pages beyond a certain depth, that level is the practical discovery limit for the site.
Common misconceptions
Crawl depth is often oversimplified. Several beliefs lead to poor decisions.
- URL folder depth is not crawl depth. A path like /a/b/c/d can be one click from the homepage; slashes do not count.
- Sitemaps do not replace links. They aid discovery but transmit no internal authority.
- Flatter is not always better. Linking every page from the homepage dilutes signals and destroys topical structure.
- Depth is not a penalty. Deep pages can rank well if they earn strong external links.
The simple explanation
Crawl depth is like counting how many doors you must open to reach a room in a large building. Rooms right off the main hall are visited all the time. Rooms behind five corridors are rarely visited, and new furniture there goes unnoticed for longer.
Search engine robots behave the same way. Pages that are only a few clicks from the homepage are found and checked more often.
How it is used in practice
An online bookstore with 80,000 titles notices that new releases take weeks to appear in search results. A site crawl shows that new books sit at depth 7 or more, because they are reachable only through chronological category lists of 20 books per page.
The team adds a “new releases” hub linked from the main navigation, raises listing size to 60 books per page and adds author and series links on each book page. After the change, most new titles sit at depth 2 or 3. Log files show crawler visits to new book pages within days instead of weeks, and new releases start appearing in search results far sooner.