Crawling

This term describes the way search engines discover and read pages on a website. Before a page can be indexed, it usually needs to be found and crawled. Search engine bots move through links, sitemaps and known URLs to understand what pages exist and how they connect to each other. For a business website, crawling matters because important pages should be easy to reach. If a valuable service page is hidden, disconnected or technically difficult to access, search engines may not understand its importance.

Crawling is closely linked with website structure. Clear menus, footer links, contextual links and related page sections help search engines move through the site. A sitemap can also support discovery, but internal links still matter because they show relationships between pages. For example, a page about Wix SEO services may link to related glossary entries, service pages and case studies. This helps both visitors and search engines understand the topic area. If a page is only created in the CMS but never linked from anywhere useful, it may be much weaker than a page that is part of a connected structure.

In Wix Studio projects, crawling should be considered when building CMS collections and dynamic pages. A collection may hold many items, but the website still needs a route for those pages to be found. Index pages, category pages, repeater lists, related content blocks and internal links can all help. Broken links, old URLs, missing redirects and confusing navigation can weaken crawlability. During a migration or redesign, crawl paths should be checked so that important pages remain connected after the new site goes live.

The definition is also practical for business owners because crawlability affects how well a website’s content can work for SEO. A page may be beautifully written, but if search engines cannot easily find or read it, the page may not support visibility. Common mistakes include creating orphan pages, changing URLs without redirects, using weak navigation and publishing large groups of dynamic pages with no category structure. Good crawling practice makes the website more organised. It helps search engines discover pages, understand their relationships and decide which pages are important enough to process further.