Site Crawler (BETA)

Submit a domain to crawl, and receive an extensive overview of potential errors and issues. We respect meta nofollow, rel nofollow and robots.txt. Paid accounts have no restrictions on the amount of URL's to be crawled, however, free users need to claim ownership over the desired domain to avoid restrictions.

Credits Cost
Also check if all external URLs are working


Site Crawler tool preview
  • We crawl the domain, and give you an extensive overview of potential errors and issues. Everything from HTTP status codes, load speeds and disadvantageous meta-information. An effective tool to optimize your domain's SEO potential.
  • Technical and or markup-errors, marked in red, are in need of immediate adjustments.
  • Technical and or markup-issues, marked in yellow, are issues that we recommend to be adjusted.
  • Areas marked with green text are satisfactory by our standards.
  • Files and external URL's are only shown if an error or issue was found.
  • Restrictions:
    • Users not logged in, are restricted to the option of 50 URL's for 5 credits and no additional data from third-parties.
    • Free users are also restricted unless ownership of the desired domain has been claimed. Click here to claim ownership of your domains.
    • Accounts with an active paid subscription has no restrictions.
  • Credit cost:
    • A maximum of 50 URL's for 5 credits.
    • A maximum of 300 URL's for 10 credits (With Ahrefs URL Rating).
    • A maximum of 1000 URL's for 20 credits (With Ahrefs URL Rating).
    • Credits cost will be adjusted to the amount of URL's found (i.e. if only 250 URL's are found when submitting for 1000, the cost will be adjusted to 10 credits.)