BigRankBot
BigRankBot is the crawler BigRank uses to audit the websites our customers add as projects. It identifies itself with its own user-agent and follows robots.txt.
Bot details
- Bot name
- BigRankBot
- User-agent
Mozilla/5.0 (compatible; BigRankBot/1.0; +https://bigrank.si/bot)- IP addresses
13.140.229.198- Operator
- Next Combinator
- Website
- https://bigrank.si
- Purpose
- Site audits of the projects our customers add, plus link status checks, competitor page checks and SERP result analysis.
- Contact
- hello@bigrank.si
How it crawls
It crawls in full only the websites that customers add as projects, starting from the homepage and the URLs in the XML sitemap.
It also makes lighter requests elsewhere: status checks on the external links those sites point to, a light crawl of the top pages of competitors a customer adds, and fetches of the search result URLs analyzed in SERP Analysis.
It follows robots.txt, limits requests per host and retries with backoff. Project crawls stay on the main host, and subdomains are only crawled when the project owner turns them on.
The only exception to robots.txt is a site's own verified owner, who can choose to override it when auditing their own site.
When a project enables JavaScript rendering, pages load in a headless browser so the audit sees the final page.
It always identifies itself with the BigRankBot user-agent, and its requests come from the fixed IP addresses listed above.
Block BigRankBot
Site owners can block BigRankBot with robots.txt.
User-agent: BigRankBot
Disallow: /To allow it again, remove the rule or change "Disallow: /" to "Allow: /", as long as no other group blocks it. Rules for BigRankBot don't affect other crawlers.
Allowlist 13.140.229.198 in your firewall or WAF if you want audits to reach the site.
Questions about crawling or a request to stop? Email hello@bigrank.si.
Last reviewed