Rankzon

About the Rankzon bot

RankzonBot is the polite web crawler behind rankzon.com, an SEO analytics platform focused on the Swedish market. This page explains what the bot does, how it behaves, and how to identify or block it.

User-Agent

RankzonBot/0.1.0 (+https://rankzon.com/bot)

What it does

RankzonBot crawls websites to extract titles, meta descriptions, canonical URLs, headings, and link graphs. The data is used to power SEO dashboards for our customers — keyword rankings, backlink discovery, internal-link auditing, and content analysis. We focus on Swedish websites, but the bot may visit any site that is linked from a tracked domain.

The bot runs in two modes. Standard crawls are lean — they collect page metadata and outgoing links, and run automatically every 24 hours for domains monitored on Rankzon. Extended crawls additionally extract headings and a Markdown-rendered copy of the main content, and are only run on explicit request from a customer for their own domain.

We do not crawl behind authentication, submit forms, or execute JavaScript that triggers side effects. Pages are fetched with standard HTTP GET requests.

Politeness contract

These limits are hardcoded in the crawler and cannot be raised by configuration. They apply per origin (hostname).

  • One request at a time per domain. Concurrency is fixed at 1 — we never open parallel connections to the same site.
  • At most 60 requests per minute per domain — one request roughly every two seconds.
  • robots.txt Disallow rules are honored. We fetch and parse /robots.txt before crawling and skip every path it forbids for the RankzonBot user-agent (or wildcard).
  • Crawl-Delay is not honored. It is not part of RFC 9309 and Google does not honor it either. Our universal 60 rpm floor is stricter than most published Crawl-Delay values anyway. If you need us slower, contact us at the email below.
  • We identify ourselves transparently with the RankzonBot token in the User-Agent string. We do not spoof browsers or rotate identifiers.

How to block RankzonBot

Add this to your robots.txt to block the bot entirely:

User-agent: RankzonBot
Disallow: /

You can also scope rules to specific paths — the same syntax as for any other bot. RankzonBot reads robots.txt on every crawl, so changes take effect immediately.

If you need a hard block at the network edge, match on the RankzonBot token in the User-Agent header from your WAF, CDN, or web server config and return a 403.

Identifying RankzonBot in your logs

Every request from our crawler carries a User-Agent that starts with the literal token RankzonBot, followed by a version and the URL to this page:

RankzonBot/0.1.0 (+https://rankzon.com/bot)

The version segment changes between releases — match on the RankzonBot prefix rather than a full string. We do not publish a static IP range: the crawler runs in our Kubernetes cluster and pod IPs rotate with deploys.

Built on

RankzonBot is a small Node.js program built on well-known open-source tooling: Crawlee for the crawl scheduling, Cheerio for HTML parsing, Mozilla Readability for main-content extraction, and robots-parser for strict robots.txt compliance.

Opt out or report abuse

If you would like RankzonBot to stop crawling your site, or if you believe the crawler is misbehaving, email us. We respond to opt-out requests within one business day.

[email protected]