Robots.txt Setup & Fix
Robots.txt controls what crawlers can access on your site — and a single wrong line can hide your entire site from Google. We review and configure the file so it blocks the noise (admin, duplicates, filters), keeps the important pages open, and points correctly to your sitemap.
How we do it
We test every rule against real URLs, not "defaults".
-
01
File audit
We review the current robots.txt and find rules that block pages you need indexed.
-
02
Access map
We define what should be blocked (admin, duplicates) and what stays open to crawling.
-
03
Configuration
We write correct directives and add a link to your sitemap.xml.
-
04
Verification
We test the rules in Search Console to confirm nothing important is blocked.
You get a correct robots.txt: crawlers see what matters and don't waste crawl budget on junk.
Transparent pricing
A flat rate per service. Scope and turnaround depend on the size of your site.
-
Up to 20 pages landing page, brochure, small site$100
-
20–100 pages corporate site, blog, services$100
-
100+ pages catalog, e-commerce, mediafrom $100
The price is the same for the service itself — site size affects the scope and turnaround, which we agree before we start.
Order this serviceWHITE-LABEL We work white-label for agencies and freelancers — we deliver under your brand, with no Funnel Media mention in reports or files.
Questions & answers
How long does it take?
Usually 1–2 working days: the file is small, most time goes into the audit and testing the rules.
Can the site be blocked from Google by mistake?
That's exactly why we test every rule before deploying. One careless "Disallow: /" can hide the whole site.
Is robots.txt the same as noindex?
No. Robots.txt controls crawling; noindex controls indexing. You often need both — we'll advise which applies to you.
Ready to grow?
Tell us about your project — we'll prepare a free audit and a plan within 48 hours.
Book a free consultation