Robots.txt Tester
Robots.txt Tester
Paste or load a robots.txt, enter a URL and a crawler, and see whether it is allowed or blocked — and exactly which rule decided it.
robots.txt content
Test a URL
Deciding rule
Matched group (user-agent)