Sensitive Files
robots.txt File Check
Understand the importance of the robots.txt file, how it influences your SEO, and which directives to add to protect sensitive data from search engines.
What this test checks
This test checks for the presence and accessibility of the robots.txt file at your site root. This file tells search engines (Google, Bing, etc.) which pages they can crawl or ignore. Its absence may lead to indexing of sensitive pages (admin, backups, etc.) and harm your SEO.
Our recommendation
Create a robots.txt file at your site root with clear rules. Block access to sensitive folders like /admin/, /backup/, /wp-content/uploads/. Use an online generator if unfamiliar with the syntax. Don't forget to include your sitemap URL.
Security tests