robots.txt editor downloads
WebSite Auditor Professional
4.58.6 download... your site; -Find resources restricted from indexing (in robots.txt, by Noindex meta tag or Noindex X-Robots tag); ... landing pages in one place; -An XML Sitemap editor; -A robots.txt generator; -A reporting module; -A Domain ...
RoboGen ROBOTS.TXT Editor
1.52 downloadRoboGen is a visual editor for Robot Exclusion Files. It allows the user to quickly and easily create the ROBOTS.TXT files required to instruct web search engines which ...