The Search Engine for Maximum Privacy
Femtosearch is a new search engine, currently under development, that offers much greater privacy than existing search engines.
FemtosearchBot is Femtosearch's web crawler, which discovers new and updated pages to add to Femtosearch's index.
FemtosearchBot implements a policy of politeness by rate-limiting its requests and respecting the robots exclusion protocol (i.e. robots.txt).
robots.txt file to disallow FemtosearchBot.
Note, however, that this will result in your site not appearing in Femtosearch search results.
See the Robots Exclusion Standard Wikipedia page,
www.robotstxt.org, or the instructions below for more details.
robots.txt:
User-agent: FemtosearchBot Disallow: *Alternatively, prevent FemtosearchBot from indexing specific locations only, for example
/internal/, by adding the following to your website's robots.txt:
User-agent: FemtosearchBot Disallow: /internal/