Cotonti comes with robots.txt
file, but it's actually a very poor one:
I think we should provide much more advanced robots.txt
file, so Cotonti will be indexed better, especially for those who don't edit this file themselves.
So, if you are familiar with robots.txt
, feel free to post your snippets, comments and advice here to help us with it.Added 52 days later:
Bump! Really nobody has dealt with robots.txt