Cotonti / Open Source PHP Content Management FrameworkContent Management Framework

Foren / Cotonti / Core Labs / Archive / Improve robots.txt

Providing a proper robots.txt file will help your sites to be indexed better

Trustmaster
#23029 19. Februar 2010, 01:24
Cotonti comes with robots.txt file, but it's actually a very poor one:
User-agent: *
Disallow: /datas
Disallow: /images
Disallow: /js
Disallow: /skins

I think we should provide much more advanced robots.txt file, so Cotonti will be indexed better, especially for those who don't edit this file themselves.

So, if you are familiar with robots.txt, feel free to post your snippets, comments and advice here to help us with it.

Added 52 days later:

Bump! Really nobody has dealt with robots.txt?
May the Source be with you!

Dieser Beitrag wurde von Trustmaster (am 12. April 2010, 19:02, vor 11 Jahre) bearbeitet