cotonti.com : [IDEA] Robots.txt Autogeneration https://www.cotonti.com Останні повідомлення в темі Cotonti en Mon, 15 Dec 2025 21:01:00 -0000 Kilandor Пн, 18 Сер 2008 20:52:25 -0000 KillerSneak
I don't think this should be added as a default as it will bloat the package ?]]>
Пн, 18 Сер 2008 18:18:01 -0000
Asmo
At this time, perhaps not fully enjoy robots.txt
Because, all the links look like ID | param | param | ...
It would be great to bring links to such mean param | param | ID (if there is a string parameters)
Then in the robots.txt can be cut duplicate pages and lists with the parameters (if required)]]>
Вт, 05 Сер 2008 17:25:45 -0000
diablo Вт, 05 Сер 2008 16:56:39 -0000 Lombi
For example you could block most of the php files which require logging in as well as all php files that could get accessed via the old URL structure which would result in duplicate content...

Thoughts?]]>
Пн, 04 Сер 2008 19:15:28 -0000