cotonti.com : [IDEA] Robots.txt Autogeneration https://www.cotonti.com Laatste forum onderwerpen Cotonti en Tue, 12 May 2026 02:30:54 -0000 Kilandor Ma, 18 Aug 2008 20:52:25 -0000 KillerSneak
I don't think this should be added as a default as it will bloat the package ?]]>
Ma, 18 Aug 2008 18:18:01 -0000
Asmo
At this time, perhaps not fully enjoy robots.txt
Because, all the links look like ID | param | param | ...
It would be great to bring links to such mean param | param | ID (if there is a string parameters)
Then in the robots.txt can be cut duplicate pages and lists with the parameters (if required)]]>
Di, 05 Aug 2008 17:25:45 -0000
diablo Di, 05 Aug 2008 16:56:39 -0000 Lombi
For example you could block most of the php files which require logging in as well as all php files that could get accessed via the old URL structure which would result in duplicate content...

Thoughts?]]>
Ma, 04 Aug 2008 19:15:28 -0000