Forum Moderators: goodroi
The info page says it's a distributed crawler, so just like my policy for the cronic robots.txt violater Grub, I banned the user agent and the entire IP block associated with the offending IP.
Ohhhhhh, my bad, I just found robots.txt he was referring to all th etime in the beginning of the thread, reposting it here
<snip ridiculous bogus robots file>
A perfect example of the sarcasm that resulted in my rescinding the offer to help you.
A perfect example of the sarcasm
Thanks dude, you sure can make other people feel appreciated!
that resulted in my rescinding the offer to help you.
I was trying to get you to help me help you, however it is now clear that you need the kind of help that is beyond my field of expertise.
I suggest you refer to post #2 in this thread where I asked for robots.txt rather politely:
Just out of interest, can you post your robots.txt?
100 posts and counting -- and no robots.txt yet.
[edited by: Lord_Majestic at 8:25 pm (utc) on June 14, 2005]
A perfect example of the sarcasm...
It's not sarcasm to hold out your hand to a friend in need - robots.txt mistakes can cause spiders to improperly interact with your site - there are tons of good validators that can clear up these issues - check 'em out!
if this is indeed the contents of your robots.txt file, it is not valid
Either you're a complete idiot or you're still trolling. If the latter, I guess you can carry on as you see fit. If the former, that's something you'll have to work out for yourself.
Oh and in case it was the former, of course it's not the contents of my robots.txt file.
I think he may be on to something... worth looking into I would say.
See above
...of course it's not the contents of my robots.txt file
Doh! Sorry about the mixup friend - can you post the contents of your actual robots.txt file here?
Doh! Sorry about the mixup friend - can you post the contents of your actual robots.txt file here?
Troll,
Already covered multiple times in this thread.
Oh and in case it was the former, of course it's not the contents of my robots.txt file.
Hey, it was a fair guess given the circumstances! :)
At least now when you send people away to check thread for that mystical robots.txt, something remotely relevant to it can be found.
If I made a few people smile today then posting here was not a complete waste of time.
Ps. Do you really have one?
