Page 1 of 1

Hiding from the robots

Posted: Mon Mar 30, 2009 11:24 am
by Martin Pickering
I'd like to hide an html page so its content can't be logged by passing "bots".
Can someone point me towards a solution, please?
My knowledge of PHP is terribly rudimentary but I don't expect you to write the script for me! I just need a few clues.

Am I being paranoid in not trusting robots.txt and 'NOINDEX'?

Re: Hiding from the robots

Posted: Mon Mar 30, 2009 11:32 am
by Martin Pickering
I found this but I welcome other suggestions:

Code: Select all

TOP OF page: 
// include the cloaking function 


<P> Visit our sponsor</P> 

if IsSpider() { 
print "Lose Weight! Grow Hair!"; 
print "<A href=\"\">Lose Weight! Grow Hair!</a>"; 


etc., etc.... 
Presumably requires this:

I'm thinking "there's got to be an easier way than listing every bot!"

Re: Hiding from the robots

Posted: Fri Apr 17, 2009 7:22 am
by gesf
Hey Martin!
I believe robots.txt will do the work. And no... i don't think you're beeing paranoid :)

Code: Select all

User-agent: *
Disallow: /