Hiding from the robots

[Martin Pickering]

I'd like to hide an html page so its content can't be logged by passing "bots".
Can someone point me towards a solution, please?
My knowledge of PHP is terribly rudimentary but I don't expect you to write the script for me! I just need a few clues.

Am I being paranoid in not trusting robots.txt and 'NOINDEX'?

[Martin Pickering]

I found this but I welcome other suggestions:
http://www.webmasterworld.com/forum24/788.htm

Code: Select all

TOP OF page: 
<? 
// include the cloaking function 
require_once('/full/path/to/cloaking/script/IsSpider.php'); 
?>

FOR EACH LINK ON THE page(s):

<P> Visit our sponsor</P> 
<P>

<? 
if IsSpider() { 
print "Lose Weight! Grow Hair!"; 
}else{ 
print "<A href=\"http://www.fat-n-hairy.com/\">Lose Weight! Grow Hair!</a>"; 
} 
?>

</P>

etc., etc.... 

Presumably requires this:
http://web-professor.net/scripts/isbot/isbot.txt

I'm thinking "there's got to be an easier way than listing every bot!"

[gesf]

Hey Martin!
I believe robots.txt will do the work. And no... i don't think you're beeing paranoid

Code: Select all

User-agent: *
Disallow: /