podarum Posted October 5, 2009 Share Posted October 5, 2009 Hi , anyone know how (or if there is any code) that can prevent bots from indexing a certain page on your site.. Fro example I have a site let's cal it... www.blablablabla.com and s page on it called www.blablablabla.com/AboutMe.com and I don't want anyone to index or be able to find this AboutMe.com site (and not willing to make it https).. need sort of like a bot blocker.. thanks Link to comment Share on other sites More sharing options...
jlhaslip Posted October 5, 2009 Share Posted October 5, 2009 google htaccess bots for information.blocking known bots is as easy as setting rules in your .htaccess file.there are also scripts that function as bot traps, but try the .htaccess method firstbot trap: http://danielwebb.us/software/bot-trap/ Link to comment Share on other sites More sharing options...
podarum Posted October 5, 2009 Author Share Posted October 5, 2009 Thanks. would you have any code that I can insert for the AboutMe page? Link to comment Share on other sites More sharing options...
jlhaslip Posted October 5, 2009 Share Posted October 5, 2009 see the link here: http://danielwebb.us/software/bot-trap/use the bot trap only on any page you want to avoid bots on Link to comment Share on other sites More sharing options...
Kingy Posted October 8, 2009 Share Posted October 8, 2009 Look up robots.txt! Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.