podarum Posted October 5, 2009 Report Share Posted October 5, 2009 Hi , anyone know how (or if there is any code) that can prevent bots from indexing a certain page on your site.. Fro example I have a site let's cal it... www.blablablabla.com and s page on it called www.blablablabla.com/AboutMe.com and I don't want anyone to index or be able to find this AboutMe.com site (and not willing to make it https).. need sort of like a bot blocker.. thanks Link to comment Share on other sites More sharing options...
jlhaslip Posted October 5, 2009 Report Share Posted October 5, 2009 (edited) google htaccess bots for information.blocking known bots is as easy as setting rules in your .htaccess file.there are also scripts that function as bot traps, but try the .htaccess method firstbot trap: http://danielwebb.us/software/bot-trap/ Edited October 5, 2009 by jlhaslip Link to comment Share on other sites More sharing options...
podarum Posted October 5, 2009 Author Report Share Posted October 5, 2009 Thanks. would you have any code that I can insert for the AboutMe page? Link to comment Share on other sites More sharing options...
jlhaslip Posted October 5, 2009 Report Share Posted October 5, 2009 see the link here: http://danielwebb.us/software/bot-trap/use the bot trap only on any page you want to avoid bots on Link to comment Share on other sites More sharing options...
Kingy Posted October 8, 2009 Report Share Posted October 8, 2009 Look up robots.txt! Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now