Matar Posted December 26, 2006 Share Posted December 26, 2006 hi any one know about web spider thats used in search enginei am trying to make a search engine using php and mysql but i don't know anything about web spider any have a script to get the page content for exmaple small program when i give it the url , he will get the metatag information or get the h1 text ..etc kindly regardsmatar Link to comment Share on other sites More sharing options...
Guest !!*AStov Posted December 26, 2006 Share Posted December 26, 2006 try http://www.phpdig.net/ PhpDig is a web spider and search engine written in PHP, using a MySQL database and flat file support. PhpDig builds a glossary with words found in indexed pages. On a search query, it displays a result page containing the search keys, ranked by occurrence. Link to comment Share on other sites More sharing options...
Matar Posted December 27, 2006 Author Share Posted December 27, 2006 hi thank you man ,,, but the link not work " not authroized to view this page " matar Link to comment Share on other sites More sharing options...
jesh Posted December 27, 2006 Share Posted December 27, 2006 This article was written for C# and ASP.NET, but you may be able to look past the code and read the article for a decent explanation on how a spider works (and is built).http://www.codeproject.com/aspnet/Spideroo.asp Link to comment Share on other sites More sharing options...
aspnetguy Posted December 27, 2006 Share Posted December 27, 2006 I wrote a small spider in C#. All you need to do is "screen scrape" the target url. Then I parsed out the links within the page and inturn scraped them aswell, and so on and so on.Do a google search there are plenty of screen scrape libraries available for PHP Link to comment Share on other sites More sharing options...
MrAdam Posted December 28, 2006 Share Posted December 28, 2006 hi thank you man ,,, but the link not work " not authroized to view this page " matar try: http://phpdig.net/ Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now