Working with Remote Directories

[ThePsion5]

Hi,I'm trying to write a script in PHP that obtains and compiles meta-tag information about a specified website. I modified a script i found at php.net to iterate through all html files and directories when given a specified directory, but this appears to only work on a local host or directory (any attemps to use this by specifing a remote host) return the following error:

failed to open dir: not implemented in (the file) (the line)

Here's my PHP code, sorry if it's a bit lengthy:

function read_dir($dir, $depth){   global $MaxDepth;   $path = opendir($dir);   $info = pathinfo(dirname($path));   $i = $info['dirname'];   while (false !== ($file = readdir($path)) && $depth <= $MaxDepth)   {     if($file!="." && $file!="..")     {       if(is_file($dir."/".$file))       {         $files[]=$file;       }       else       {         $dirs[]=$dir."/".$file;       }     }   }   if($dirs)   {     natcasesort($dirs);     foreach($dirs as $dir)       {         read_dir($dir, $depth+1);       }   }   if($files)   {     natcasesort($files);     foreach ($files as $file)     {        search($dir . '/' . $file);     }   }   closedir($path);}

[justsomeguy]

When you call the function to get the error, what parameters do you pass?

[ThePsion5]

Here are all the error messages i recieve:

Warning: opendir(http://www.google.com) [function.opendir]: failed to open dir: not implemented in (my file and line #)Warning: readdir(): supplied argument is not a valid Directory resource in (my file and line #)Warning: closedir(): supplied argument is not a valid Directory resource in (my file and line #)

I get the feeling that I'm going about this the wrong way, but I was also under the impression that i could pass an http reference as a path. Perhaps I was mistaken...

[justsomeguy]

The HTTP path must support directory listing. http://www.google.com does not support directory listing, when you go there you get the search page, not a directory list. If you can visit the URL and see a list of files, then you can use opendir with that URL, or else you can't. However, in PHP4, only the file:// protocol supports directory listing. You can't use this function to list an arbitary server's contents, that's a security issue that the servers block.http://www.php.net/manual/en/function.opendir.php

[ThePsion5]

Hm...that's a shame. I was hoping that I could create a PHP function that would scan the HTML files in the root directory of the website and compile information on their meta-tags. Is there a way to do this without requiring ftp access to the server?

[justsomeguy]

As far as I know it is not possible to get a list of files on an arbitrary server, if you think about that it is a security problem. If I run a server, I do not want my server giving out a list of files.What you would need to do is build a crawler, it would load the index page, parse the html, and find links inside the html to follow. That's how search engines do their own indexing.