confused and dazed Posted April 22, 2015 Share Posted April 22, 2015 Hello internet, Recently I have been working with curl sessions and scrapping data from webpages. I have been fairly successful until I tried to access data from pages that are username and password protected. I have the username and password so that's not an issue - but I am not able to get the data on the page. Any thoughts? $username='usr1'; $password='pswd1'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'somelink'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY); curl_setopt($ch, CURLOPT_USERPWD, $username.":".$password); $out = curl_exec($ch); if(curl_exec($ch) === false) { echo 'Curl error: ' . curl_error($ch); } echo $out; curl_close($ch); Link to comment Share on other sites More sharing options...
thescientist Posted April 22, 2015 Share Posted April 22, 2015 what's the error?? Link to comment Share on other sites More sharing options...
confused and dazed Posted April 22, 2015 Author Share Posted April 22, 2015 (edited) There is no error message. Maybe it has something to do with what I expect to see. From the echo $out; I expect to see the logged in page embedded in the current page. All I see is the login page embedded in the current page. I expected to see the user info page instead of the login page as well as have access to the source data from the user page so I can scrape the source code. Edited April 22, 2015 by confused and dazed Link to comment Share on other sites More sharing options...
Ingolme Posted April 22, 2015 Share Posted April 22, 2015 The login page is what's being served. If you want something else to be served, you will need to send a POST request with a proper username and password and get a cookie in return. Since curl probably won't save cookies for you, you'll have to manually read the Set-Cookie header and manually send a Cookie header when requesting a new page. Link to comment Share on other sites More sharing options...
confused and dazed Posted April 22, 2015 Author Share Posted April 22, 2015 Awesome! I have another string going for this topic so I will close this one down because the other one starts from the beginning. As usual thanks for the response!! http://w3schools.invisionzone.com/index.php?showtopic=52997&hl= Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now