Getting data from a DIV on another website.

[pandaking]

Hi I wrote a google gadget a little while back for displaying a kind of "Image of the day". It was easy because they just named the image with the current days date.The website has now been updated, and they now use a random method of naming the image.So, I need a way of being able to get the data contained in a DIV tag, to then display though my gadget.I guess it's sort of like an Iframe in HTML, but I couldn't work out how to just display one element of a page using that either.The command in a iGoogle Gadget is:

_IG_FetchContent(url, func)

For more info see: http://code.google.com/apis/gadgets/docs/remote-content.htmlWhen I posted this question on the iGoogle Developer group, I got this response:

When dealing with HTML, you can use other Javascript String methods tofind the bits you need. Here is for instance a page providing most ofthe Javascript String methods:http://www.quirksmode.org/js/strings.htmlWith combinaison of substr(), substring(), indexOf(), etc. you shouldbe able to get what you need.

Now I have been playing around for quite a while now and still can't work it out.Could anybody give me some pointers please?Many thanks,Panda

[Ingolme]

It returns the content of the given URL as text.I'd recommend reading the string with Javascript's XML parser, but most likely you're not receiving valid XML from the remote page.So you'll have to search for the string, something like this:Get the indexOf("<div id=\"something\">") or whatever the <div> has that makes it unique. And store it in a variable:M = yourString.indexOf("<div id=\"something\">")Get the indexOf() the first closing <div> tag using the previous value as a fromIndex and store it's content in a variable:N = yourString.indexOf("</div>",M)+5; //added 5 to include the closing </div> tag tooGet the substring() between them:finalString = substring(M,N);And there you have it, everything within the whole <div> element.

[pandaking]

It returns the content of the given URL as text.I'd recommend reading the string with Javascript's XML parser, but most likely you're not receiving valid XML from the remote page.So you'll have to search for the string, something like this:Get the indexOf("<div id=\"something\">") or whatever the <div> has that makes it unique. And store it in a variable:M = yourString.indexOf("<div id=\"something\">")Get the indexOf() the first closing <div> tag using the previous value as a fromIndex and store it's content in a variable:N = yourString.indexOf("</div>",M)+5; //added 5 to include the closing </div> tag tooGet the substring() between them:finalString = substring(M,N);And there you have it, everything within the whole <div> element.

Wow, you guys are good! I will go try this out now.Thanks very much Ingolme...

[Ingolme]

Don't thank me until you're sure it works. Somehow there always seem to be little problems when I write little scripts.

[pandaking]

Don't thank me until you're sure it works. Somehow there always seem to be little problems when I write little scripts.

And modest too! Well this is what I have so far. To try it out go to: http://code.google.com/apis/gadgets/docs/gs.html#GGEAdmittedly it's not quite working, but it's nearly there I think.

<?xml version="1.0" encoding="UTF-8" ?>	<Module>		<ModulePrefs 			title="Matt Cartoon"			title_url="http://www.telegraph.co.uk/core/Matt/pMattTemplate.jhtml?pPage=/core/Matt/pcMatt.jhtml"			description="This gadget adds the very popular Matt Cartoon from The Telegraph to your webpage, with option to choose background colour. Hope you enjoy ^^" 			height="350"			scaling="false"			screenshot="http://pandaking.googlepages.com/screenshot.png"			thumbnail="http://pandaking.googlepages.com/logo.png"					author="pandaking"			author_location="Brighton, UK"			author_email="shh@shhh.com" 			author_photo="http://pandaking.googlepages.com/pandaking_logo_70x100.png"			author_aboutme="Just a little panda eating some bamboo ^^"			author_link="www.red88music.co.uk"			author_quote="Yum Yum Panda Burgers!"		/>		<UserPref name="mycolour" display_name="Colour"			default_value="White" datatype="enum" >			<EnumValue value="Aqua" />			  <EnumValue value="Gray" />			  <EnumValue value="Lime" />			  <EnumValue value="Orange" />			  <EnumValue value="Pink" />			  <EnumValue value="Red" />			  <EnumValue value="White" />			  <EnumValue value="Yellow" />		</UserPref>		<Content type="html">		<![CDATA[			<div id="titleimage" align="center"><img src="http://pandaking.googlepages.com/title_image.png"/></div>		<div id="cartoon" align="center"></div>		<script type="text/javascript">						// Get userprefs				var prefs = new _IG_Prefs(__MODULE_ID__);				// Set the background colour according to the mycolour userpref				var element = document.getElementsByTagName('body')[0];				element.style.backgroundColor=prefs.getString("mycolour");											// Get the data from site				_IG_FetchContent('http://www.telegraph.co.uk/news/matt/', function (responseText) {											// Get the Url of Image				  var M = responseText.indexOf("<div class=\"mattimage\">")				var N = responseText.indexOf("</div>",M)+5; //added 5 to include the closing </div> tag too								var url = substring(M,N);    			  // Output Image				  _gel('cartoon').innerHTML = '<img src="' + url + '" alt="" />';  			});						]]>		</Content>	</Module>

[Ingolme]

You're already getting an <img> tag inside the code, so change this line: _gel('cartoon').innerHTML = '<img src="' + url + '" alt="" />';for this: _gel('cartoon').innerHTML = url;

[pandaking]

Yes, you're right - I seem to be incorrectly using the method "_gel"I tried your suggestion and still no luck.I might try using "document.write" function to see what exactly our variable "url" contains...Thanks again

[Ingolme]

Use alert() to test variables. document.write() will probably clear the content of your page and then keep it in a loading state.I'd say the _gel() function is just a shortening for getElementById(), but I'm not sure. Do you know what's inside the _gel() function?

[pandaking]

Well I found one big problem, I forgot the line:

 </script>

[pandaking]

Use alert() to test variables. document.write() will probably clear the content of your page and then keep it in a loading state.I'd say the _gel() function is just a shortening for getElementById(), but I'm not sure. Do you know what's inside the _gel() function?

OK, I will try this.Cheers

[pandaking]

Well the _gel function is working fine.If I manually set the variable "url" to be an image of some kind it gets displayed.So I have to assume the problem is within this part:

 // Get the data from site				_IG_FetchContent('http://www.telegraph.co.uk/news/matt/', function (responseText) {											// Get the Url of Image				var M = responseText.indexOf("<div class=\"mattimage\">")				var N = responseText.indexOf("</div>",M)+5; //added 5 to include the closing </div> tag too								var url = substring(M,N);

Scratching my hair out

[Ingolme]

When you run the program, what do you get when you output the url variable?

[pandaking]

When you run the program, what do you get when you output the url variable?

Not much. Heck I am even having trouble with the alerts! :)See for yourself: http://code.google.com/apis/gadgets/docs/gs.html#GGEJust copy and paste the following code into the editor:

<?xml version="1.0" encoding="UTF-8" ?>	<Module>		<ModulePrefs			title="Matt Cartoon"			title_url="http://www.telegraph.co.uk/core/Matt/pMattTemplate.jhtml?pPage=/core/Matt/pcMatt.jhtml"			description="This gadget adds the very popular Matt Cartoon from The Telegraph to your webpage, with option to choose background colour. Hope you enjoy ^^"			height="350"			scaling="false"			screenshot="http://pandaking.googlepages.com/screenshot.png"			thumbnail="http://pandaking.googlepages.com/logo.png"					author="pandaking"			author_location="Brighton, UK"			author_email="shh@shhh.com"			author_photo="http://pandaking.googlepages.com/pandaking_logo_70x100.png"			author_aboutme="Just a little panda eating some bamboo ^^"			author_link="www.red88music.co.uk"			author_quote="Yum Yum Panda Burgers!"		/>		<UserPref name="mycolour" display_name="Colour"			default_value="White" datatype="enum" >			<EnumValue value="Aqua" />			  <EnumValue value="Gray" />			  <EnumValue value="Lime" />			  <EnumValue value="Orange" />			  <EnumValue value="Pink" />			  <EnumValue value="Red" />			  <EnumValue value="White" />			  <EnumValue value="Yellow" />		</UserPref>		<Content type="html">		<![CDATA[			<div id="titleimage" align="center"><img src="http://pandaking.googlepages.com/title_image.png"/></div>		<div id="cartoon" align="center"></div>		<script type="text/javascript">						// Get userprefs			var prefs = new _IG_Prefs(__MODULE_ID__);				// Set the background colour according to the mycolour userpref			var element = document.getElementsByTagName('body')[0];			element.style.backgroundColor=prefs.getString("mycolour");							// Get the data from site			_IG_FetchContent('http://www.telegraph.co.uk/news/matt/', function (responseText) {							// Get the Url of Image			var M = responseText.indexOf("<div class=\"mattimage\">");			var N = responseText.indexOf("</div>",M)+5;			var url = substring(M,N);			var test = "Test Data";  			// Output Image			_gel('cartoon').innerHTML = url;  			});										alert(url);			alert(M);			alert(N);			alert(test);				</script>		]]>	</Content>	</Module>

[Ingolme]

I forgot to mention: You're supposed to be taking a substring from somewhereThat somewhere is the rsponseText!var url = responseText.substring(M,N);

[pandaking]

I forgot to mention: You're supposed to be taking a substring from somewhereThat somewhere is the rsponseText!var url = responseText.substring(M,N);

Holy *smurf*...IT WORKS!!! Dude, you are a bloody legend. Thanks so much for all your help.I have learned a lot from you. Now to find some other projects I can continue to learn from :)Thanks again mate...

[pandaking]

Hi again, back with another question:Is there a way to do this with an image which is in a div without an ID assigned to it?It's in a class, but many other images are in divs with the same class.The image in question is:...matt_hol.gif / ...matt.gifwhich you can find on the main page here:http://www.telegraph.co.ukThe reason for this is depending if he goes on holiday, the image changes to alert people the writer is on holiday.This way users won't worry if they see the same cartoon for a few days in a row.I was thinking I could perhaps do a search of the whole page for the two terms "matt.gif" or "matt_hol.gif" and display the one it finds?Not sure if it's the most effective way or not.Cheers...

[pandaking]

I was thinking something like this:

 // Get the data from site_IG_FetchContent('http://www.telegraph.co.uk/', function (holiday) {// Get the Url of Imagevar hol = holiday.indexOf("matt_hol.gif");if (hol<1){_gel ('title').innerHTML = "http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif";}else{_gel ('title').innerHTML = "http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif";}});

I have tested it, and I get a number for variable hol, but the if statement doesn't seem to be working.If I use the _gel function to output hol then it displays the number - it just doesn't like the image...

[pandaking]

Right, so I have to put the img tags in

_IG_FetchContent('http://www.telegraph.co.uk/', function (holiday) {var hol = holiday.indexOf("matt_hol.gif");if (hol<0){_gel ('title').innerHTML = "<img src=http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif />";}else{_gel ('title').innerHTML = "<img src=http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif />";}});

I still have a problem with the if statement however.The variable hol is definately greater than zero (it's like 3241), so why would it be displaying the second image?

[Ingolme]

I was thinking something like this:

 // Get the data from site_IG_FetchContent('http://www.telegraph.co.uk/', function (holiday) {// Get the Url of Imagevar hol = holiday.indexOf("matt_hol.gif");if (hol<1){_gel ('title').innerHTML = "http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif";}else{_gel ('title').innerHTML = "http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif";}});

I have tested it, and I get a number for variable hol, but the if statement doesn't seem to be working.If I use the _gel function to output hol then it displays the number - it just doesn't like the image...

If a string doesn't exist, the indexOf() will return -1, so that's what you'd have to search for.And also, for an image, you'd have to write the whole image tag in the innerHTML.

var hol = holiday.indexOf("matt_hol.gif");if (hol == -1) {_gel ('title').innerHTML = "<img src=\"http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif\" alt=\"[image]\">";} else {_gel ('title').innerHTML = "<img src=\"http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif\" alt=\"[image]\">";}

[pandaking]

I'm an idiot - the "<" needs to be a ">".Problem solved EDIT:Just seen your post ingolme. I didn't realise it returns -1 if the is no result, I assume I can do:

if (hol>-1)

Or can you not use -1 in that?

[pandaking]

As far as I can tell this seems to be working, could you by any chance just check it to make sure there are no other nooby mistakes please?

_IG_FetchContent('http://www.telegraph.co.uk/', function (holiday) {var hol = holiday.indexOf("matt_hol.gif");if (hol > -1){_gel ('title').innerHTML = "<img src=http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif />";}else{_gel ('title').innerHTML = "<img src=http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif />";}});

Thanks again ingolme for you help, you're a star!

[Ingolme]

Well, it probably works just well like this (if it doesn't, I really can't see why at the moment), but to make valid (X)HTML you should put quotes around the attributes and add an alt attribute to the images:_gel ('title').innerHTML = "<img src=\"http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif\" alt=\"[image]\" />";_gel ('title').innerHTML = "<img src=\"http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif \" alt=\"[image]\" />";

[pandaking]

Well, it probably works just well like this (if it doesn't, I really can't see why at the moment), but to make valid (X)HTML you should put quotes around the attributes and add an alt attribute to the images:_gel ('title').innerHTML = "<img src=\"http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt_hol.gif\" alt=\"[image]\" />";_gel ('title').innerHTML = "<img src=\"http://www.telegraph.co.uk/portal/graphics/TargetedContent/images/matt.gif \" alt=\"[image]\" />";

Will do, thanks very much