Xml Databases

[mstransky]

Just looking for some input from other people who dabbled with XML during it's Highlights. I created many xml databases and asp. 1.) I have xml banner databases which randomly chooses a banner to display on a client site. It also counts impressions and click throughs. 2.) I have created also online catalog xml databases displaying magazine archives that is very fast with over 10k images, 2k menus, and a few hundred articles from those archives.3.) an many other various xml databases with admin dashboards.My questions to those people is....I have seen a decline in xml database use for the past two years. XML database scripts are being replaced with many xml purchase software links.those scripts out there in the begining were full of holes with no security to protect locations and datafiles.I have done very well with asp to lock down xml databases, I was wondering if any PHP guru's seen the same thing with xml and asp.also is there any myths/truth floating around that makes people not use xml/asp or xml/php?Just wondering what some of the lastest needs, wants and let dones that are going around.Thanks

[boen_robot]

XML is not suitable for every data need, and neither are RDBMS systems (MySQL, MS SQL, etc.).For the banners, it's certainly a good case, especially if there aren't very much banners to choose from. For the admin dashboards - maybe. Depends on what you do with this dashboard.I've been at both ends, and at each, when you think you're holding THE hammer, everything appears to be nails. But since then, I've learned to choose one or the other on a case by case basis, and then finally combine the two to produce best results.My general rule of thumb is to judge whether the data I'm presenting is relational or heirarical.For example, if I'm describing items in a catalog (price, category), and then want to find either all products with a certain category or all products with a certain price (from any category), I'd go for RDBMS. Using DOM or SimpleXML to read a large catalog is very memory consuming (an extreme case: ~2MB file can translate to 12MB RAM), and writing and maintaining an XMLReader parser for that is not only tedious, but doesn't really yeld any benefits.If I'm having a simple page that has data, or I have interface data for the catalog above (e.g. The word "Price:", after which the price is supposed to follow), I'd declare all of this content in an XML file, and then I'd process it, and replace the relevant part with the DB results. For even simpler data (like "About us" and "Contacts" pages), there isn't even a need for post-processing - directly processing the file is enough. And by "processing" I mean passing it to XSLT, which joins the page with the common stuff, and possibly does some content checks (e.g. add query string parameters to links in the content, etc.).P.S. I'm writing in your copy in the "General" forum, as this is a language wide issue (ASP, PHP, etc.), plus it's a comparrison of XML and SQL, so it's not suitable for those forums either... this forum is the most neutral territory, so to speak.

[justsomeguy]

Every system has its pros and cons, but the main thing I like about databases is the fact that they include software that is specifically designed to retrieve or update the data as quickly as possible, often much faster than you can make a system open and parse the files. A properly indexed database can be very fast at most operations.

[mstransky]

Every system has its pros and cons, but the main thing I like about databases is the fact that they include software that is specifically designed to retrieve or update the data as quickly as possible, often much faster than you can make a system open and parse the files. A properly indexed database can be very fast at most operations.

"Boen Robot" and "Justsomeguy" I had my head buried in work when I got that spam email yesterday, looking at both your posts I hold you both in high regards of how you have replied to others post and the information you replied with. I may have been hasty in how quick I point some things out, but exaplianed later by examples.I agree with Boen Robot that it depends on how it is display for which methood is used. Would you look at www.wartimepress.com that is where I made a beta code for those guys to just display an archive of images from wwii collection. I would say it is pretty quick compared to what others wrote that a xml can bog down.The first page may load slow because of the random backdrops on page load, theres like 20 random images.after they are done, try looking at us magazines, army yank. or just this linkhttp://www.wartimepress.com/archives.asp?T...q=42&FID=41this code is in beta test, I dont have the dashboards protected with sessions yet. I plan to share some of these codes after I put an open source zip together. Then let others modify file and folder locations later.right nowmenus xml 258kbrecords xml 1.6mband articles xml is 2.9mbQ."justsomeguy", how would you rate if an xml is bogged down or speed, and what do you think of its speed from a web visitors point of view?Q."Boen Robot", I have tried numours ways to take the xml node preserving the white space to display the articles like this locationhttp://www.wartimepress.com/archive-public...=681&FID=68pick any one of the articles.The only way I have been able to retain the white spaces and tab returns for pasting a word doc text into the xml input. I am stuck useing this code. <textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly"><xsl:value-of select="DD"/></textarea> I would love to beable to wrap text in html output, I have tried about 4 dozen attempts, but most times it would just make the page go from say 500px and push it out to 8345px disregaurding all format restrictions.how to display it like in a text area without the textarea white box and scroll bars?------------ps. I am looking at making a few open source scripts for sharing. I have not read any php codes yet to mach php or asp to do xml databases, that is why I was looking for input to go down that road or not, being whorth while for now.

[justsomeguy]

"justsomeguy", how would you rate if an xml is bogged down or speed, and what do you think of its speed from a web visitors point of view?

If you've already got an XML structure set up, which it sounds like you do, then you could definitely put together a meaningful test to see which one is quicker. I would think that a traditional database would be faster than using XML, but the speed difference is probably going to be less than a second, so the user probably wouldn't see much of a difference until the data set gets bigger. But, since you've already got your XML set up, it might not be too bad to design a database structure to hold the same info, and maybe build a script to go through the XML and add everything to the database. If you have the same data in both an XML structure and a database, then you can build a couple test pages to keep track of the time that it takes for various operations, such as lookup or update operations. For tests like that it's usually best to run them in a loop and then take an average. I would assume that the database is going to be quicker, but for data sets below a certain size the XML might be faster.

[mstransky]

If you've already got an XML structure set up, which it sounds like you do, then you could definitely put together a meaningful test to see which one is quicker. I would think that a traditional database would be faster than using XML, but the speed difference is probably going to be less than a second, so the user probably wouldn't see much of a difference until the data set gets bigger. But, since you've already got your XML set up, it might not be too bad to design a database structure to hold the same info, and maybe build a script to go through the XML and add everything to the database. If you have the same data in both an XML structure and a database, then you can build a couple test pages to keep track of the time that it takes for various operations, such as lookup or update operations. For tests like that it's usually best to run them in a loop and then take an average. I would assume that the database is going to be quicker, but for data sets below a certain size the XML might be faster.

I never did a sql code yet, and never linked one yet either. the host provider I had in the past was military grade servers and locked down the servers so bad you could not even get a guest book to work, he would complian if your site was not locked down from hackers and stuff like that, or put his servers under a memory strain. I am now very happy with the hosting I have had the past year. Now that I have more freedom to do anything I want to persue.if interested...I wrote three seprate xml files, one to hold catagories as menu links, then a xml that holds the publictaions titles, image locations, names etc...then the article pages for the text documents.structure...just like ms access...Each xml file has an auto number like access would, then each xml refers back to a parent key column ID.I wrote the xml files to acted just like msaccess data base tables, or like how excel refers to other books and sheets.both the display pages and dashboard pages use asp vbscript to run variables/arrays the xml and output with xslt. the only differance is the admin dashboards have edit, add, modify buttons, once clicked run asp with "dim's" to modify any of the xml files.I never read one book on asp, I just looked at similair codes to get ideas, then put all the best into one. I have not had a chance to even look at php code yet, that is why I asked which is worth while. I have mess with some really slow xml codes which were sad and were not very large at all. The beta one you looked at has like over 10,000 image records and all the data that goes with each record.justsomeguy, Thank you for your input about the speed that not much "speed" between the two types. Last week I had a few people play on my dashboard admin screens and have ask me to build a huge srcipt to run bank transactions and batch files. which I am still debating getting involed in that stuff.Q.justsomeguy, I also have been asked about making this code into a store front, I have no problem with building that but have never seen what an actual paypal coding looks like. Do you know of a place on the web which explains what codes commands are needed to submit a value or charge against a paypal accout? Or what paypal is looking for as a command line?PS. I have enjoyed talking for the first time in three years, most forums were not responsive with sharing code or ideas.So I might shot out to much info at one time.

[boen_robot]

right nowmenus xml 258kbrecords xml 1.6mband articles xml is 2.9mb

I don't like those numbers. You're way too close to the extreme case I mentioned above. Even if your page loads fast, you're sucking WAY too much memory out of your server. There are a lot of possible consequences, like the host banning you (I've had this one client who had thumbnails generated by PHP dynamically and without any sort of caching...), or you eventually getting "Out of memory" error messages... and soon that is.Storing the menus is fine. But XML is not meant for storing large dataset like articles and records. You should really consider using a database for that. The only way not to be sucking a lot of memory is to use XMLReader, but like I said, maintaining the handler can be hard (probably harder than maintaining a DB) and doesn't give any benefits over DBs. Heck, in theory, processing time is actually slightly slower with XMLReader on large data sets... though I haven't done any practical benchmarks to confirm that part.

"Boen Robot", I have tried numours ways to take the xml node preserving the white space to display the articles like this locationhttp://www.wartimepress.com/archive-public...=681&FID=68pick any one of the articles.The only way I have been able to retain the white spaces and tab returns for pasting a word doc text into the xml input. I am stuck useing this code.<textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly"><xsl:value-of select="DD"/></textarea>

I'm guessing you mean to preserve the contents of the "DD" element, correct? If so, in your XML at that point, add the attribute

xml:space="preserve"

When XSLT receives the XML, it will get a text node with no stripped white spaces. In addition (if that doesn't work), try to add it into the XSLT too, like:

<textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></textarea>

[mstransky]

I don't like those numbers. You're way too close to the extreme case I mentioned above. Even if your page loads fast, you're sucking WAY too much memory out of your server. There are a lot of possible consequences, like the host banning you (I've had this one client who had thumbnails generated by PHP dynamically and without any sort of caching...), or you eventually getting "Out of memory" error messages... and soon that is.Storing the menus is fine. But XML is not meant for storing large dataset like articles and records. You should really consider using a database for that. The only way not to be sucking a lot of memory is to use XMLReader, but like I said, maintaining the handler can be hard (probably harder than maintaining a DB) and doesn't give any benefits over DBs. Heck, in theory, processing time is actually slightly slower with XMLReader on large data sets... though I haven't done any practical benchmarks to confirm that part.I'm guessing you mean to preserve the contents of the "DD" element, correct? If so, in your XML at that point, add the attribute

xml:space="preserve"

When XSLT receives the XML, it will get a text node with no stripped white spaces. In addition (if that doesn't work), try to add it into the XSLT too, like:

<textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></textarea>

I will give that a try those four commandsunselectable="on" wrap="soft" contenteditable="false" readonly="readonly"that is one way I did not try.I had with or without wrap="hard" but never tried softI thought hard was like hard force it to wrap.and it push the right side of the template off the screen by like 400%On memory I have clear memory not to use up much memory in my asp commands.here is some stats.So far the site has only used Monthly limit: 1000.00 GB Used to date: 0.41 GB 250 Visits 1,731 Pageviews 6.92 Pages/Visit 40.40% Bounce Rate 00:02:28 Avg. Time on Site 73.60% % New Visits last month Monthly limit: 1000.00 GB Used to date: 1.02 GB 1,308 Visits 15,119 Pageviews 11.56 Pages/Visit 39.07% Bounce Rate 00:05:42 Avg. Time on Site 62.54% % New Visits I am figuring I would need like 15,000,000 Pageviews or due to size growth of the 3mb of database files before the band width is hit.would you say 3mb for 15,000,000 Pageviews of bandwidth would be fair?That beta script could be broken down into country/types publications xmls vs loading all records to filter them. BUT I will do that if the file becomes to big.

[mstransky]

I don't like those numbers. You're way too close to the extreme case I mentioned above. Even if your page loads fast, you're sucking WAY too much memory out of your server. There are a lot of possible consequences, like the host banning you (I've had this one client who had thumbnails generated by PHP dynamically and without any sort of caching...), or you eventually getting "Out of memory" error messages... and soon that is.Storing the menus is fine. But XML is not meant for storing large dataset like articles and records. You should really consider using a database for that. The only way not to be sucking a lot of memory is to use XMLReader, but like I said, maintaining the handler can be hard (probably harder than maintaining a DB) and doesn't give any benefits over DBs. Heck, in theory, processing time is actually slightly slower with XMLReader on large data sets... though I haven't done any practical benchmarks to confirm that part.I'm guessing you mean to preserve the contents of the "DD" element, correct? If so, in your XML at that point, add the attribute

xml:space="preserve"

When XSLT receives the XML, it will get a text node with no stripped white spaces. In addition (if that doesn't work), try to add it into the XSLT too, like:

<textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></textarea>

I just tried that <textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></textarea>And it did notmake much changeI then did <text unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></text>And that is how I want it to display at the same html level without being contained in a white textbox without scroll bars.OR I would accept a white text box if the textarea would auto size in hieght to fit to the size of text.Kind of auto hieght in a white box, when I do that it hides about 60% of an article.http://www.wartimepress.com/archive-articl...=23&FID=681I left that look up temporaly but will need to put it back shorlty

[mstransky]

I put that xslt code back the wat it was, I will ask that q on a new thread another day.I thank you both for your input, I will spend some time putting together a zip file of it to be shared.I am willings to do a test of a large sql and xml the way I put it together and bench mark its useage. I know other people did such benchmarks in the past and how much bandwidth was eaten up. But that was with older scripts and some dlls have been updated and server side handlers have been updated since.Is it worth making a bench test again? any way thanks to you both for todays fgeedback!

[justsomeguy]

justsomeguy, I also have been asked about making this code into a store front, I have no problem with building that but have never seen what an actual paypal coding looks like. Do you know of a place on the web which explains what codes commands are needed to submit a value or charge against a paypal accout? Or what paypal is looking for as a command line?

I've only set up one site to really use most of the features PayPal offers, which definitely took quite a bit of trial and error. Most of the best resources are on the PayPal site. You'll need to sign up for a developer account with them, which will give you access to all of the API documentation and test servers and things like that. You can set your site up to use the PayPal sandbox where you can send payment information to be processed as normal, except the transactions won't actually move money around until you use the live servers.There are a few PayPal APIs that might be useful. One is the Instant Payment Notification, IPN, where you set up a script on your server to receive requests from PayPal. You'll get a request when something happens like a payment goes through, or gets cancelled or refunded, or if it's a subscription you'll get a notice every time a payment gets made or the subscription renewed or cancelled, etc. You configure the URL of your script in your PP account. Your script collects the information from PP and sends it straight back to them to ask if it actually came from them, and once they verify it then you can do whatever processing you need to do with it. They have examples for that stuff in some giant PDFs buried around their developer site.The other method is when someone clicks the pay button on your site, then goes to PP to pay for it, then gets redirected back to your site once the payment is finished. I can't remember what they refer to that as, but that's probably a more common situation. They have code examples for that one on their developer site also.

[mstransky]

GREAT!!!!I was asked to make a number of things to use paypal, like a site permission like you would do with ancestry to view database articals, download page copyies and stuff like taht.Or another site to place an order for a custom made item to start the process.Also membership or donations on another site.I will look for those api or sandbox to get started with that and see if I can put into one of my xml databases.

[boen_robot]

On memory I have clear memory not to use up much memory in my asp commands.here is some stats.So far the site has only used Monthly limit: 1000.00 GB Used to date: 0.41 GB 250 Visits 1,731 Pageviews 6.92 Pages/Visit 40.40% Bounce Rate 00:02:28 Avg. Time on Site 73.60% % New Visits last month Monthly limit: 1000.00 GB Used to date: 1.02 GB 1,308 Visits 15,119 Pageviews 11.56 Pages/Visit 39.07% Bounce Rate 00:05:42 Avg. Time on Site 62.54% % New Visits

You're talking about bandwidth, not RAM consumption. The bandwidth is the total data that the server sends to clients. The output of your page is in normal size, so you don't have a problem there. The RAM consumption is not visible in those stats. I don't know how you can get the current RAM usage in ASP.RAM is consumed every time you set a variable or execute a function, and is freed when you unset that variable, when the variable is out of scope or when the request ends (whichever comes first), but the RAM that each variable consumes depends on it's size and type. A boolean will take b+1bytes, whereas a string of 5 characters will take at least b+5bytes. "b" in this case is whatever amount of memory the ASP engine needs to store "any" variable. The size that the DOM representation of an XML takes is much larger than its text representation. An XML with a text representation (a string...) of about b+2097152bytes takes about b+12582912bytes. And again, this your XML alone. Coupled with everything else you might want to do, it can result in memory consumption you'd never reach with a database. Actually, this 2->12 mapping I'm describing is for PHP. PHP has a smaller "b", so ASP may have even bigger issues.The point is that "something" happens once you start using too much RAM. IIS (7 at least; don't know for earlier ones) has the ability to automatically kill all ASP's and CGIs using more than a predefined amount of memory within their "application pool". If the host doesn't have that enabled, it will probably do so at one point, or ask you to "optimize your code" before finally banning you (even if you're a paid customer, your memory consumption disturbs everyone else - they'll sacrifice one monthly payment for the sake of n-1 monthly payments, "n" being all of their customers).

I just tried that <textarea unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></textarea>And it did notmake much changeI then did <text unselectable="on" wrap="soft" contenteditable="false" cols="65" rows="50" readonly="readonly" xml:space="preserve"><xsl:value-of select="DD"/></text>And that is how I want it to display at the same html level without being contained in a white textbox without scroll bars.OR I would accept a white text box if the textarea would auto size in hieght to fit to the size of text.Kind of auto hieght in a white box, when I do that it hides about 60% of an article.http://www.wartimepress.com/archive-articl...=23&FID=681I left that look up temporaly but will need to put it back shorlty

Add xml:space="preserve" in your XML then. This is an XML parser issue, so it's unavoidable.

[mstransky]

Actually, this 2->12 mapping I'm describing is for PHP. PHP has a smaller "b", so ASP may have even bigger issues.

Dont get me wrong I am going out on a thought here....Ok that is some good pro con's between PHP and ASP, 1.) Maybe that is why you see more and more falling off of xml and more sql with many searches on the web.2.) If PHP has a different way (less ram) doing a function then that could be a big factor to rewrite a script in PHP vs. ASP.3.) but one draw back is that, lately the big push with asp.net seems to be married at the hip with xml controls and databases? These comments gives me something to go look at other sites to find out more. I have been an energy management controls designer/programmer for over 20years. I worked heavy with web base internet interfaces to system controls that tied into msaccess sql databases (I never had to line code an sql connection). Those access databases would corrupt from time to time and lose the whole data block. At the same time I was into personal website stuff I have never had a desire to merge sql with a website so i went with xml. if one file(sheet) would go bad (not that it ever happened) you would only lose one sheet(file) not all the seprate table(files).It would be intresting to talk to my tier three techs if there is a why to track ram usage on the server side. I was halted at first after a week, but was told by them that it was no more then a permission, after they looking into my programming they allowed my codes to run freely. I was told my stuff did not even make a dent/differance no more that any other database useage. I take it that is where the server haults came in to effect, because I was not MSACCESS and had no (limit) permission loop because of an (Identified web standard application).This brings me to think about this....If one has two identical tables of data, 15,000 rows and 30 colubmns across.Both an XML parse and SQL will have to load, filter and sort the same block of information (file size the same).Both will display the data in html output forms as well. so it has to be in the command operation (mapping) where the ram is used up, not the data source format.I will bounce that off a co-worker why, when, and where, does the ram useage get abused (wasteful) more with one then the other during a compile commands of an operating language command process handling files. I am curiuos to compare the command routines in size, and see where at is actually at.boen robot, I have read many threads were people are asking how to sql to an xml file, I guess that can be done? It makes me think why they would rather sql to an xml then an access file. I guess there have been allot of apple an orange comparrisions in the past, and how they actually did speed results and ram stats a few years back. If so, it seems more and more, the better of both are slowing getting to a common playing field, since then(6years ago).---------------------------------------------------On the xslt, yes I figure that I would be stuck with it like that for a while, I can live with it for now till say new updates come out for xslt formats.Add xml:space="preserve" in your XML then. This is an XML parser issue, so it's unavoidable.

[mstransky]

Boen Robot, Here is what I am useing.<%var objXMLDoc = Server.CreateObject("MICROSOFT.FreeThreadedXMLDOM");objXMLDoc.async = false;objXMLDoc.load(Server.MapPath("../thefile.xml"));var xmlQuery = "//ITEM[iD = '"+ request +"']";var objNodes = objXMLDoc.selectNodes(xmlQuery);var xsl=Server.CreateObject("MICROSOFT.FreeThreadedXMLDOM");xsl.async = false;xsl.load(Server.MapPath("build-cat.xsl"));var xslt = Server.CreateObject("Msxml2.XSLTemplate");xslt.stylesheet = xsl;xslProc = xslt.createProcessor();xslProc.input = objXMLDoc;xslProc.addParameter("sortBy", "CC"); xslProc.addParameter("strXPath", objNodes); xslProc.transform();var sortedData=xslProc.output;Response.Write(sortedData)%>just to give you actual fact of what i use and how.

[mstransky]

its about 6 hrs since my last post and I found out some stuffmany people mistake MSXML knowing that it consume allot of memory, but DOM parser is many times faster than MSXML, and consumes far less memory. So when many people think xml databases they sometimes mistake a xml database as a massive memory user.also programming is like art work, depending on how an xml file is designed it can kill a servers memory.Example data fileCols AA - EEID | name first | Name Last | Age | etc...compared to an xml filethose like <personID fname="mickey">and those like <personID><firstname>Mickey</firstname> <lastname>Mouse</lastname><age>143</age><Country>United States<Country><State>Florida<State>etc...</personID>now complie that by 1000's records for each datablockevery text tag takes memory to handle as well as the information.compared if broken down like<record> <ID></ID> <AA>Mickey</AA> <BB>Mouse</BB> <CC>143</CC> <DD>3<DD> <EE>12<EE>etc...</record>then on another xml like how Access does itTables: Name Country StateIf the country was needed to be display 3 would = United States 3 repeated is much less then "United States" being repeated each record.also <DD>3</DD> being repeated is much less than <Country>United States<Country> for each record.AND if any one says they need to output the database it is very easy to display with xslt<DD> as "Country" just as <Country> as "Country" would be.Just like if you had a Employee database, it is better to create a reoccurring table likebirth places, or what state they live in, then to keep repeated the same State name over and overOr even company accounts as to which Sales person they belong to.To dont put "Henry Mark Thompson III" under every account who they sales agent is.You create a sales agent table on a seprate file"Henry Mark Thompson III" = 3So instead of repeating 25 sales peoples full names over and over in a main xml tableyou give them key ID's.-------------------------------------------So what I have found out is some people have made ONE HUGH XML file making to many repeats causing the file to become clunky and that is would over work a servers memory each pass.Q.does that make sense the way I explianed it?I believe it is all in the design how it is setup.same thing goes for access data bases, if all the tables were pushed from 5 cols to 26 cols repeating all data text in each record set ALL ON ONE TABLE, that mysql or even mssql would ###### up alot of memory on the server having to make a pass each time.--------------------------That makes me wonder what those bench test where done comparing "php mysql" vs. "asp mssql" vs "php/asp xml" and what the actual datablock structure was set up as.I hope I make sense on that the biggest problem is the programmers database design/latout which plays the largest factor in memory used on the server.would you agree?

[boen_robot]

Does the "strXPath" parameter work as expected? Most XSLT processors pass parameters as strings only (and AFAIK, the XSLT specification requres that XSLT parameters be strings), so I'd expect this to only return the text contents of the first element matching the XPath expression.RAM gets consumed at XML parsing. In order for XSLT to work, the XML source and XSLT files are both converted into DOM, and that DOM is then processed by the XSLT processer. The size of the DOM depends on the size of the source document, and the number of nodes in it to be more precise. Besides storing the content and name (if any) of the node, additional data about each node (such as a reference to it's parent, childNodes, position, etc.) must also be gathered and stored. To reduce CPU cycles, some XML parsers also store additional data that may otherwise be fetched at request time, which further increases RAM consumption.Also, a database doesn't exactly "parse" the data into a DOM-like structure. The data in databases is stored in a binary, DOM-like, i.e. "ready to use" structure, thereby removing that overhead.As far as data corruption goes, it all depends on how well the application is written. Both XML and SQL have equal chances of getting corrupted with a bad programmer in place. SQL can get corrupted when you execute a statement that later makes the application unable to handle the new data it entered. XML can get corrupted when you use DocumentFragment instead of creating DOM nodes, or use an XMLReader/XMLWriter duo that doesn't flush properly. Both can get corrupted if the data is used by two or more application, and the second one performs changes that the first can't understand, or if both applications operate at the same time, writing different stuff to the same place. Both have defences ("LOCK TABLE" and "XML Schema", respectively), but have to be explictly used by the programmer. In the case of data concurrency (writing to the same file/table by two or more apps), databases are better at handling those cases... you could potentically be good at it yourself if you're careful, but if you do that, you'll further kill any possible performance benefit.ASP has a bigger base and RAM consumption than PHP for a reason, which is especially true for ASP.NET - the .NET framework itself. ASP.NET uses objects from the .NET framework, which means that both .NET and ASP.NET's host language need to store a reference to the value in memory and keep it in sync. ".NET" is good in that objects can be used across .NET enabled languages (C#, JScript, VBScript, etc.), and that (AFAIK) it supports threads, meaning that, for example, you can keep a file opened or a database connected across several HTTP requests (up to a certain time limit of course), again for the price of actually keeping the file/connection in memory.

[justsomeguy]

I hope I make sense on that the biggest problem is the programmers database design/latout which plays the largest factor in memory used on the server.

That's probably a bigger issue in XML than it is with a database. In an XML file you need to repeat the field names for each record, that's not necessary in a database table. With a database you can name your tables and fields with long descriptive names, but those names only get stored once by the database, they don't get duplicated for every record like in an XML file. With a database though you have to worry more about data types, and choosing a data type that is appropriate for the data you're storing in that column.

[mstransky]

I agree with you, I read up on articles which explianed it like this....sql's databases are for storing data while xml is for transffering or translating data like a common format. I read both pro's and con's but maybe I am not rewording the point properly.one person wrote that php is a language while asp is a frame work, I myself don't agree 100% with that. But where PHP is concernd it can use a lot less line of code to do the same thing in ASP like connecting to a database. While some basic commands in asp can be just a tad longer in PHP.My Conclusion is PHP is mostlikely much more flexiable then asp or asp.net because asp still rely on MS applications running in the back ground. SO 'boen_robot' you have made me open my eyes a bit more to trying out PHP. Seeing that I have done alot of VB and Java I don't think php would be hard at all.Last questions, You seem more knowledgeable in the SQL. I have made many MS ACCESS databases and stuff and understand that MSSQL runs /connects to a MSACCESS data. q1.now talking just SQL is SQL more like excel books and sheets like access is like tables with keyID'sq2.I can understand that sql does not act like parse to a DOM like point, but when the command to fetch data that equals a certin value, is not sql run each row to match that value in a Col.?I think it does, but I agree that the flat data in sql is say read to run a match per line, while xml had to parse it once (useing memory) to get it at a state like a sql flat file to run a match per row.Thanks.

[mstransky]

That's probably a bigger issue in XML than it is with a database. In an XML file you need to repeat the field names for each record, that's not necessary in a database table. With a database you can name your tables and fields with long descriptive names, but those names only get stored once by the database, they don't get duplicated for every record like in an XML file. With a database though you have to worry more about data types, and choosing a data type that is appropriate for the data you're storing in that column.

I see your point and Boen Robots, I believe I finally grasped the point where memory is lost with an xml file during DOM parse, then both a SQL and the XMLDOM are then ready to be ran down each row for datamatches."In an XML file you need to repeat the field names for each record, that's not necessary in a database table."-justsomeguyThat is why I broke up my xml files into seprate xml files, more like excel sheets which would refrance other tables like keyID's in an MSACCESS table database would. I figure that sped process up, lowered repeating data over and over like how some of the first XML examples came out with child nodes of childs nodes, I wrote them more like a excel table without any funky secondary and third levels inside child nodes. My thought was if you have to have a child, put them on a new table not having to write extensive handlers to pick out a few of many with extesive extra coding on the proccessor.So even though ACCESS uses row keyIDs and columb A B CD in tables, they really dont lose out like XML does actually tagging the data blocks the same way.Thank you for your input aslo!

[justsomeguy]

one person wrote that php is a language while asp is a frame work, I myself don't agree 100% with that. But where PHP is concernd it can use a lot less line of code to do the same thing in ASP like connecting to a database. While some basic commands in asp can be just a tad longer in PHP.My Conclusion is PHP is mostlikely much more flexiable then asp or asp.net because asp still rely on MS applications running in the back ground.

One of the advantages of PHP over ASP is in the community of users. They probably almost have the same amount of people using them, but ASP is used in more corporate environments and PHP is used a lot more among hobbyist users or people trying to set their own stuff up. Because of those differences, there are more people available to help with PHP. Most of the people writing ASP code can't release a lot of their code because it's owned by the company they wrote it for.The other advantage is in the examples. Since ASP isn't actually a language in the classical sense, you can find a lot of examples that don't match what you're working with. With ASP classic, for example, the vast majority of examples online and in books are given using VBScript as the language. It turns out that VBScript as a language is much less efficient and less powerful than Javascript, which you can also use to write ASP pages with. I prefer to write all of my ASP in Javascript, so when I was starting with ASP it was really difficult to find examples written in Javascript, I had to take the VBScript examples and try to convert the VBScript to Javascript before understanding the example, which wasn't always easy. ASP.NET has even more possible languages, such as C#, but thankfully a lot of the ASP.NET examples are given in both VB and C#. With PHP, on the other hand, all of the examples are in the same language: PHP. To make it even more confusing, it's even possible to write an ASP.NET application using PHP as the language.Here's the list of languages you can use to write .NET applications with:http://www.dotnetpowered.com/languages.aspx

now talking just SQL is SQL more like excel books and sheets like access is like tables with keyID's

SQL is only a language, it's the language that you use to interact with a database system. Access and MSSQL each have their own variants of SQL that they understand (MSSQL uses Transact-SQL), MySQL and PostgreSQL each understand a certain variant of the SQL language, etc. It's not really a bad comparison to think of a database like a spreadsheet though. An entire workbook might be considered as a database, and each worksheet would be a database table, with the columns and rows for data.

I can understand that sql does not act like parse to a DOM like point, but when the command to fetch data that equals a certin value, is not sql run each row to match that value in a Col.?I think it does, but I agree that the flat data in sql is say read to run a match per line, while xml had to parse it once (useing memory) to get it at a state like a sql flat file to run a match per row.

That's where a database system has huge advantages over XML. With a database table you can define indexes. An index is sort of like another table that the database system uses for fast lookups. If you define a column as a primary key for a table, and then look up a record from that table based on the primary key, the lookup will be very fast, it won't need to scan all records. I'm not real familiar with the specifics of how different databases implement their indexes, so I don't want to say too much beyond that. But indexing is one of the major factors in database performance in general. I've seen some database queries that might take minutes to finish when they're operating on tables with greater than a million records, but when I've indexed the table to take advantage of the type of queries I was sending, those same queries that used to take minutes now took seconds, just because it was able to use the index to find the data a lot quicker.MySQL specifically has a feature that allows you to specify the storage engine you want to use for each table in the database. The storage engines in MySQL are the layer between the actual data and the database software. The database software sends the query to the storage engine, which does whatever it needs to do for the query. I have an application that was using one storage engine (MyISAM), but it was having a lot of problems when many users were trying to access the site at the same time or look up the same data, and some of the queries were taking a while to finish. I switch several of the tables to a different engine (InnoDB), which stored its indexes and data differently, and handled multiple users better, and the performance of the application jumped way up just because the other storage engine was storing and accessing the data in a more efficient way for my database structure.So, there are a lot of subtleties with database systems, but that's where they have an advantage over XML. The database system itself, which manages indexes, concurrent access, etc, is designed and built specifically for that one job, and it does it well. The MSSQL database system isn't too bad either, but it does have some quirks that would take me a while to figure out and troubleshoot when I was working with it a lot. Access is known to be a low-performance, low-stability database compared to something like MSSQL or MySQL. Access is a good tool to use to learn about how databases work, that's what they had us using in college to illustrate the concepts, but in terms of performance and stability it leaves a lot to be desired. Foxpro, on the other hand, is probably the worst database I've ever had to work with and wouldn't recommend it to anyone for anything. Oracle is known to be a high-performance, high-stability system, but is more expensive than the others.

[mstransky]

Ok thats great advice, I kind of feel like I bit the big one. Kind of like protecting my effort doing what I did. Wow now to go back and trash it all in a way. I thought I was doing pretty good having at times over 800 visitors peak once and to people in the dashboards editing the same xml database.1.) So I now did a great code useing multi xml files, like an oonline catalog2.) I have an affliate like banner database which manages mutli website shares ads. 3.) xml template admin dashboard for website control look.4.) xml user portal manager for a web siteand about 100+ javascripts like tabbed forms, navagation, random ads, 5.) I even had made a adsense like script prior to my XML all in js to run random ads from other sites ad bank of displays.I have alot of neat scripts that would be great if I evr finish a mach type to be display to take from mysite.Well I guess I wont give up 100% onxml and asp but I feel the need to start looking at redoing stuff in php. And everything you metion above that there is more php sources and help out there confirms why more and more asp scripts and code are disappearing from the web. And lately the past year where there was good source of examples and turn into many dead links and directory site only redirect you to purc hase software in the place where the original examples were. I would download many of those examples and have saved many of them to a harddrive is I ever need them to learn from. So if you every ask I might just have one from the past. I was kind of force off track with the last host provider, which I am sure you read that comment above. Many codes that I tried to run on the server would never work so I removed them, tried to alter them, no avail, I pulled them down and it seemed the only thing I was able to run was js and xml stuff so I stuck it out with that. So it really fustrates me that I could have been twenty two steps ahead where his services held me back for quite some time.Both you and boen robot made a web freind today. Thanks!!!