Jump to content

Database


Recommended Posts

Mongodb is a JSON database, but you will find heated arguments regarding its performance claims because it simply works differently, with different primary goals in mind. A conventional database seeks, first and foremost, to provide ACID transactions with extreme reliability and very good error recovery, with a focus on the manipulation of normalized tables. Mongodb seeks to provide an efficient way to quickly store and search JSON documents. Mongodb doesn't even provide a native JOIN. If you need to use joins you will have to write each JOIN yourself in Javascript.

I can use couchdb than.
Link to post
Share on other sites
  • Replies 94
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Let me clarify one other thing: I am not, and I don't think I have represented myself as, an expert in databases. I am not trying to assert that SQL is the only option you have. I am responding instea

OK, let's look at the wall of text you posted and then summarized yourself in a single sentence. First, that is a big "if". Many applications have nothing to do with consuming or producing XML data.

Another 32 bits, obviously.It's the maximum instruction size that the processor can support. 64-bit processors have been mainstream for only a few years now, so there are still a lot of 32-bit applica

SQL first appeared in 1974. It is old, and very mature. PHP first appeared in 1995. PHP is still maturing. SQL is not a part of PHP, it is a distinct language. PHP does not understand SQL. Databases understand SQL. You can use languages like PHP, C#, C++, C, Java, VB, Perl, Python, Ruby etc to connect to a database and then send commands to the database. The commands are sent as SQL code, because the database understands SQL. The database does not understand PHP code any more than PHP understands SQL code. They are completely separate languages for completely different purposes. That statement is meaningless considering the fact that you don't understand what SQL is, let alone what it can do.Here's a question to ask yourself: if you think XQuery and XML are so much better than relational databases and SQL, then can you point to a single major website which uses XQuery/XML instead of a real database and SQL? I'll save you some research time - you can't. Consider a site that has 1 million users in its database. Assume that this site sees only 10 requests per second, so the traffic is not very high. Assume that each user record in the database averages 1KB of data. Since the entire XML structure needs to be read into memory in order to be used (unlike a real database), for that small traffic load of people just logging in, the server would need to use (1,000,000 * 1024 * 10) = 10,240,000,000 bytes ~ 10GB of memory just to handle 10 people trying to log in at the same time. Our dedicated servers each have 24GB of RAM, and it is not all dedicated to the database. That means that, with XML, our servers would only be able to handle 24 people using the system at the same time, assuming that every last byte of RAM was going to the database. Think about how many people use Facebook at the same time. Does it make any sense to use a data store technology that requires the entire database in memory every time any person accesses the system? When someone logs in to Facebook to get their list of messages, it would need to load the entire list of messages that every person has ever sent just to get that one person's messages? Does that really sound like a good plan to you? You have no idea about all of the optimizations that relational databases employ to make data storage and access as fast as possible, technologies that have been used for over 40 years, so what exactly qualifies you to make the judgement that XQuery and XML are so much better?By the way, the XML standard was introduced in 1996. PHP, let alone SQL, is older than XML.

I ask for help from some other wizard to reply to this post buddy. So just you wait.In the mean time: so what about if I store my user data in JSON? And my webpage via XML to give me manipulation on how they are displayed.And.., when you use
document.getElementsByTagName("p");
does this mean that the hold document is loaded first, and then the paragraph tag is extracted, or are we just using the already defined and load document in memory? --either way, it seem it'll have to be in memory as you said, but I'm just not all the clear on it-- If you'll be so kind as to proved clarity on this. Edited by L8V2L
Link to post
Share on other sites

The document is loaded into memory as soon as you open the page and stays there until you leave the page or close the window.

 

JSON has the same drawbacks as XML does. It has to be parsed and loaded into memory before data can be accessed.

Link to post
Share on other sites

The document is loaded into memory as soon as you open the page and stays there until you leave the page or close the window.

So as soon as I open the page, the document is loaded with a variable referencing to it call
document
.Sax does not load the hold page in particular into memory before parsing it. Why didn't they go that rout?To bad sax is an api met for java.So yeah, since the site itself is under my control. I feel it would be safe to store it's information into XML. That way I can change the theme and display of it via xslt, instead having per defined webpage to load.I would rather store my data in JSON format, than relational --tubular-- format.I don't really know much about SQL, and I'm not driving to learn it.I found XML and JSON to be my choice for database store age format.If push comes to shave, and I'm getting more user than JSON can handle, than I'll look into SQL--but I doubt that, I believe JSON can handle the same intensety as tabulare database can--.
Link to post
Share on other sites

SAX, and any XML parser, loads the whole document into memory. You can't get around that.

 

JSON is just as bad as XML when it comes to performing a database role. It also has to be parsed and loaded into memory before being used.

 

You can do whatever you like with your own website, I'm not even sure how you intend your website to work, though, because you still don't know any server-side languages. You can't store your website's database on the client.

Link to post
Share on other sites

I still don't understand what the goal is. You want to avoid using an ordinary SQL database with normalized tables because you like XML so much?

... Yes.---that's when davej realize he was conversing with an idiotic. He set back in his chair, rub his eyes, and look toward the moon through his window. He spoke softly; "Alibell, when can I see you again my dear".---As I said, I wish to use XML database to store my XHTML pages information to Beale to manipulate them. And JSON to store my user information. That's my game plan. Edited by L8V2L
Link to post
Share on other sites

SQL first appeared in 1974. It is old, and very mature. PHP first appeared in 1995. PHP is still maturing. SQL is not a part of PHP, it is a distinct language. PHP does not understand SQL. Databases understand SQL. You can use languages like PHP, C#, C++, C, Java, VB, Perl, Python, Ruby etc to connect to a database and then send commands to the database. The commands are sent as SQL code, because the database understands SQL. The database does not understand PHP code any more than PHP understands SQL code. They are completely separate languages for completely different purposes. And bla bla BLA!

READ/EAT THIS!I have been asked many times: What is faster, XML or Relational?. Of course, this question oversimplifies a complex issue, and so the only valid answer is It depends!. Sometimes people ask the same question in a slightly different way: If I have a relational table and convert each row into a small XML document and store these documents in a separate table with 1 XML column, whats the performance difference between the two tables (for inserts/updates/queries)?. But, in most cases such a conversion is not recommended and this type of comparison is, again, too simplistic.Lets say you want store, index, and query 1 million addresses and each address has a first name, last name, street, city, state, and zip code. Thats a simple and fixed structure and its the same for all records. Its a perfect fit for a relational database and can be stored in a single table. Relational databases have been optimized for decades to handle such fixed records very efficiently. However, if the application needs to convert the address data to XML format anyway, it can often be faster to store the data permanently in XML and avoid the repeated conversion from relational format to XML format in the application.Now consider a scenario where the business objects of interest are a lot more complex and variable than simple addresses. For example, derivative trades in the financial industry are modeled in XML with an XML Schema called FpML (financial products markup language). It defines more than 7000 fields (many are optional) with hundreds of 1-to-many relationships between them. Designing a relational schema to represent such objects is very hard and leads to hundreds of tables. The process of inserting (normalizing) and reading (denormalizing) a single object into such a relational schema can easily be 10x or 100x slower than inserting and reading a corresponding XML document in a native XML column (e.g. in DB2 pureXML).So, any performance comparison of XML versus relational depends heavily on which data you choose for the comparison, and what type of operations you measure.A large DB2 customer recently compared XML to relational performance because their business objects are currently mapped to 12 relational tables. Their application executes at least 12 SQL statements to retrieve all the relational data that comprises one of the logical business objects. Then the application reassembles the original business object. An alternative is to store these business objects as XML so that each object is stored as a single document. Instead of 12 tables, only 1 table with 1 XML column is then needed. In multi-user tests for data retrieval the the company found that the XML-based solution allows them to retrieve objects with 55% higher throughput than the existing relational SQL-based solution. The reasons for the performance benefit include fewer JDBC interactions between application and database as well as fewer distinct pages that need to be read when one logical business object is represented as one XML document (and not scattered over 12 tables). These tests were later repeated and verified at an IBM lab.Another important considertion in the question of XML versus relational is the data format in which the data is produced and consumed outside the database. If the data is produced and/or consumed in XML format anyway, it is often better to also store the data as XML in the database.So, the question What is faster, XML or Relational? is somewhat like asking What is faster, a truck or a ship?, because XML and relational are meant for different purposes, and either one can outperform the other depending on which use case you look at. And there are also use cases (with high schema complexity and schema variability over time) that cannot reasonably be implemented in a relational data model. (If you need to go to Hawaii, the boat always beats the truck!).The beauty of a hybrid database system such as DB2 is that you can use both native XML and relational capabilitues side by side, in a tightly integrated manner. Some data is better representated in relational tables, other data is better represented in XML, and you can manage both in the same database or even the same table, and with the same APIs and utilities.
Link to post
Share on other sites

Their database, IBM DB2, does not store data in XML files, it is simply optimized to store and extract data in XML format and this database actually uses SQL to access the XML data, as in this example: http://en.wikipedia.org/wiki/XML_database#Example_of_XML_Type_Query_in_IBM_DB2_SQL

 

And a quote from the article on XML databases:

The term "native XML database" (NXD) can lead to confusion. Many NXDs do not function as standalone databases at all, and do not really store the native (text) form.

 

Link to post
Share on other sites

OK, let's look at the wall of text you posted and then summarized yourself in a single sentence.

However, if the application needs to convert the address data to XML format anyway, it can often be faster to store the data permanently in XML and avoid the repeated conversion from relational format to XML format in the application.

First, that is a big "if". Many applications have nothing to do with consuming or producing XML data. Second, notice the terms they use, things like "can often". Saying that storing data as XML "can often" be faster than in a relational schema is not the same thing as saying that storing data as XML "is" faster than in a relational schema.

Now consider a scenario where the business objects of interest are a lot more complex and variable than simple addresses. For example, derivative trades in the financial industry are modeled in XML with an XML Schema called FpML (financial products markup language). It defines more than 7000 fields (many are optional) with hundreds of 1-to-many relationships between them.

This is a fantastic use case for a database that can natively handle XML-based records. Is that what your game is going to be? Are you going to be dealing with individual data structures composed of thousands of fields with hundreds of relationships? Do you want to base the technology of your game off of the same technology used by the financial sector for no reason other than that you like XML and don't want to have to learn anything else?

So, any performance comparison of XML versus relational depends heavily on which data you choose for the comparison, and what type of operations you measure.

This is a true statement, and it references the statement he made about how optimized modern relational databases are for the workloads that they are designed for. The MyISAM storage engine, for example, is fast when it is in a workload of doing a tremendous amount of select statements, with relatively fewer inserts, updates, and deletes. The InnoDB engine, by contrast, places its focus on data integrity instead of performance, since it supports things like transactions and foreign keys. Each storage engine is optimized for specific use cases. It is ridiculous to try and make blanket statements that any one technology is always superior to any other technology. The question you need to ask is what kind of technology is best suited for a game. Right now the answer that you have is not based on requirements and needs, it is based on the fact that you want to use the first thing you learned instead of learning something new. That is not a good way to make a decision.

A large DB2 customer recently compared XML to relational performance because their business objects are currently mapped to 12 relational tables. Their application executes at least 12 SQL statements to retrieve all the relational data that comprises one of the logical business objects. Then the application reassembles the original business object. An alternative is to store these business objects as XML so that each object is stored as a single document. Instead of 12 tables, only 1 table with 1 XML column is then needed. In multi-user tests for data retrieval the the company found that the XML-based solution allows them to retrieve objects with 55% higher throughput than the existing relational SQL-based solution. The reasons for the performance benefit include fewer JDBC interactions between application and database as well as fewer distinct pages that need to be read when one logical business object is represented as one XML document (and not scattered over 12 tables). These tests were later repeated and verified at an IBM lab.

This is pointless. Not only is this story completely anecdotal and without any evidence or citations, but we also don't know anything about the relational schema. It's easy to design a horrible relational schema, maybe their original schema could have been optimized by someone who knows what they're doing with relational databases in order to achieve the same or better performance gains that they saw with XML. I know from my own experiences that the difference between a query taking 100 seconds and less than 1 second can be as simple as adding a few indexes, exchanging subqueries for joins, or replacing IN with EXISTS. A slow query doesn't mean you need to throw out all your tried-and-tested technology, it means you need to figure out where the bottlenecks are and address them.

The beauty of a hybrid database system such as DB2 is that you can use both native XML and relational capabilitues side by side, in a tightly integrated manner. Some data is better representated in relational tables, other data is better represented in XML, and you can manage both in the same database or even the same table, and with the same APIs and utilities.

And, as Ingolme already pointed out, he is not referring to a SQL-free database, his database still uses SQL. It simply has native support for XML-based columns. That is not the same technology that you are fixated on using. That database, for example, does not need to read an entire XML database into memory to find individual results.But, none of this really matters. Here's what matters: you have looked into Javascript and XML, you think that neither of them can do any wrong, you want to build your application on them, and you're not going to be swayed by anything we say. So, what's stopping you? Go out and do it, design your application the way you want to do it, use all of the technologies that you've learned about, and make it happen. I'll guarantee one thing right now, and it doesn't even matter which set of technologies you ultimately decide to use: by the time you're finished with your first application, you'll want to dump the whole thing and rewrite it from scratch. Count on it.
  • Like 2
Link to post
Share on other sites

But, none of this really matters. Here's what matters: you have looked into Javascript and XML, you think that neither of them can do any wrong, you want to build your application on them, and you're not going to be swayed by anything we say. So, what's stopping you? Go out and do it, design your application the way you want to do it, use all of the technologies that you've learned about, and make it happen. I'll guarantee one thing right now, and it doesn't even matter which set of technologies you ultimately decide to use: by the time you're finished with your first application, you'll want to dump the whole thing and rewrite it from scratch. Count on it.

:-( .... Why is there synch a large community for XML?I wish to be apart of that community, and be able to say I'm an XML programmer.I'm only using XML as a storage for my site information.I'm storing my user information in JSON. I can organize my JSON file that would better preform in action. Instead of having a large file of my user long in, i could break up that file. I'm not storing much information. Just things they'll like to maybe keep to look back on.If this becomes to much of a hassle, I'll see toward relational database. Or maybe a hybrid.I'll like to work with JSON and XML as my data storage for the use case of how I'll be using them.Plus I feel this will help me better myself toward html via XHTML.Oracles is one of the main database provider, right?And 28minsec --I believe that how it's spell-- and marklogic too, right?
Link to post
Share on other sites

 

Here the respond from what you wrote me that I post to the wizard of xslt:This is off topic for the XSL List.And the answer is that the person who wrote this has obviously never heard of XML databases. In fact he knows very little about databases at all if he thinks the only real database is a SQL database.Michael KaySaxonica

So I'm on a forum being school by some wizard. I don't know how to respond to this:XQuery to my knowledge exceed SQL very!

It has different strengths.There actually _are_ large Web sites using XQuery in the backend, sometimes in conjunction with relational databases. There's no point in naming them to someone who has already decided the answers based on false premises, nothing could be better than his/her favourite technology!It is worth noting that XQuery/XML-native databases do not read XML in for each query any more than relational databases read CSV files off disk for each SQL query.Note also that IBM DB2, Oracle and Microsoft SQL Server all have XQuery support (to varying extents and in varying ways).

By the way, the XML standard was introduced in 1996. PHP, let alone SQL, is older than XML.

The XQuery specification was edited by the co-inventor of SQL, and XML is based on SGML, which has a heritage going back to the 1960s. So?Liam----End----What you have to say to THAT! justsomeguy.On another note:quote name="justsomeguy" post="282463" timestamp="1408386193"timestamp! Unix!!! Edited by L8V2L
Link to post
Share on other sites

Oh, goodness. We all know that there are more databases than SQL databases. It just happens to be that the XML database that you pointed us to is one that uses SQL, which is what you wanted to avoid.

 

It would be nice to talk with Michael Kay directly to clear this up because you didn't do a very good job as a middleman. Actually, all the people on that forum you got these quotes from. Please link them to this thread.

 

And this quote you just provided proves what we've been trying to tell you:

It is worth noting that XQuery/XML-native databases do not read XML in for each query any more than relational databases read CSV files off disk for each SQL query.

XML Databases do not use XML files and they have complex engines just like the SQL databases do. You have told us you intended to use XML files on your website to store your data but that's not an XML Database.

 

There's no point in naming them to someone who has already decided the answers based on false premises, nothing could be better than his/her favourite technology!

This sounds a whole lot like you, you like XML so much you don't care to investigate relational databases. I'll repeat that the XML you are using is not an XML Database, just plain XML files.

Link to post
Share on other sites

Oh, goodness. We all know that there are more databases than SQL databases. It just happens to be that the XML database that you pointed us to is one that uses SQL, which is what you wanted to avoid. It would be nice to talk with Michael Kay directly to clear this up because you didn't do a very good job as a middleman. Actually, all the people on that forum you got these quotes from. Please link them to this thread. And this quote you just provided proves what we've been trying to tell you:XML Databases do not use XML files and they have complex engines just like the SQL databases do. You have told us you intended to use XML files on your website to store your data but that's not an XML Database. This sounds a whole lot like you, you like XML so much you don't care to investigate relational databases. I'll repeat that the XML you are using is not an XML Database, just plain XML files.

None of them is me... But I'll send the link out like you said... I was hoping to see what justsomeguy would say.
Link to post
Share on other sites

Another respond to justsomeguy post I copy and paste:The answer to end all arguments.All database technologies (including your favourite one, including NoSQL), entail a series of compromises which make certain things easy and certain things hard.

Link to post
Share on other sites

I'm only using XML as a storage for my site information.I'm storing my user information in JSON.

Great. Go do it. You'll learn about which things work well and which don't from experience. Just telling you doesn't seem to help. If you want to build your site around an XML data store, then stop talking about it and start doing it.

And the answer is that the person who wrote this has obviously never heard of XML databases. In fact he knows very little about databases at all if he thinks the only real database is a SQL database.

I'm not going to have a proxy discussion with someone that I don't know through you. The solution that you have come up with is not to use a native XML database, you were talking about storing everything in XML files, apparently by storing JSON strings in XML files (which sort of defeats the purpose of using XML). If you want to use an XML database, fine, go download eXist-db and start here:http://exist-db.org/exist/apps/doc/development-starter.xml
  • Like 1
Link to post
Share on other sites

Let me clarify one other thing: I am not, and I don't think I have represented myself as, an expert in databases. I am not trying to assert that SQL is the only option you have. I am responding instead to your "journey" where you have scratched the surface of a few technologies and decided to try and come up with an entire application architecture when you have never even written an application. I am responding to your claim that you want to use XML for the reason that you think SQL is too hard and don't want to learn anything else. I'm responding to things like you not wanting to play with Linux because you think the mascot is intimidating. I'm responding to you coming in and asking "what is a database" after you've already made up your mind about which technologies you want to use as your data store.XML and XQuery are fine technologies if they are used correctly, but the reasons for your decisions are not based on the pros and cons of the various technologies, they seem to come from a desire not to learn anything else. I'm not even trying to assert that SQL is the best tool for a database job. The fact is that SQL is very widespread, it is mature, it is used relatively successfully by a lot of beginners (although there is a ton to learn beyond the basics), and I don't think that you have given a compelling reason why you don't want to use it. If you want to focus on XML and XQuery, that's fine, go do it. But I would suggest saving your application architecture design decisions until you actually have real experience using the technologies that you want to use. Right now your experience is zero, I have yet to see you post any code that you have written yourself beyond basic exercises. When you say something like you are storing your user information in JSON, I don't think you have any clue what that means. I also don't see any reason to separate something like the site data from the user data, why use different data stores for those? Aren't they all part of the same application? Why not use a single data store for the entire application?Hopefully you understand where my argument is coming from. It's not the technologies that you have chosen that I am attacking, I am attacking the reasons why you've chosen them. The reasons are definitely not based on any sort of experience, which they should be. The point I am trying to make is that you need to get experience actually using things rather than just talking about what you are going to do. You don't even need to design your game first, write small applications that actually work but which use the various technologies that you are trying to learn, that's how you get experience.

  • Like 3
Link to post
Share on other sites

Great. Go do it. You'll learn about which things work well and which don't from experience. Just telling you doesn't seem to help. If you want to build your site around an XML data store, then stop talking about it and start doing it.I'm not going to have a proxy discussion with someone that I don't know through you. The solution that you have come up with is not to use a native XML database, you were talking about storing everything in XML files, apparently by storing JSON strings in XML files (which sort of defeats the purpose of using XML). If you want to use an XML database, fine, go download eXist-db and start here:http://exist-db.org/exist/apps/doc/development-starter.xml

Thanks for the link, and being a good sport --did I use that in the right context--truth is I'm learning everything that comes my way --or at least try-- sometime I'll like to talk to others on what I'm learning, but have nothing to really talk about here, cause non of you are familier with node programming.Learning node proved to be not easy for someone who have no experience with server side programming concept, and there is no learning material that introduce node to you in a linear fashion --except for this one book I found... But it might change through out the chapters-- . But learning programming is never a linear expericence. The JSDG, MDN, and W3S does a good to great job at this. But MDN only speaks on it, there is no NDG --yet-- and W3S doesn't seem to want to do a tutorial on it. Edited by L8V2L
Link to post
Share on other sites

What's a process: In computing, a process is an instance of a computer program that is being executed. It contains the program code and its current activity. Depending on the operating system (OS), a process may be made up of multiple threads of execution that execute instructions concurrently.What's a thread:In computer science, a thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler (typically as part of an operating system). The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is a component of a process. Multiple threads can exist within the same process and share resources such as memory, while different processes do not share these resources. In particular, the threads of a process share the latter's instructions (its code) and its context (the values that its variables reference at any given moment).

Link to post
Share on other sites

You didn't ask a question. All you did was provide two definitions of computer terminology.

Ingolme, I'll put you in a bottle like a firefly... -- what horror is such a season to promote kids to chase after and catch little buys, and incapsulate them in jar -some with out holes for air-. And to leave them in there to suffer, starve, and die. What horror--... I did answer**ask** a question. Above the definitions.Those are just the definition. I'm asking for a use case, i.e. In Unix or Linux, or Solaris case. In the terminal. Edited by L8V2L
Link to post
Share on other sites

A use case for a process? Well, your browser is a process that lets you view this web page.

 

In programming you use threads so that your process can have multiple things being done independently from each other at the same time. Javascript doesn't have threads because it's a very high level language.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...