read-unread

[birbal]

how to do this unread -read post as it is in this forum...ho do they work?(i mean like there is unread post..there is no new post)

[Fmdpa]

Probably using the the a:visited CSS property. I would suspect that the icons for "new posts" involve a database, but I can not think of a way to do this that would not be resource intensive.

[boen_robot]

I think IPB in particular uses a cookie... a cookie that stores the time of the last visit/login, plus cookie(s?) for all topic IDs that have been read since that last visit/login. All topics since the last visit/login time that have not been flagged as "read" are assumed to be "unread".I'd guess other forum systems follow a similar, if not the same, approach.

[jeffman]

I think boen is on track. I visit from work and home, and one computer does not seem to know what I have read on the other computer. I imagine the server reads out that cookie data and prepares the page appropriately. JavaScript could do it, but that would mean a lot of DOM junk that could be handled much more easily at the server level, especially since the server is generating all these pages anyway.

[ShadowMage]

If that were the case, then wouldn't deleting cookies erase my read/unread history? I clear cookies quite often and it doesn't seem to affect which topics are marked as read and which ones are unread.Also, I seem to recall my read/unread topics matching on my home computer and at work, though I can't be sure since I never really gave it a whole lot of thought. But I don't recall ever seeing an unread topic and thinking "Hey I read this already"

[jeffman]

Yeah, I'm not exactly trusting my memory on this.

[ShadowMage]

Do different browsers use different cookies? Or can each browser access the same cookie?

[jeffman]

Different browsers use different cookies, yes. There is no central cookie repository on your hard drive.

[thescientist]

If that were the case, then wouldn't deleting cookies erase my read/unread history? I clear cookies quite often and it doesn't seem to affect which topics are marked as read and which ones are unread.Also, I seem to recall my read/unread topics matching on my home computer and at work, though I can't be sure since I never really gave it a whole lot of thought. But I don't recall ever seeing an unread topic and thinking "Hey I read this already"

I have this same experience, mostly with the clearing of cookies, as I work on the same computer that I bring home.

[birbal]

If that were the case, then wouldn't deleting cookies erase my read/unread history? I clear cookies quite often and it doesn't seem to affect which topics are marked as read and which ones are unread.Also, I seem to recall my read/unread topics matching on my home computer and at work, though I can't be sure since I never really gave it a whole lot of thought. But I don't recall ever seeing an unread topic and thinking "Hey I read this already"

yeah i was suppose to tell that. it recall after cookie deletion. at first i thought that its using cookie but after it i change my mind...is not it mean that it using database?..but as fmpda said that it will take long data flow in db. though i saw some cookie as forumread and topicread in cookies. not sure what are they for where they seems no use after deletion cookieshere is thousands of topics. to flag them in cookie or db sounds lot of work.

[jeffman]

A lot of work for the computer -- maybe. But the code will be easy to write, once you have decided on a table structure.And I'm not so sure the DB will work hard anyway. First it will generate a list of recent topics to display. There really are not many, and all PHP needs at first is a thread-number, which is a short field to return. Then your own table of read topics will be opened, and a loop or two will determine which get certain flags.I'm not an SQL genius, but there may be a way to JOIN the query and do it all at once with no looping.If the DB were forced to generate the list of every topic ever posted, yes it would take a long time. But it does not. And we are only pulling data for the contents pages, not the actual posts. So again, not so much data to sift.

[ShadowMage]

Different browsers use different cookies, yes. There is no central cookie repository on your hard drive.

That's what I thought. And that confirms that cookies are not being used because if I view this forum with Chrome, read a topic in FF, then refresh in Chrome, the topic becomes read.

And I'm not so sure the DB will work hard anyway. First it will generate a list of recent topics to display. There really are not many, and all PHP needs at first is a thread-number, which is a short field to return. Then your own table of read topics will be opened, and a loop or two will determine which get certain flags.I'm not an SQL genius, but there may be a way to JOIN the query and do it all at once with no looping.If the DB were forced to generate the list of every topic ever posted, yes it would take a long time. But it does not. And we are only pulling data for the contents pages, not the actual posts. So again, not so much data to sift.

Precisely. It would only have to sift through the 10/20/30 (or however many topics are on a page) topics listed for the current page and set the appropriate flags.How are you thinking on storing which topics are read by whom, DD? I can think of two ways. First store a delimited string of topic id's in the user's record, and second store a list of user id's in the topic's record.The first is probably not a good solution since that could be very, very long (considering ppl like jsg, DD, boen, and synook who have thousands of their own posts, I don't even want to know how many they've read )Using the second method you can set the read/unread flag by checking if the user's id is in the list of id's for each displayed topic.

[justsomeguy]

It uses both, it stores that information in the database and also sets a cookie with it. If you have the cookie, it uses that instead of needing to calculate it again from the database. If you don't have the cookie or it's out of date, it figures out what the data should be and sets a cookie.

[Fmdpa]

It uses both, it stores that information in the database and also sets a cookie with it.

Doesn't that mean there would be a table of the topics for each user, with values representing whether they read topics? I still can't think of an efficient way to do it.

[justsomeguy]

Basically yeah, you need to store that data somehow. I'm not sure how this forum does it specifically, but you could download a forum like phpbb and see how they do it. One way to do it may be to do it in levels, where you have your table that stores something like the user ID, forum or thread ID, and then a comma-separated string or something to list the IDs of individual posts. So when a user is viewing a particular forum then you could get the data for just that user and forum, and figure out what they've read. Obviously that's going to get to be a pretty big piece of data, so there may be another way which would scale better to larger data sets. A pair of lookup tables would work, but that's going to reach a large number of rows in a short time. Table indexes would help speed up the lookups though.

[jeffman]

If you're a serious business like Amazon or something, use a wicked-fast server array with redundant storage. If you're joe blow, don't worry about it. Efficiency is relative. <insert Einstein icon>Write yourself a plain text file with 100,000 lines. Each line is CSV data representing a user, with password, email, all the usual crap.Now open the thing using file() so you have an array of 100,000 elements. Loop through it to find the one entry that matches a certain password and username.Make it tough. Make sure the one you're searching for is the last one.You'll find it so fast your head will spin around. Reading a hard drive (especially an optimized hard drive) is pretty darned fast, and that's all a database request really does. (Writing data is a slightly different matter.)Except for EXTREMELY processor-intensive activities, involving HUGE chunks of data, or image/audio processing or something like that, or when you're getting pounded by thousands of requests per minute, scripts run pretty fast.By comparison, the HTTP process is usually the slowest piece of the puzzle. Bandwidth, multiple requests, hubs, routers, dns, etc. -- that stuff is SLOW. Lerdorf himself shrugs his shoulders at things like PHP compilers for web use, because the microseconds saved in code execution are silly compared to the time it takes to transfer data from host to client.If it ever did become a problem, you'd track your logs and figure out which table structures made the thing run fastest.

[justsomeguy]

That's generally true for the small stuff. I may not be Amazon, but even for a dedicated server hosting 10-20 installations of an application where each installation has hundreds to hundreds of thousands of users, you certainly have a chance to find out exactly how well-designed your database really is. I've seen new programmers come in and add a feature which drops performance by orders of magnitude (remember kids: a SQL join scales much more efficiently than a query inside of a loop). Hardware is definitely getting much faster, but when you've got tens or hundreds of thousands of people hitting the server at 20-30 requests per second, benchmarking and database analysis will go a long way to finding your problems. Another good lesson: keep in mind that the default database engine in MySQL, MyISAM, uses whole-table locking. That means that while one person is doing an operation that reads or writes to a table, even if it's only one row, MyISAM locks the entire table keeping other requests waiting until the first one finishes. InnoDB does row-level locking instead, so two process are only restricted from reading or writing to the same row instead of the entire table. I learned that the hard way, because I didn't do my research. This is a problem where one process is reading many rows for something like a report, and while it's doing that all of the processes trying to update or add individual rows get locked out, even if they don't conflict.Also, here's another good tip: if you're making a script like a cron job or other command-line script, keep in mind that PHP's normal timeout limit does not apply. Even if you schedule a script to run once every minute, and set the timeout in that script to one minute, that doesn't mean the script will stop after a minute. If you're doing something like, for example, processing emails to 80,000 people, and you have your script running once a minute and each is set to only send 1000 emails before quitting after a minute (or so you think), it really will send all 1000 emails before quitting. So, for example, if another process of the same script starts after a minute and starts sending the first 1000 emails in the queue, but the previous script which started a minute prior has only managed to send 100 emails, then those last 900 are going to be sent twice. Then a minute later another process starts, and it doesn't time out either, believe me. Pretty soon people from work start calling you and wondering why the clients are receiving a hundred of the same emails. One possible fix to that, by the way, is to use something like a text file to store a value indicating whether or not the script is currently running, and if one starts while another is running then it just quits.Also, it sucks when a client demands a change which changes the database structure, thereby dropping performance, and then they blame you.

[Fmdpa]

Any option that includes CSVs couldn't be right- they seem so disorganized.

[jeffman]

If you're referring to my post, I wasn't suggesting a CSV solution. I was suggesting an easy test you can run yourself to simulate the speed with which your hardware runs. Then, keeping in mind that your DB is optimized (indexing, b-trees, etc) in a way that a simple PHP loop is not optimized, you can get an idea how fast an "ordinary" project can be executed.(I chose CSV as a test, because you can copy/paste a massive CSV file into existence in a minute or so. Unless you already have a whopping big database, or you want to go out and import one, a true simulation would require more time to set up.)