Tracking Processor Time?

[davej]

I am curious about the idea of tracking processor time used per page update on a shared host. With a typical shared hosting account what sort of times will be considered acceptable by a typical hosting service? I'm guessing they will have a throttling mechanism in place so that one site cannot hog the processing resources. Thanks.

[Guest LH91325]

Processor (CPU) time mostly isn't it. The stuff that takes up the doing slower things that access the server's HDD array and accesses to databases (also on HDD arrays). It's HDD speed (loading pages and loading scripts) and MySQL (or other database) query speed that affects how quickly your pages are served. I've recently posted a method of tracking total page load time and MySQL query times in the forum's PHP section (see PHP debugging techniques), although you'll need to write a fair amount of code if you want good answers. You snapshot microtime() at the beginning of the access, then subtract that from current microtime() after your script has served your page, then log it in a database. If your site consists of static pages then about all you can do (AFAIK) is to write a short test script (PHP or any server side language) that can do approximately the same thing--and put it on a META refresh and point your browser at it. You can throw in some computationally intensive calculations (take up some CPU time), some file reads (take up some HDD time) and some database queries (take up some database server time), and record any or all of the elapsed times in a database. Let it run for a few hours or a few hundred refreshes. Then either download your database or look at it in phpMySQL or other query tool depending on which database you use, or write a script to analyze the values and report them to you. But to answer your question, your shared server is multi-threaded, meaning it is executing everybody's requests all at the same time (by means of time-slicing or other multi-tasking method) and yes there is throttling involved or otherwise one stupid or selfish site owner could tie up the server at the cost of slowing everybody else down. A good sharing service won't let that happen. I'm getting page serve times in the 80-200 millisecond range. That is just for the dynamically generated HTML page itself. I am not monitoring image service times.

Edited May 14, 2012 by LH91325

[davej]

Processor (CPU) time mostly isn't it. The stuff that takes up the doing slower things that access the server's HDD array and accesses to databases (also on HDD arrays).[...]But to answer your question, your shared server is multi-threaded, meaning it is executing everybody's requests all at the same time (by means of time-slicing or other multi-tasking method) and yes there is throttling involved or otherwise one stupid or selfish site owner could tie up the server at the cost of slowing everybody else down. A good sharing service won't let that happen. I'm getting page serve times in the 80-200 millisecond range. That is just for the dynamically generated HTML page itself. I am not monitoring image service times.

Well up until recently the only live pages I had on the web were static HTML pages, but now I'm finally trying to get an ASP.NET page running and I'm having some problems with it, so I'm trying to figure out if it is throwing an exception or getting into an infinite loop, or perhaps is being throttled. How many SQL Server accesses are one too many?

[Guest LH91325]

First of all let me point out that I know absolutely nothing about ASP.NET other than it's Microsoft. Right? Anyway I can help you only in a general way. Are you getting any kind of error messages? Are you getting any response at all? Or are you getting extremely slow responses? If my Linux-Apache-PHP-MySQL throws an exception I expect I'd get an error message. If it hangs I get something like a 500 "internal server error" message. I think it's reasonable for a typical content delivery system to run perhaps a few dozen to a few hundred queries. I'm assuming you're using a shared hosting service and have low traffic. My own low traffic site generates perhaps two dozen queries for a typical site visitor. You'll have to post exactly what you're observing for specific help. The forum has an ASP section and a .NET section, not sure which applies to you. You might get better help posting there.

[boen_robot]

Try to compile, as well as debug and run your ASP.NET app locally. If it's an exception or infinite loop, it should become apparent, especially if you using Visual Studio, where (in the case of exceptions) it will even highlight the line at which the exception was thrown.In the case of throttling though, the server should simply delay your script, not terminate it prematurely, unless perhaps your pure time is longer than a certain timeout (similarly to PHP's max_execution_time ini setting).

[davej]

Things are working better now but I'm not sure what the elapsed time is telling me. I am told the servers are supposed to have something like four dual core processors in them, but judging from the number of visible databases there are plenty of users squeezed onto them. I see elapsed times coming back that are anywhere from less than a millisecond to several seconds.

[Guest So Called]

I'm pleased that you got your programming together enough to get real answers from your analysis. I presume you threw some code at it since you said you know the elapsed time. I'll assume that's from beginning to script execution until you've sent the page to the browser. If you're including MySQL accesses, less than a millisecond sounds too fast to be real, but with no database accesses then probably so. I'm getting execution times of a few tens to a few hundreds of milliseconds, with the odd occasional several or a dozen or more seconds. I have no idea what caused the very few 1+ second accesses. It's interesting to know how many other users are on your shared host server. Here's some links: http://www.yougetsignal.com/tools/web-sites-on-web-server/http://www.myipneighbors.com/ Sharing a server with 80-120 other websites is probably reasonable for shared hosting services. Note that this is just the "front end" server (the one that runs your Apache (http) and PHP. You'll need more code to determine how much load is running on your MySQL "back end" server.

[davej]

It's interesting to know how many other users are on your shared host server. Here's some links: http://www.yougetsig...-on-web-server/http://www.myipneighbors.com/ Sharing a server with 80-120 other websites is probably reasonable for shared hosting services. Note that this is just the "front end" server (the one that runs your Apache (http) and PHP. You'll need more code to determine how much load is running on your MySQL "back end" server.

Hmmm... very interesting. The first link says I am sharing with 299 other websites, although some seem to show up twice -- with and without the www prefix. The second link says I am sharing with 115 other websites. I am now thinking that the elapsed time that I am measuring will include the time my process is suspended pending disk accesses for the database. I think I will have to add a counter to count the number of database accesses for each of those periods.

[Guest So Called]

Just remember the difference between front end server and back end server. The front ends runs your Apache and PHP, the back end runs MySQL. Those sites just count how many URLs point to that IP address. It has no way of telling which sites have traffic. Probably many of them have little or none. I have several URLs on my server but only one has any real traffic. Anyway throw more code at it and you can find out almost anything. Use your imagination.

[davej]

I have now added a preliminary database access counter. Most of the time when idle I am just checking for any new chat messages that might have been posted to the database. If this is my only query it does occur in less than 1 ms.

[Guest So Called]

I'll have to add a feature to monitor my script loading time. My site design is that every page served requires 4 script loads (index.php + 3 others). Maybe HDD accesses are slower than I guessed. But of course I have no way of knowing unless I monitor them.

[davej]

I suppose the next step is to measure the duration of each database access. I have seen occasional error messages where the database timed out or refused my log-in, so something is crazy somewhere, but perhaps it is not entirely my code that is at fault.

[Guest So Called]

I posted how to monitor your MySQL database access times in the PHP area of this forum, look for a topic on debugging techniques. Basically, you funnel all your MySQL accesses through one function, and use microtime() before submitting the query and then after you get the results. I've done that and saved the queries too, then display them in the page footer but only to those logged in as admin (i.e. me!).

[davej]

I'm writing in C# and ASP.NET but I think I have a working scheme using a function which calls DateTime.Now.Millisecond and DateTime.Now.Second. It builds a string of access times which I then download at the end of the page processing. It looks like all of my page time is spent on the database accesses. This morning I see times of as much as 1.5 seconds per access but then the database must cache some accesses because the query that I repeat every two seconds settles down to 0 ms.

[Guest So Called]

My MySQL accesses usually complete in under 1 millisecond. One that accesses an IP-to-country database can go a second or a few seconds on occasion. I'll add script loading timing next time I feel like I want to write a bunch of code.