Server time out using PHP and perl

[GeraJ]

Here is the problem. I'm trying to upload a file that contains a list of to be processed files, I then check the files using a perl script that runs an Rscript. I'm using those same files for subsequent processing. Everything is working fine on our site, but it gives a time out on someone elses site. The Rscript does not give feedback until it has finished running and starts processing. The results of the Rscript sends information back to the browser and to the php script. But in that time, the connection may already have been broken, giving a TCP time out error, especially for large data sets. I have the following files:index.php: form to upload a list of files and set some settingsparams.php: generates a unique folder, calls a perl (find.pl) script to generate the Rscript, the results from the Rscript are saved in the unique folder and are then used to call the processing script. After the Rscript has returned it's value, the browser will supply the settings page, which is needed to go the results page.find.pl: runs an Rscript to find information I need to keep the connection open so that the results can be send to the browser after the Rscript has finished running. I have used the following which I got from someone else with some modification, but this didn't work as the header('Location:') needs to come before anything else, but I first need to generate a unique userID that can be used to generate the unique folder, so it still keeps hanging. extra line in params.php

header("Location:Load_array_type.php?folder=$userid"); // If I don't use

different file to keep connection open

Please wait! <?php// Add this code into you html page which the message " Pleasse wait while reading array types ! "// This html page is called after the user press submit (providing text file with cel files and starting R script)// These are additions that I had to make$userid=$_GET['folder']; #date['U']  // date['U']: I originally generated the unique userID here, but that meant that it would be regenerated everytime this page loaded, and this would mean that I would never get a message that said the Rscript finished running$project_name = $_POST['project_name'];$readfile=file("../master_files/configuration.conf");$path;for ($k=0; $k<=count($readfile)-1; $k++){  $fields = split("\t",$readfile[$k]);  if ($fields[0]=="PATH"){	$path = $fields[1];	$path = str_replace(PHP_EOL,null,$path);  }}$folder = $path."scratch/$userid/";$inputfolder = $path."scratch/$userid/input/"; // Set variables$statusRefresh = 5000;	 							// set time for Auto-reload in milliseconds$arrayTypes	= "path/filename_arrayTypes";		   // file name, in which R script writes the array types (file must be finished, if you write cel by cel, then you need a small text file, which is saved when finished.) This needs the unique userID generated by params.php$thisPage	  = "Load_array_type.php?folder=$userid";	   // name of the wait page  $nextPage	  = "params.php";					   // relative path and name of page with settings    // Check if file with array types is saved$isFinished = file_exists($arrayTypes); echo "<form name='form1' enctype='multipart/form-data' method='post' action='$thisPage'></form>"; // Auto-reload while runningif ($isFinished) {	   // If finisched go to settings	header("Location: $nextPage");	exit();}else {				  // If not finisched reload page in x seconds (as specified above)	echo "<script language='JavaScript' type='text/javascript'>\n";	echo "var t = setTimeout('document.form1.submit();',$statusRefresh);\n";   // Change 'form1' to your name of the form	echo "</script>\n";}?>

I have also tried the following javascript, but this goes to a blank page, and doesn't open the settings page which I need to get to the results page.

JAVASCRIPT:function keepMeAlive(imgName) {   myImg = document.getElementById(imgName);   if (myImg) myImg.src = myImg.src.replace(/\?.*$/, '?' + Math.random());}window.setInterval("keepMeAlive('keepAliveIMG')", 100000);HTML:echo "<script type="text/javascript" src="javascript.js"></script>";echo "<img id='keepAliveIMG' width='1' height='1' src='../master_files/Picture1.png?' />";

Thanks for your help.

[justsomeguy]

At which point does the timeout happen? Does it happen when one of the other scripts are running? The PHP doesn't look like it's doing all that much, that shouldn't take very long. If PHP is sending the timeout because it's waiting for the other scripts to finish, then you can use set_time_limit to set how many seconds the timeout should be, or disable it. It might also help to show the rest of the code, particularly the code that calls the other scripts.

[GeraJ]

This is the main code:

$flag = 0;$userid = date('U');	  $readfile = file("../master_files/configuration.conf");$path;$server1;$server2;$path_cdf;// some other text removed $folder = $path."scratch/$userid/";$inputfolder = $path."scratch/$userid/input/";if ((!mkdir($folder, 0777)) or (!mkdir($inputfolder, 0777))) {  die("Failed to create folders ... $folder and $inputfolder");}else{  system("chmod 777 $folder");  if ($_POST['project_name']){	$project_name = $_POST['project_name'];	if (preg_match('/^[a-z0-9\-\_]/i',$project_name)){		}else{	  print "<center><b>Project name must start with a letter, number, _ or -</b><br><br><br>";	  bottom();	  exit;	  $flag=1;	}  }else{	echo "<center><b>Please provide project name</b></center><br>";	bottom();	exit;	$flag = 1;  }   ### Cel files ###  $flag_cel = 0;  $cel_tmp = $inputfolder."cel_file.tmp";  $cel = $inputfolder."cel_file.txt";  if ($_FILES["cel_file"]["tmp_name"]){	if (move_uploaded_file($_FILES['cel_file']['tmp_name'],$cel_tmp)){	  system("sed -e 's/$server1/$server2/' < $cel_tmp > $cel");//some other text removed$flag_cel = 1;	  }else{echo "<center><b>Text file is not formatted correctly. Please provide a text file with a list of CEL files in the correct format: only .CEL extension and no extra empty lines.</b></center><br>";bottom();exit;	  }//some code removed  ### Find Array Information ###  $arraytype = $inputfolder."array";  $array = `perl find_array.pl -file='$cel' -output='$arraytype' -path='$path_cdf'`;//This is where the php links to a perl script that generates and runs the R script. It is when the perl script (R) runs that the connection time out happens.//The results are parsed through to the next method. But the results are not shown on the webpage as stated below.  if (empty($array)){	print "<center><b>No array found. Check that the extensions in the CEL file list are all .CEL</b></center><br>";	bottom();	exit;  }else{	print "<center>array is: $array<center><br>"; //results placed on webpage	if (stristr($array, 'error') == TRUE){	  print "<center>".$array."<br><br><br><br></center>";	}else{		  if ($_POST['method']) {$method = $_POST['method'];   ### RMA method ###}if ($method == 'RMA'){   ### Background Correction ###   $bg = $_POST['bg'];     ### Normalisation ###   $norm = $_POST['norm'];   $inv_option2=$quantile_option1=0;   if ($norm == 'choose_norm'){	 echo "<center><b>Please choose a normalisation method from the drop down box</b><br><br><br></center>";	 bottom();	 exit;   }elseif ($norm == 'quantile'){	 $quantile_option1 = $_POST['quantile_option1'];   }else{	 $inv_option2 = $_POST['inv_option2'];   }       ### Summarisation ###   $sum = $_POST['sum'];     $cmd_rma = "-project='$project_name' -celfile=$cel -userid=$userid -array=$array -bg=$bg -norm=$norm -quanoption1=$quantile_option1 -invoption2=$inv_option2 -sum=$sum";   system("perl pp_params_rma.pl $cmd_rma");  

So the R script is running and PHP is indeed waiting for the results of this run. The results are then parsed to a different script with some extra information specific for that script.About the set_time_limit, do I need to set it? Because this is running on a different server I don't have access to, I can't change any of the server settings.Thanks

Edited January 22, 2013 by GeraJ

[justsomeguy]

About the set_time_limit, do I need to set it? Because this is running on a different server I don't have access to, I can't change any of the server settings.

set_time_limit is a function that you use in runtime to set the execution timeout for the script. I would start with that, use it to set the time limit to 0 to disable the time limit and see if it finishes. I'm still not clear on whether this is a PHP timeout or HTTP timeout. http://www.php.net/manual/en/function.set-time-limit.php Also, this script is not secure, you're opening your server up to attack. You're using user-supplied data in shell commands, someone could submit whatever they want for $_POST['sum'], for example, and you would put that in a command to run it. You should use escapeshellcmd on any user-supplied data that goes into a shell command. That goes for $sum, $project_name, and whatever other data you're using that comes from a user submission. http://www.php.net/manual/en/function.escapeshellcmd.php