-
Notifications
You must be signed in to change notification settings - Fork 6
Upload_Module
The upload module has been extended to allow the use of a dynamically-generated progress bar for uploads.
This feature requires the "Upload Progress" PECL extension to PHP which is available from http://pecl.php.net/package/uploadprogress.
Installation and Requirements
The PECL extension should be downloaded and compiled as per the standard PECL package instructions. The resultant shared object should be installed into the PHP shared libraries directories. The PECL extension requires that either PHP 5.2 or a patched version of PHP 4/5 is installed. The module requires browser support for javascript xmlhttp fetch to work. You can try a custom version for windows (a compiled dll file is included).
Testing : Success, failure and Untested
This module has been tested successfully upon Linux Debian/Woody.
There are, at present, no known server configurations on which this module fails.
This module has not been tested on the following server configurations:
- Debian Linux pre Woody
- Any other non-Debian derivative Linux distro
- Free BSD
- Windows
- Apple System X
The module has been tested successfully on the following OS/Browser combinations: Windows 2000/Firefox 2.0.0.3 Windows 2000/Internet Explorer 6.0 SP1 Windows 2000/Opera 9.20 Windows 2000/Opera 8 Windows Vista/Firefox 2.0.0.3 Windows Vista/Internet Explorer 7 Windows Vista/Opera 9.20 Windows XP/Firefox 2.0.0.3 Windows XP/Internet Explorer 6/7 Windows XP/Opera 9.20 Linux Ubuntu/Firefox 2.0.0.3 Linux Ubuntu/Opera 9.2 Linux Ubuntu/Konqueror 3.5.6 Apple X/Firefox 2.0.0.3 Apple X/Opera 9.2
The module was tested unsuccessfully on Apple X/Safari
The module was not tested on: Windows 9x (Reason: browsers too old) Netscape / Mozilla (Reason: codebase identical with Firefox) Explorer < v6 (Reason: too old to support xmlhttp javascript call) Netscape <= 4 (Reason: too old, does not support features required) Other browsers : (Reason : Market share too poor)
Method of Operation
The upload bar operates by making multiple calls to a php server, during a normal http upload. The php server is able to extract progress data from the upload via the PECL Upload progress extension, which is then coded as an XML document, and sent to the browser.
The browser, on receiving this document, parses the data out of it. This data is then used to update a DIV segment with text, and also to display the progress bar by dynamic XHTML calls.
The browser will continue to do so until the file finally uploads, and a new page is passed to the browser.
Implementation Considerations
Although the javascript/XHTML makes use of core methods also used by AJAX, AJAX libraries were considered to be too large and cumbersome, due to AJAX consisting of a large javascript source library which has to be downloaded.
The upload must degrade gracefully to the standard html upload system when either the PHP/PECL extension or javascript xmlhttp is unavailable.
Timing of the communication between the server and the browser must be carefully considered so that, on one hand, the user sees a fairly accurate depiction of the upload progress, but on the other hand, communication between the server and multiple upload clients does not cause load balancing problems.
Some switchable debugging information must be available, which avoids use of clumsy javascript alerts.
Detailed Code Description
The Javascript section of the code is contained in papaya-scripts/upload.js
Upon the user clicking on the "submit" button of the upload form, the javascript function "uniqueid()" is called, and then the main function, "start()".
The uniqueid function generates a unique 32-character hexadecimal number that is used to identify the upload. This is deliberately called before the start of every upload to reserialise that upload, and prevent previous upload data from being accidentally reused.
The start function performs a number of initialisation functions to initialise cross-browser support and to make sure that each upload is uniquely identified, using the unique id string generated by uniqueid.
The function starts by setting a fail fallback condition for the xmlhttp variable. The variable is set to null, so that if detection of xmlhttp capabilities fail, the javascript will fail gracefully, and not run during the upload. This means the upload will progress in the standard http upload manner, with no progress indication.
Next, the function attempts to identify the browser as a version of Internet explorer with xmlhttp capabilities. This is done first due to Explorer being the most popular browser on the web, meaning that the most popular option is detected the quickest, and because Explorer xmlhttp and xml support relies on non-standard ActiveXObject calls.
Explorer support for xmlhttp is further complicated by there being various different versions of the activeXobject available on various versions of Explorer (notably version 7). This is tested by going through an array containing the different names of each activeXobject until one is found that works, or we exhaust the possibilities.
If the explorer activeXobject is not detected, then the program attempts to create a standard xmlhttp instance (this is the method supported by all other browsers).
If an xmlhttp instance (either Explorer or standard) has been created, then the program then goes on to set up several other parts of the upload progress. Firstly, it sets the unique id for the upload in the form. It then deactivates the "Upload" button on the form to prevent the user from reclicking it. The URL string used by xmlhttp to retrieve the progress xml data is modified by adding the unique id to it, so that the server can find the correct upload data.
The name of the file is then retrieved from the form, and if the browser has included the full pathname, then the filename is parsed out of the pathname. An initial message, "Uploading, please wait..." is displayed in the progress bar container div, and the div that represents the container bar is set to a width of 0%. (The width of the div is deliberately set in percentage of the total width because this makes it easier for the progress bar to be altered in size without too much work).
Finally, the start function sets a timer which will call the "callevery()" function, and exits. The remainder of the functions in the onsubmit are then processed, submitting the form and then giving the uniqueid another spin.
The "Callevery()" function consists of just a function stub that calls the "loadxmldoc()" function, with the url of the xml document server and a flag representing whether or not the xmlhttp call is asynchronous or not. Callevery exists as an easy way of setting various timeouts.
Loadxmldoc is a function that attempts to set up a connection between the browser and the xml document server. The xml document server is the same server that papaya is running on. The server is merely the other half of the upload progress module that grabs the progress data and reformats it as an xml document. Loadxmldoc starts by checking if we are running in Explorer, and, if so, starting another instance of the correct xmlhttp activeX object. There is a reason for this. The Microsoft activeX objects appear to be one-shot handlers, needing to be reinitialised after usage, whilst the standard xmlhttp object is robust enough to allow repeated usage after only being initialised once.
The function then sets up a link to the "statechange" function in the xmlhttp object. This is the function that will be called whenever a state change is registered in the dialogue between the browser and server. The function then sets up an asynchronous connection between the browser and the server, and performs a send. The function then exits.
Statechange is a function called by the xmlhttp object whenever a state-change is recorded in the dialogue between the browser and the server. It is important to note that not all versions of the xmlhttp object work in the same way. The microsoft version, for example, returns a number between 1 and 4 depending on how far it's got with loading the xml document, whilst the Firefox implementation just returns 4 when the document has been loaded. The logic of the script reflects this, rescheduling a timer event only after a successful document load. This can be seen by the two separate if statements, first checking for a successful load (4), and then checking for a http return code of 200. Once it's checked this, it checks if the debug flag has been set. If it has, then the raw XML document is output using function xmldump().
Next, we have to set up an XML parser so that we can translate the XML into raw data. Again, because Explorer has to do everything differently, we have two possible ways of starting the parser. First of all, the boolean flag "explorer" is checked to see if we're using Microsoft, and then, if so, the relevant activeX object is initialised. Otherwise, we initialise a standard DOMparser object for all the other browsers.
The data is then parsed out of the XML document into percent (percentage complete upload), message (a string containing the status), eta (estimated time to completion of the upload), total (size of the file), upl (amount uploaded as bytes/K/MB) and speed (the estimated speed in bytes/KB/MB per second). This is then formatted into a string which is placed in the DIV containing the progress bar. The progress bar DIV is then set to the percent value. Finally, a timeout is set for calling function callevery().
Function XMLdump is only used if debugging is turned on. When debugging is turned on, then an extra DIV is planted in the XHTML template, which is used to output debugging information. XMLdump takes the text of the XML document and converts any angle-brackets into their HTML equivalent (< and >) so that they won't be interpreted as HTML. The document is then written into the Debug DIV, encased in "
...
" markup to maintain the document formatting. Additional information is also posted into the DIV, such as the version of XMLrpc that is being used.
This is a wrapper function used to make javascript's string-slicing functions look a bit more like php's string replace function. It is used to replace characters with their html mark-up equivalents.
This function supplies a browser-independent method for turning off the submit button in a form.
The PHP module for the upload consists of the file content_upload.php which resides in papaya-lib/modules/free/thumbs.
The file is largely an inheritance of the base_content class, with several extra methods, which are described below.
The server method is a public method which is responsible for outputting the XML for each download. The method starts by setting a default return value of false, and then checks to see if the PECL upload progress module is installed. If not, the server exits and returns the default value. If the module is installed, then the serial number is validated, first by calling the php function escapeshellcmd to remove any suspicious -looking additions, then by calling the method validateID to check that the Id consists of only hex digits. If the ID fails validation, then a dummy XML document is output with the message "Invalid ID".
If the ID is valid, then the PHP pecl extension uploadprogress_get_info is called to retrieve upload information. The upload information usually takes a while to be created, (usually when the server actually starts to receive the file) so the program checks to see if the extension returns false, and outputs a dummy XML document which contains the message "waiting for upload to begin".
If the progress info is valid, it is returned as an array, which is then expanded by calculating sizes in Kb or Mb where possible, and calculating the percentage and estimated time of completion. The array is then converted to an XML document which is then output.
OutputXML takes an associative array as input, and builds an XML document from it. The XML document is output after the correct mime-type headers for XML are output. Cache-control pragmas for the document are also output with the headers so that the document is not cached locally or globally, which would cause problems with the status bar.
Formatbytes takes an integer value and formats it depending on the size. If it is bigger than one megabyte, it is formatted as megabytes. If it is greater than one kilobyte, then it's formatted as kilobytes, and if not, then it's just left as bytes.
The validateId function makes sure that a given Id is valid, by checking to see if it is completely made up of 32 hex digits. The length is checked first, and then a loop extracts each character individually and tests to see if it is present in a string which contains all valid hex digits. This is done this way, rather than using preg due to the possibility of badly-formed strings causing excessive backtracking in the preg interpreter.
The XML Upload Document is generated by the server method of the PHP module. This document contains a snapshot of an upload in progress, and is sent every time the javascript code queries the server.
An example XML upload document follows :
<?xml version='1.0'?>
<status>
<id value='8d750e5ad86cf79b1e332c8d1ee7c076'>
<percent>42</percent>
<eta>00:06</eta>
<speed>449.97Kb</speed>
<upl>2.20Mb</upl>
<total>5.18Mb</total>
<message>Uploading, please wait.</message>
</id>
</status>
The upload document is designed to be as small as possible, to minimize load on the server, but to be extensible. The 'id' tag allows the embedding of several different uploads' details. For example:
<?xml version='1.0'?>
<status>
<id value='8d750e5ad86cf79b1e332c8d1ee7c076'>
<percent>42</percent>
<eta>00:06</eta>
<speed>449.97Kb</speed>
<upl>2.20Mb</upl>
<total>5.18Mb</total>
<message>Uploading, please wait.</message>
</id>
<id value='1ee78d79b1e332c8d750e5ad86cfc076'>
<percent>42</percent>
<eta>00:06</eta>
<speed>449.97Kb</speed>
<upl>2.20Mb</upl>
<total>5.18Mb</total>
<message>Uploading, please wait.</message>
</id>
<id value='d86cf798d750e5abe332c8d1ee17c076'>
<percent>42</percent>
<eta>00:06</eta>
<speed>449.97Kb</speed>
<upl>2.20Mb</upl>
<total>5.18Mb</total>
<message>Uploading, please wait.</message>
</id>
</status>
- Upload form does not appear
Front-end users are not allowed to upload to "Desktop". Ensure that a separate folder has been defined and select that for the upload.
- Uploaded files do not appear in the list
Check that the table papaya_surfer has an index of varchar(32) no autoincrement. If the table has an index of bigint auto-increment, you are working with an outmoded version. The new version requires a 32-digit hexadecimal number for each surfer.
Convert the table to the varchar, then convert existing records to a 32-digit number with the md5 function.
- Internet Explorer 6.1 Gives Error Message complaining about Javascript Submit.
Solution : removed javascript to new function, submission(), and then commented out the submit, as submission is implicit in the action of the form.