Skip to content

Commit dc89f17

Browse files
fdintinofdintino
committed
README: add upload protocol, toc
1 parent 3126898 commit dc89f17

File tree

2 files changed

+191
-1
lines changed

2 files changed

+191
-1
lines changed

‎README.md

+24-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,30 @@
22

33
A module for [nginx](https://www.nginx.com/) for handling file uploads using
44
multipart/form-data encoding ([RFC 1867](http://www.ietf.org/rfc/rfc1867.txt))
5-
and resumable uploads according to [this](#upload-protocol) protocol
5+
and resumable uploads according to
6+
[this](https://github.com/fdintino/nginx-upload-module/upload-protocol.md)
7+
protocol.
8+
9+
* [Description](#description)
10+
* [Directives](#directives)
11+
* [upload_pass](#upload_pass)
12+
* [upload_resumable](#upload_resumable)
13+
* [upload_store](#upload_store)
14+
* [upload_state_store](#upload_state_store)
15+
* [upload_store_access](#upload_store_access)
16+
* [upload_set_form_field](#upload_set_form_field)
17+
* [upload_aggregate_form_field](#upload_aggregate_form_field)
18+
* [upload_pass_form_field](#upload_pass_form_field)
19+
* [upload_cleanup](#upload_cleanup)
20+
* [upload_buffer_size](#upload_buffer_size)
21+
* [upload_max_part_header_len](#upload_max_part_header_len)
22+
* [upload_max_file_size](#upload_max_file_size)
23+
* [upload_limit_rate](#upload_limit_rate)
24+
* [upload_max_output_body_len](#upload_max_output_body_len)
25+
* [upload_tame_arrays](#upload_tame_arrays)
26+
* [upload_pass_args](#upload_pass_args)
27+
* [Example configuration](#example-configuration)
28+
* [License](#license)
629

730
## Description
831

‎upload-protocol.md

+167
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Resumable uploads over HTTP. Protocol specification
2+
3+
Valery Kholodkov [\<[email protected]\>](mailto:[email protected]),
4+
2010
5+
6+
## 1. Introduction
7+
8+
This document describes application protocol that is used by [nginx
9+
upload module](upload.ru.html) to implement resumable file uploads. The
10+
first version of the module that supports this protocol is 2.2.0.
11+
12+
<span id="2"></span>
13+
14+
## 2. Purpose
15+
16+
The HTTP implements file uploads according to
17+
[RFC 1867](http://www.ietf.org/rfc/rfc1867.txt). When the request length
18+
is excessively large, the probability that connection will be
19+
interrupted is high. HTTP does not foresee a resumption mechanism. The
20+
goal of the protocol being described is to implement a mechanism of
21+
resumption of interrupted file transfer or suspension of upload upon
22+
user request.
23+
24+
<span id="2.1"></span>
25+
26+
## 2.1. Splitting file into segments
27+
28+
When TCP-connection interrupts abnormaly there is no way to determine
29+
what part of data stream has been succesfully delivered and what hasn't
30+
been delivered. Therefore a client cannot determine what position to
31+
resume from without communicating to server. In order to eliminate
32+
additional communication file is represented as an array of segments of
33+
reasonable length. When TCP-connection interrupts while transmitting
34+
certain segment, client retransmits the whole segment until a positive
35+
reponse will be received from server or maximal number of tries will be
36+
reached. In the protocol being described the client is responsible for
37+
choosing optimal length of a segment.
38+
39+
For tracking the progress of file upload client and server use identical
40+
numbering scheme for each byte of a file. The first byte of a file has
41+
number 0, the last byte has number n-1, where n is the length of file in
42+
bytes.
43+
44+
The order of transmission of a segment is not defined. Client may choose
45+
arbitrary order. However it is recommended to send segments in order
46+
ascention of byte numbers. Moreover, a user agent might decide to send
47+
multiple segments simultaneously using multiple independent connections.
48+
If a client exceeds maximal number of simultaneous connections allowed,
49+
server might return 503 "Service Unavailable" response.
50+
51+
In case of simultaneous transmission it is prohibited to send 2 or more
52+
requests with overlapping ranges within one session. Whenever server
53+
detects simultaneous requests with overlapping ranges it must return an
54+
errorneous response.
55+
56+
<span id="2.2"></span>
57+
58+
## 2.2. Encapsulation
59+
60+
Each segment of a file is encapsulated into a separate HTTP-request. The
61+
method of the request is POST. Each request contains following specific
62+
headers:
63+
64+
| Header | Function |
65+
| ------------------- | ---------------------------------------------------------------------- |
66+
| Content-Disposition | `attachment, filename="name of the file being uploaded"` |
67+
| Content-Type | mime type of a file being uploaded (must not be `multipart/form-data`) |
68+
| X-Content-Range | byte range of a segment being uploaded |
69+
| X-Session-ID | identifier of a session of a file being uploaded (see [2.3](#2.3)) |
70+
71+
`X-Content-Range` and `X-Session-Id` can also be `Content-Range` and `Session-ID`, respectively.
72+
73+
The body of the request must contain a segment of the file,
74+
corresponding to the range that was specified in `X-Content-Range` or
75+
`Content-Range` headers.
76+
77+
Whenever a user agent is not able to determine mime type of a file, it
78+
may use `application/octet-stream`.
79+
80+
<span id="2.3"></span>
81+
82+
## 2.3. Session management
83+
84+
In order to identify requests containing segments of a file, a user
85+
agent sends a unique session identified in headers `X-Session-ID` or
86+
`Session-ID`. User agent is responsible for making session identifiers
87+
unique. Server must be ready to process requests from different
88+
IP-addresses corresponding to a single session.
89+
90+
<span id="2.4"></span>
91+
92+
## 2.4. Acknowledgment
93+
94+
Server acknowledges reception of each segment with a positive response.
95+
Positive responses are: 201 "Created" whenever at the moment of the
96+
response generation not all segments of the file were received or other
97+
2xx and 3xx responses whenever at the moment of the response generation
98+
all segments of the file were received. Server must return positive
99+
response only when all bytes of a segment were successfully saved and
100+
information about which of the byte ranges were received was
101+
successfully updated.
102+
103+
Upon reception of 201 "Created" response client must proceed with
104+
transmission of a next segment. Upon reception of other positive
105+
response codes client must proceed according to their standart
106+
interpretation (see. [RFC 2616](http://www.ietf.org/rfc/rfc2616.txt)).
107+
108+
In each 201 "Created" response server returns a Range header containing
109+
enumeration of all byte ranges of a file that were received at the
110+
moment of the response generation. Server returns identical list of
111+
ranges in response body.
112+
113+
<span id="appa"></span>
114+
115+
## Appendix A: Session examples
116+
117+
### Example 1: Request from client containing the first segment of the file
118+
119+
```http
120+
POST /upload HTTP/1.1
121+
Host: example.com
122+
Content-Length: 51201
123+
Content-Type: application/octet-stream
124+
Content-Disposition: attachment; filename="big.TXT"
125+
X-Content-Range: bytes 0-51200/511920
126+
Session-ID: 1111215056
127+
128+
<bytes 0-51200>
129+
```
130+
131+
### Example 2: Response to a request containing first segment of a file
132+
133+
```http
134+
HTTP/1.1 201 Created
135+
Date: Thu, 02 Sep 2010 12:54:40 GMT
136+
Content-Length: 14
137+
Connection: close
138+
Range: 0-51200/511920
139+
140+
0-51200/511920
141+
```
142+
143+
### Example 3: Request from client containing the last segment of the file
144+
145+
```http
146+
POST /upload HTTP/1.1
147+
Host: example.com
148+
Content-Length: 51111
149+
Content-Type: application/octet-stream
150+
Content-Disposition: attachment; filename="big.TXT"
151+
X-Content-Range: bytes 460809-511919/511920
152+
Session-ID: 1111215056
153+
154+
<bytes 460809-511919>
155+
```
156+
157+
### Example 4: Response to a request containing last segment of a file
158+
159+
```http
160+
HTTP/1.1 200 OK
161+
Date: Thu, 02 Sep 2010 12:54:43 GMT
162+
Content-Type: text/html
163+
Connection: close
164+
Content-Length: 2270
165+
166+
<response body>
167+
```

0 commit comments

Comments
 (0)