Skip to content

Commit 6952875

Browse files
author
Jorge Samuel Mendes de Jesus
authored
Merge pull request #50 from jorgejesus/pywps_4.2
Pywps 4.2.1 implementation
2 parents 86db2e4 + cc493f9 commit 6952875

18 files changed

+252
-147
lines changed

.travis.yml

+5-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
11
language: python
2+
3+
dist: xenial
4+
25
python:
36
- "2.7"
47
- "3.4"
@@ -9,8 +12,8 @@ git:
912

1013
install:
1114
- sudo apt-get update -qq
12-
- sudo apt-get install -qq gdal-bin libgdal-dev libgdal1h libgdal1-dev libgeos-dev python-dev python3-dev
13-
- pip install GDAL==1.10.0 --global-option=build_ext --global-option="-I/usr/include/gdal"
15+
- sudo apt-get install -qq gdal-bin libgdal1i libgdal1-dev libgdal-dev libgeos-dev python-dev python3-dev
16+
- pip install GDAL==1.11.1 --global-option=build_ext --global-option="-I/usr/include/gdal"
1417
- pip install -r requirements.txt
1518

1619
before_script:

README.rst

+148-48
Original file line numberDiff line numberDiff line change
@@ -1,100 +1,134 @@
1-
PyWPS demo with docker support
2-
==============================
3-
Clone pywps-flask, install libraries, build docker image (due to GDAL compiling lasts quite long)::
1+
============
2+
Installation
3+
============
4+
5+
The app depends on PyWPS and several other libraries that are listed in
6+
``requirements.txt``. It is advisable to run it using a python virtualenv to prevent package instalation problems::
47

8+
$ virtualenv -p python3 pywps_flask_env
9+
$ cd pywps_flask_dir
10+
$ . bin/activate
11+
$ git clone https://github.com/geopython/pywps-flask
12+
$ cd pywps-flask
513
$ pip3 install -r requirements.txt
6-
$ cd pywps-flask/docker/alpine/flask
7-
$ docker build -t pywps_container .
814

915

10-
Clone pywps and OWSLib::
16+
If python virtualenv is not an option::
1117

12-
$ git clone https://github.com/lazaa32/pywps.git
13-
$ git clone https://github.com/lazaa32/OWSLib.git
18+
$ git clone https://github.com/geopython/pywps-flask
19+
$ cd pywps-flask
20+
$ pip3 install -r requirements.txt
1421

15-
Set PYTHONPATH::
1622

17-
$ export PYTHONPATH=$PYTHONPATH:$PWD/OWSLib:$PWD/pywps
1823

19-
Run server::
24+
For Debian based systems you will need to install GDAL with::
2025

21-
python3 demo.py
26+
$ sudo apt-get install python3-gdal
2227

2328

24-
PyWPS example service
25-
========================
29+
When using only using `requirement.txt`, the `pywps-flask` will run for the directory that was pulled from github, for a system wise installation is it advisable to use `setup.py`::
2630

27-
This is a simple example service written using PyWPS. It has been tested with
28-
QGIS 1.8.
31+
$ git clone https://github.com/geopython/pywps-flask
32+
$ cd pywps-flask
33+
$ python3 setup.py install
2934

3035

31-
Installation
32-
------------
33-
The app depends on PyWPS and several other libraries that are listed in
34-
``requirements.txt``. You can install them with pip::
35-
36-
$ pip install -r requirements.txt
36+
=======
37+
Running
38+
=======
3739

38-
For Debian based systems you will need to install GDAL with::
40+
Simply run the python file::
3941

40-
$ sudo apt-get install python-gdal
42+
$ python3 demo.py -a
4143

42-
For Windows systems install you need to install Shapely and GDAL by using Python Wheels.
43-
If you have Shapely already installed you might have to uninstall it and installed as a Wheel for it to work::
44+
The flag `-a` will bind to the ip range `0.0.0.0` and is normally the safest option access to `pypwps-flask`
4445

45-
Download the corresponding wheel for Shapely: http://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely
46+
The `-d` option will run pywps-flask as daemon and to stop it is necessary to determine the PID and kill it, one trick is to use fuser to determine PID, and then use it to kill the process::
4647

47-
Download the corresponding wheel for GDAL: http://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal
48+
$ fuser tcp/5000
49+
$ kill -15 <PID RETURNED PREVIOUSLY>
4850

49-
$ pip install wheel
5051

51-
$ pip install Shapely?x.x.x?cpxx?none?win_xxx.whl
5252

53-
$ pip install GDAL?x.x.x?cpxx?none?win_xxx.whl
53+
==============
54+
Docker images
55+
==============
5456

57+
The docker folder contains 2 subfolders, each subfolder contains a differente pywps implementation.
5558

56-
Running
57-
-------
58-
Simply run the python file::
59+
Folder ``flask`` has the default pywps-flask implementation using only Flask while folder ``nginx`` implements pywps using Nginx and Green unicorn as WSGI server. While folder ``ubuntu`` has the same images but using phusion image (ubuntu 18.04)
60+
5961

60-
$ python demo.py
6162

63+
Flask-Alpine (basic)
64+
--------------------
6265

63-
Docker
64-
------
65-
The docker folder contains 2 subfolders, each subfolder contains a differente pywps implementation. Folder ``flask``
66-
has the default pywps-flask implementation using only Flask while folder ``nginx`` implements pywps using Nginx and Green unicorn as WSGI server.
66+
Basic pywps image is based on Alpine 3.8 and will run the native Flask service, this is not apropriate for production. The docker file can be found in: ``docker/alpine/flask/Dockerfile``
6767

6868

69-
Docker-flask
70-
------------
7169

7270
To build the image (inside the folder with the Dockerfile)::
7371

74-
$ docker build -t pywps4-demo:latest .
72+
$ docker build -t pywps/flask-alpine .
7573

7674
And to run it::
7775

78-
$ docker run -p 5000:5000 pywps4-demo:latest
76+
$ docker run -p 5000:5000 pywps/flask-alpine:latest
7977

8078

8179
Pywps will be available in the following URL::
8280

8381
$ http://localhost:5000
8482

8583

86-
Docker-nginx
84+
85+
Gunicorn-Alpine (production)
86+
----------------------------
87+
88+
This image implements the previous ``flask-alpine image`` (you need to build it first, or it will be automatically pulled from dockerhub) but wrapping flask in a WSGI server (gunicorn) where each worker runs a an app. This image allows for the following environment variables:
89+
90+
- GU_WORKERS - Numer or workers. Gunicorn uses a set of workers to run pywps (normally ``workers = (2 * cpu) + 1``). (default: 5)
91+
- GU_PORT - Port running Gunicorn (default:8081)
92+
93+
94+
95+
Gunicorn-Alpine is locate in folder ``docker/alpine/gunicorn/Dockerfile``
96+
97+
This image can already be implemented in production but it is advisable to use Nginx for HTTP load balance and Gunicorn as WSGI server (see below)
98+
99+
To build the image (inside the folder with the Dockerfile)::
100+
101+
$ docker build -t pywps/gunicorn-alpine:latest .
102+
103+
104+
And to run it::
105+
106+
$ docker run -p 8081:8081 -it pywps/gunicorn-alpine:latest
107+
108+
or::
109+
110+
$ docker run -e GU_WORKERS=10 -e GU_PORT=8082 -p 8082:8082 -it pywps/gunicorn-alpine:latest
111+
112+
Pywps will be available at the following URL::
113+
114+
$ http://localhost:8082
115+
116+
117+
Nginx-Alpine
87118
------------
88119

89-
To build the image (inside the folder with the Dockerfile)::
120+
This is the complete stack intented for production, to have a stack we require to use ``docker-compose``
121+
to build two images: ``pywps/gunicorn-alpine:latest`` and ``pywps/nginx-alpine:latest``
90122

91-
$ docker build -t pywps4-demo .
123+
Those images will be pulled from dockerhub, but they can compiled locally by building Flask-Alpine, Gunicron-Alpine and Nginx-Alpine, in this case only showing for nginx::
92124

93125

94-
Gunicorn uses a set of workers to run pywps (normally ``workers = (2 * cpu) + 1``), the default value used was 5 but it can be overwritten by setting the env flag GU_WORKERS::
126+
$ cd docker/alpine/nginx/Dockerfile
127+
$ docker build -t pywps/nginx-alpine:latest .
95128

129+
Then the stack can be started using docker compose::
96130

97-
$ docker run -e GU_WORKERS=10 -p 80:80 -it pywps4-demo:nginx
131+
$ docker-compose up
98132

99133

100134
In this case pywps (only the WPS) will be avalable on::
@@ -103,11 +137,77 @@ In this case pywps (only the WPS) will be avalable on::
103137
http://localhost
104138

105139

140+
Flask-Ubuntu (basic)
141+
--------------------
142+
143+
The same as ``Flask-Ubuntu`` but using phusion image (ubuntu 18.04)::
144+
145+
146+
$ cd docker/ubuntu/flask
147+
$ docker build -t pywps/flask-ubuntu:latest .
148+
149+
And to run it::
150+
151+
$ docker run -p 5000:5000 pywps/flask-ubuntu
152+
153+
154+
Nginx-Ubuntu (production)
155+
-------------------------
156+
157+
This image is based on ``Flask-Ubuntu`` and will require it (either build locally or pull from dockerhub). This image has Nginx and Gunicorn totally integrated as services in a docker image::
158+
159+
160+
$ cd docker/ubuntu/nginx
161+
$ docker build -t pywps/nginx-ubuntu .
162+
163+
And to run it::
164+
165+
$ docker run -p 80:80 pywps/nginx-ubuntu
166+
167+
It is possible to set the number of Gunicorn workers:
168+
169+
* GU_WORKERS - Numer or workers. (default: 5)
170+
171+
e.g::
172+
173+
$ docker run -e GU_WORKERS=10 -p 80:80 pywps/nginx-ubuntu
174+
175+
176+
177+
Volumes
178+
-------
179+
180+
181+
Named volumes allow for container content to be available in the host system. The most important folders in pywps containers are:
182+
183+
* /pywps-flask/logs
184+
* /pywps-flask/outputs
185+
* /pywps-flask/processes
186+
187+
And file:
188+
* /pywps-flask/pywps.cfg
189+
190+
Named volumes need to be created prior to ``docker run``::
191+
192+
$ docker volume create pywps_logs
193+
$ docker volume create pywps_outputs
194+
$ docker volume create pywps_processes
195+
196+
To check the path on the host to volume and other information::
197+
106198

199+
$ docker volume ls pywps_processes
107200

108201

202+
To run a docker will all the volumes available in the host::
109203

204+
$ docker run -p 5000:5000 -v pywps_logs:/pywps-flask/pywps_logs \
205+
-v pywps_outputs:/pywps-flask/pywps_outputs \
206+
-v pywps_processes:/pywps-flask/pywps_processes \
207+
-v pywps_cfg:/pywps-flask/pywps.cfg pywps/flask-alpine:latest
110208

111209

210+
THE END
211+
=======
112212

113213

VERSION.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
4.0.0
1+
4.2.1

docker/alpine/flask/Dockerfile

+16-15
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
1-
FROM alpine:3.6
2-
MAINTAINER Jorge S. Mendes de Jesus <jorge.dejesus@geocat.net>
1+
FROM alpine:3.8
2+
MAINTAINER Jorge S. Mendes de Jesus <jorge.dejesus@protonmail.com>
33

4-
ARG GDAL_VERSION=2.3.2
4+
ARG GDAL_VERSION=2.4.0
55
ARG XERCES_VERSION=3.2.2
66
ARG PROCESSOR_N=4
7+
ARG FLASK_GIT=https://github.com/jorgejesus/pywps-flask.git
8+
ARG FLASK_BRANCH=pywps_4.2
79

810
RUN apk update && apk add --no-cache \
911
git \
@@ -27,35 +29,34 @@ RUN apk add --no-cache \
2729
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
2830
geos \
2931
geos-dev
32+
3033

3134
#Compiling Xerces
32-
RUN wget http://www.apache.org/dist/xerces/c/3/sources/xerces-c-${XERCES_VERSION}.tar.gz -O /tmp/xerces-c-${XERCES_VERSION}.tar.gz && \
33-
tar xvf /tmp/xerces-c-${XERCES_VERSION}.tar.gz -C /tmp && \
34-
cd /tmp/xerces-c-${XERCES_VERSION} && \
35+
RUN wget http://www.apache.org/dist/xerces/c/3/sources/xerces-c-$XERCES_VERSION.tar.gz -O /tmp/xerces-c-$XERCES_VERSION.tar.gz && \
36+
tar xvf /tmp/xerces-c-$XERCES_VERSION.tar.gz -C /tmp && \
37+
cd /tmp/xerces-c-$XERCES_VERSION && \
3538
LDFLAGS="-s" ./configure --prefix=/usr/local/src/xerces && \
3639
make -j $PROCESSOR_N install
3740

3841
# Install GDAL
39-
RUN wget http://download.osgeo.org/gdal/${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz -O /tmp/gdal.tar.gz && \
42+
RUN wget http://download.osgeo.org/gdal/$GDAL_VERSION/gdal-$GDAL_VERSION.tar.gz -O /tmp/gdal.tar.gz && \
4043
tar xzf /tmp/gdal.tar.gz -C /tmp && \
41-
cd /tmp/gdal-${GDAL_VERSION} && \
44+
cd /tmp/gdal-$GDAL_VERSION && \
4245
LDFLAGS="-s" ./configure --with-expat=yes --with-xerces=/opt/xerces --with-geos=yes \
43-
&& make -j ${PROCESSOR_N} && make install
46+
&& make -j $PROCESSOR_N && make install
4447

45-
RUN cd /tmp/gdal-${GDAL_VERSION}/swig/python \
48+
RUN cd /tmp/gdal-$GDAL_VERSION/swig/python \
4649
&& python3 setup.py install
4750

4851
RUN rm -rf /var/cache/apk/*
4952

50-
RUN git clone https://github.com/geopython/pywps-flask.git
51-
53+
RUN git clone $FLASK_GIT -b $FLASK_BRANCH --single-branch
5254
WORKDIR /pywps-flask
5355
RUN pip3 install -r requirements.txt
54-
5556

5657
ENTRYPOINT ["/usr/bin/python3", "demo.py","-a"]
5758

58-
#docker build -t pywps-flask .
59-
#docker run -p 5000:5000 pywps-flask
59+
#docker build -t pywps/flask-alpine .
60+
#docker run -p 5000:5000 pywps/flask-alpine:latest
6061
#http://localhost:5000/wps?request=GetCapabilities&service=WPS
6162
#http://localhost:5000/wps?request=DescribeProcess&service=WPS&identifier=all&version=1.0.0

docker/alpine/gunicorn/Dockerfile

+5-5
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
FROM pywps/flask-alpine:3.6
2-
MAINTAINER Jorge Samuel Mendes de Jesus <jorge.dejesus@geocat.net>
1+
FROM pywps/flask-alpine:latest
2+
MAINTAINER Jorge Samuel Mendes de Jesus <jorge.dejesus@protonmail.com>
33

44
#For Gunicorn
55
ARG GU_WORKERS=5
@@ -17,7 +17,7 @@ RUN ln -s /pywps-flask/wsgi/pywps.wsgi /pywps-flask/wsgi/pywps_app.py
1717

1818
ENTRYPOINT ["/run_all.sh"]
1919

20-
#Build: docker build -t pywps4:gunicorn .
21-
#Usage: docker run -p 8081:8081 -it pywps4:nginx
22-
#Usage w/ 10 workers: docker run -e GU_WORKERS=10 -e GU_PORT=8082 -p 8082:8082 -it pywps4:gunicorn
20+
#Build: docker build -t pywps/gunicorn-alpine:latest .
21+
#Usage: docker run -p 8081:8081 -it pywps/gunicorn-alpine:latest
22+
#Usage w/ 10 workers: docker run -e GU_WORKERS=10 -e GU_PORT=8082 -p 8082:8082 -it pywps/gunicorn-alpine:latest
2323

docker/alpine/nginx/Dockerfile

+4-3
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
FROM nginx:1.13.7-alpine
1+
FROM nginx:1.15.7-alpine
22
MAINTAINER Jorge Samuel Mendes de Jesus <[email protected]>
33

44
RUN rm /etc/nginx/conf.d/default.conf
55
COPY pywps.conf /etc/nginx/conf.d/pywps.conf
66

77

8-
#Build: docker build -t pywps4:nginx .
9-
#Usage: docker run -p 80:80 -it pywps4:nginx
8+
#Build: docker build -t pywps/nginx-alpine:latest .
9+
#Usage: docker-compose up
10+
#Usage: docker run -p 80:80 -it pywps/nginx-alpine:latest

docker/alpine/nginx/docker-compose.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@ version: '3.1'
22

33
services:
44
pywps:
5-
image: pywps/gunicorn-alpine:3.6
5+
image: pywps/gunicorn-alpine:latest
66
nginx:
7-
image: pywps/nginx-alpine:3.6
7+
image: pywps/nginx-alpine:latest
88
build: .
99
depends_on:
1010
- pywps

0 commit comments

Comments
 (0)