GEOSS Banner

Steps for Plymouth Marine Laboratory implementation of Python WPS


Plymouth Marine Laboratory (PML), implements the python WPS implementation  (PyWPS) and it has actively supported new developments (SOAP/WSDL) contained in a separated project branch. In the near future the  SOAP-WSDL branch will be merged with the project's trunk code and released as an official PyWPS release. 

Code development  was done in the NETMAR project [2] under FP7-EU financing, and had the following objectives:

  • Improvement of WPS-WSDL interface and SOAP support
  • Automatic conversion of WPS processes into WSDL
  • Easy integration with Taverna-workbench
  • Support for asynchronous services in WSDL

The automatic conversion of WPS processes into WSDL was a  successfully strategy, that allowed for GRASS GIS modules to be ported into a WSDL document with help of the WPS-GRASS-Bridge project [3].

PML also supports PyWPS but running an updated Wiki with the latest tutorials and information [4]

<<Prev T.O.C


PyWPS Web Processing Service 3.2: is a Python program which implements the OGC WPS 1.0.0 standard.  PyWPS can be run in windows and linux, but windows implementation lack asynchronous capabilities due to problems forking python processes in Windows. PyWPS doesn't provide out of the box processes since it is mainly focused on providing a framework where developers can developer their own processes, nevertheless PyWPS supports easy-to-do connections to GRASS, GDAL and Rpy. PyWPS is normally run as a CGI script but it can also be run using mod_python and support for mod_wsgi is being finalized, since the code is written in standard python it can also be ported as a Javalet (using Jython)  and run from TomCat application server . It's lightweight and only 5000 lines of code make it a good candidate for testing new concepts. 


PyWPS follows the default Python installation procedure using the script, further configuration is necessary to deal with server paths and process folder location.  WPS-GRASS-Bridge requires GRASS 7.0 (SVN) to run the served modules and to generate their WPS description based in XML metadata. Taverna-workbench 2.3 only requires the download of a compressed folder and the execution of a lanch script to start the software.

Installation procedures will use Ubuntu 10.04.

PyWPS installation

PyWPS-3.2-SOAP requires the following debian packages:

  • python-lxml
  • python-magic
  • apache2
  • python-mapscript (optional)

To download the code it is necessary to install the SVN package

  • subversion

Packages can be installed using GUIs available in the distribution, or by running the apt-get install command:

:~#>sudo apt-get install subversion

Installing PyWPS code

Code is downloaded using the svn command:

:~#>cd ~

:~#>svn co pywps

Installation follows the default python procedure:

:~#>cd pywps

:~#>sudo python install


Inside folder  /usr/local/bin will contain a script that can be called to check if the pywps module is operational


PyWPS NoApplicableCode: Locator: None; Value: No query string found.

Content-Type: application/xml

<?xml version="1.0" encoding="utf-8"?>

<ows:ExceptionReport version="1.0.0" xmlns:ows="" xmlns:xsi="" xsi:schemaLocation="">

<ows:Exception exceptionCode="NoApplicableCode">

<ows:ExceptionText>'No query string found.'</ows:ExceptionText>




PyWPS will report an exception since the request lacks any operations/inputs. No configuration file is being used therefore no processes have been assigned to the WPS instance.

PyWPS contains a default configuration file that needs to be configure: pywps.cfg, the file may have a global or local version, the global location is added to /etc, while the local file should be  defined in the wrapper script (see ahead).  Normally pywps will locate the configuration file by searching the PYWPS_CFG environment variable.


PyWPS package provides an example file that can be used as starting template


:~#> sudo cp ~/pywps-3.2-soap/pywps/default.cfg  /etc/pywps.cfg


The /etc/pywps.cfg file contains some mandatory parameters that need to be configure. The parameters relate to server paths and process locations. Assuming the default paths defined by the Apache2 configuration file (/etc/apache2/sites-available/default) and that pywps processes directory is also inside the cgi-bin/ folder:













The log file should be set allowing for apache2 to write into it

~:#>sudo touch /var/log/apache2/pywps.log

(apache2 user:group defined in /etc/apache2/envvars)

~:#>sudo chown www-data:www-data  /var/log/apache2/pywps.log

~:#>sudo chmod 744 /var/log/apache2/pywps.log

The /var/log/apache2 directory maybe have permission restrictions and the pwps.log may not have access permission, it's advisable to change directory permission:

~:#>sudo chmod a+x /var/log/apache2

PyWPS requieres that processes are contained inside a specific folder, following Python's module structure.

~:#>sudo mkdir /usr/lib/cgi-bin/processes

and another folder to store the WPS's output

~:#>sudo chown -R www-data:www-data /var/www/wpsoutptus

PyWPS follows the Python mantra "batteries not included" meaning the process development is either done by the user, or by interfacing to other systems  (e.g GRASS-GIS, GDAL R)

Processes are simple scripts that need to be structured as a  Python module (for more information read the README file in the /pywps/processes directory). 

 PyWPS provides a small set or processes as examples, in this example we will copy the processes to the server's cgi-bin/process folder

~:#>sudo cp ~/pywps-3.2-soap/pywps/processes/* /usr/lib/cgi-bin/processes/


PyWPS follows a wrapper strategy, meaning multiple access points (wrappers) to one  WPS API. Each wrapper file may contain the location of diverse configuration files, system variables and process paths. Using Nano editor (or vim) the user has compose a wrapper file named pywps.cgi as follows:


export PYWPS_CFG=/etc/pywps.cfg

export PYWPS_PROCESSES=/usr/lib/cgi-bin/processes

/usr/local/bin/ $1

The file requires the following permissions:

~:#> sudo chown www-data:www-data /usr/lib/cgi-bin/pywps.cgi

~:#>sudo chmod 744 /usr/lib/cgi-bin/pywps.cgi

Openning an browser and making the following request should generate a WPS reply:


Figure 1-pml - WPS response to a getCapabilities request

WPS-GRASS-BRIDGE INSTALL (for extra WPS processes)

WPS-GRASS-Bridge request GRASS-GIS 7.0 that is under active development and may contain some bugs and the PyXB package.

Install GRASS 7.0 (SVN)

GRASS-GIS 7.0 is obtained using the svn checkout:

:~#> svn checkout grass_trunk

(Checked out revision 48788)

For convenience it's better to use the ubuntu GIS repository [5], that contains several useful packages, necessary to compile GRASS.


It's advisable to use the PPA-Unstable repository since it contains the latest GDAL version (1.8.x)

To include this repository (in Natty 11.04):

:~#> sudo add-apt-repository ppa:ubuntugis/ubuntugis-unestable

:~#> sudo apt-get update


GDAL packages to install:

  • libgdal1-1.8.0
  • libgdal1-dev
  • gdal-bin


Packages supporting GRASS:

  • flex
  • bison
  • libproj-dev
  • libreadline6-dev
  • libtiff4-dev
  • libglut3-dev
  • tk8.5-dev  NOTE: tk8.4-dev will raise "Unable to locate Tcl includes" erros. Also proper paths to the libraries must be set.
  • libfftw3-dev
  • libblas-dev
  • libatlas-dev
  • liblapack-dev
  • libcairo2-dev
  • libftgl-dev
  • libxmu-dev
  • python-wxgtk2.8
  • python-scientific (for numpy)



:~#> CFLAG="-O3 -march=native" CXXFLAGS="-O3 -march=native" ./configure --with-proj-share=/usr/share/proj --with-freetype-includes=/usr/include/freetype2 --with-freetype=yes --with-tcltk-includes=/usr/include/tcl8.5 --with-tcltk-libs=/usr/lib/tcl8.5 --wht-blas --with-lapack --with-gdal --with-readline --enable-largefile --with-sqlite --with-cairo --with-python --with-geos --with-pthread

(proj path is necessary to that wxWindows [GUI[ knows the correct path epsg codes)

:~#> make

:~#>sudo sh -c  'make install && ldconfig'

Install WPS-GRASS-Bridge

Download pyXB version 1.1.3

:~#> svn co pyxb

Standard python install:

:~#>cd pyxb/

:~/pyxb#> sudo python install

Download the WPS-Bridge-GRASS:

:~#>svn checkout wps-grass-bridge

Get WPS-BridgeGRASS code is also from SVN (revision 82)


WPS-Briidge-GRASS doesn't require installation, only the configuration of file  containing parameters pointing to folders  ( mainly the GRASS location, log paths, temporary folder and GrassAddonPath).

The GrassAddonPath points to a folder inside the WPS-GRASS-Bridge that contain new processes based in python, supporting generic raster math calculations (e.g r.math, r.add etc):

Example of










The log file needs to have RW permissions

:~#>sudo mkdir /var/log/grass-bridge

:~#>cd /var/log/grass-bridge

:~#>sudo sh -c "touch logfile.txt &&  touch logfile_module_stdout.txt && touch logfile_module_stderr.txt"

:~#>sudo chmod 666 *


The WPS-GRASS-Bridge can be tested by calling the command the PyWPSGrassModuleStarter that will test the v.voronoi process

:~#>cd  ~/wps-grass-bridge


Note: The needs write permission, since it  save inputs files into folder containing the script.

As good practice it is advisable to regenerate the XML GRASS process description, and then the python process.

:~/#>cd ~/wps-grass-bridge/gms/Testing/Python/XML

:~/wps-grass-bridge/gms/Testing/Python/XML#>grass70 -text (enter the grass bash)


:~/wps-grass-bridge/gms/Testing/Python/XML(GRASS70)#>cd -

(to go to  the last path (wps-grass-bridge))

:~/wps-grass-bridge/#> sh


The  script will generate the pyWPS processes that will be stored in the wps-grass-bridge/pywps_services


In the cgi-bin folder  a new wrapper script should be created pointing the pywps_services/ folder. The wrapper will be named grass.cgi


export PYWPS_CFG=/etc/pywps.cfg

export PYWPS_PROCESSES=/home/pywps/wps-grass-bridge/pywps_services

python /usr/local/bin/ $1


The WPS-GRASS-Bridge processes can now be tested from the browser. The following URL will v.delaunay module, generating delaunay polygons from random points.








PyWPS log will contain all the loaded processes from /pywps_service,  it also contains Warnings reporting any processes that couldn't be loaded, eg:

PyWPS [2011-10-13 22:07:07,326] WARNING: Could not import process [<class r_terraflow.r_terraflow at 0x14fd598>]: NameError("global name 'inf' is not defined")

WPS-Grass-Bridge Add-on


WPS-GRASS-Bridge provides a special wrapper script for r.mapcalc, the wrapper script is called r.math. The r.math script is located in the ~/src/wps-grass-bridge/gms/Testing/Python/GrassAddons, the script has other 4 siblings r.add, r.div,r.mult and r. sub that perform addition, division, mutiplication and subtraction operations (respectably) to input rasters.



These add-on script are based on python's grass module, WPS-GRASS-Bridge automatically sets the Python path and loads the modules, in case of problem the Python path can be added to the wrapper script by adding it as environment variable:


export PYTHONPATH=$PYTHONPATH:/usr/local/grass-7.0.svn/etc/python

export PYWPS_CFG=/etc/pywps_grass.cfg

export PYWPS_PROCESSES=/home/pywps/wps-grass-bridge/pywps_services

/usr/local/bin/ $1


The grass python module will have problems while loading, reporting a "Unable to load GDAL library", this maybe caused by a naming problem in the GDAL library, since it is installed as, while python module will search for a The solution is to make a symbolic link to the gdal library.

:~/#>cd /usr/lib

:/usr/lib#>sudo ln -s

:/usr/lib#>sudo ldconfig

WSDL generation

WSDL generation is automatic and transparent, without any need of configuration by the user, any WPS process is represented as 1 or 2 WSDL processes according to the following decision tree:

 - WPS process with only synchronous support:

  • 1 WSDL service identified as ExecuteProcess_<service name>

- WPS with asynchronous support:

  • 1 WSDL service identified as ExecuteProcess_<service_name> returning the service's final result
  • 1 WSDL service identified as  ExecuteProcessAsync_<service_name> returning the service's status URL


The WSDL description only requires that the PyWPS configuration file properly indicates the server adress (/etc/pywps.cfg). If serveraddress is incorrect the WSDL document will contain an incorrect location attribute in the address element, and a WPS Exception will be raised as result.





It takes 1.75 min to generate a dynamic WSDL file using the request:


Since PyWPS has to generate a describeProcess&identifier=all XML document (a document with complete process description with +/- 137 GRASS processes) and then apply an XLST to convert it into WSDL.

The proper strategy is to request the document once, saved in the /cgi-bin (eg. grass.wsdl) and use it as reference point. Another strategy would be to split the pywps_service folder into a vector and raster folder and build wrapper scripts for such folder, diminishing the number of processes in a WPS instance.

Taverna  worbench

Taverna workbench is a desktop application for designing and executing workflows. The desktop application includes a workflow engine that orchestrates services based in SOAP, WSDL and RESTful protocols, the engine is also integrated in the Taverna server and which allows for remote execution of workflows.


Taverna requires the graphviz package:

:~/#>sudo apt-get install graphivz-dev graphivz

Taverna installation instructions and packages can be found here: [6]

Taverna is downloaded as package that is decompressed into a folder and run from the terminal with command:


It's advisable to change the JAVA memory options in the script.  The script contains the following execution line:

exec "$javabin" -Xmx400m -XX:MaxPermSize=140m

The memory usage could be increased as follows:

exec "$javabin" -Xmx800m -XX:MaxPermSize=280m

OGC proxy

W3C has introduced some changes to XML schemas and as introduced a "tar-pit" policy around some XML supporting servers, causing problems when fetching schemas and XML contain. The use of catalogues is not possible in the current version of Taverna 2.3, and direct access to XML contain is mandatory. OGC hasn't updated the service schemas to reflect the W3C changes, and therefore some validation problems may occurs


The OGC proxy is a reverse file proxy script written in python, that proxy will serve the W3C/OGC schemas from local files. The file serving is transparent, meaning an URL request to the schema will be redirect to the file location of the schema and served. The proxy is configure to work only in specific IP numbers (therefore W3C and OGC urls) by automatic configuration of IPTables.


Download proxy contents from PyWPS SVN:

:~/#>svn co ogcproxy

The necessary W3C schemas and all OGC schemas are contained in 2 compressed packages.

:~/#>tar -xvf w3c.tar.gz &&  tar -xvf ogc.tar.gz

Note: The ogc.tar.gz contains all the ogc schemas, the majority will not be used by the WPS

The current folder should have the following directories:



The src/ should be configured to point to the folder containing the schemas.

On line 19:

#Add here HOST and local path to folder with content, change /users/rsg/jmdj/ogc{w3c}, to folder with ogc/w3c content


  The cachedHost python dictionary should be changed to point to the folder with the schemas, the dictionaty follows the structure URL as key ==> Folder path as value.


The ipDic contains the IPs or URLs defined in the cachedHost dictionary. There is no need to change them unless the W3C and OGC sites change locations


The proxy script is initiated by running it as sudo

:~/#>cd ~/ogcproxy/src/

:~/ogcproxy/src/#>sudo python

6) An iptable listing should output the following

:~/#> sudo iptables -L -t nat



target     prot opt source               destination        


Chain OUTPUT (policy ACCEPT)


target     prot opt source               destination        


REDIRECT   tcp  --  anywhere    tcp dpt:http /* "set by PID:17010" */ redir ports 8000


REDIRECT   tcp  --  anywhere    tcp dpt:http /* "set by PID:17010" */ redir ports 8000




target     prot opt source               destination


The OUTPUT chain will redirect any traffic going to the W3C/OGC into port 8000 where the reverse proxy is listening, and serving the files.


7) The proxy is shutdown using the kill command with the PID that is indicated in the iptables rule information:

:~/#>sudo kill -9 17010

Using Taverna

The taverna workbench has a friendly-user interface divided in 3 sections: Service Panel, Workflow editor and explorer (Figure 2-pml).

The user start by indicating the WSDL location (obtained from the WPS service using a getCapabilities request), and introducing it in the "WSDL import service". Figure 3-pml and 4-pml

Figure 2-pml - Taverna-workbench GUI

The Taverna-workbench consumes the WSDL service description provided by the WPS instance.

Figure 3-pml - WSDL service import

Figure 4 -pml -Adding service URL (example)

The Service is then populated with service list (Figure 5-pml), in this case services from GRASS-GIS

Figure 5-pml - Service panel populated with several WPS processes.

The service can then be drag/drop into the workflow editor on the right. The service will appear as a box wrapped in 2 I/O boxes (DataInputs and ProcessOutputs) (Figure 6-pml)


Figure 6-pml - Service representation

Figure 7-pml - Service representation with inputs contained inside the DataInputs element

The service representation follows the WPS standard as DataInputs is the parent element of inputs and ProcessOutputs is the parent element of processe's output (Figure 7-pml).


By positioning the mouse over the process name and right click, the service menu is open and the Add XML input Splitter (Figure 8-pml) should be clicked to show the actual inputs expected in the DataInputs

Figure 8-pml - Adding XML input splitter to determine the WPS's inputs

The same situation for the ProcessesOutputs. Taverna assumes that DataInputs is the service Input and ProcessOutptus is the output and in turn these I/O are assemble by a pre (or pos) service that assemble the XML content.

The workflow will not be complete with a "Workflow input ports" and "Workflow output ports", these boxes represent the actual I/O of the workflow and the input one will query the user for input locations. The workflow I/O are added by clicking the red and green triangle located in the tool bar (Figure 9-pml)

Figure 9-pml - Toolbar icons. Red triangle for input, inverted green triangle for output and play symbol to launch workflow execution

Connecting the several components a final workflow would look as follows (Figure 10-pml):

Figure 10-pml - Complete workflow calling the r.add process.

The r.add process is provided by GRASS-Bridge-WPS and its "inputs" content is described as follows:

<Input minOccurs="1" maxOccurs="1024">


                <ows:Title>Raster maps to add.</ows:Title>














The input is a multiple input that can range from 1 till 1024 images. In Taverna this sort of input is refereed  as "list input" (Figure 11-pml,  12-pml and 13-pml)

Figure 11-pml - Input port configuration.

By right click on the input port  then "Edit workflow input port" and then select "list of depth", it's possible to use WPS's multi input capabilities.

Figure 12-pml - List select. Multiple input WPS

To run the workflow the user can use Ctr+R or click on the run button in the toolbar, this will open the input request port.

Figure 13-pml - Input dialogue GUI containing two inputs (elev_srtm_30m.tif file)

The user can then add the inputs to be sent to the service, in this example 2 identical URLs will be sent used as input (Figure 13-pml). After setting the inputs and clicking on the "Run workflow" the user will be conducted to the Results view, there is is possible to check the workflow as it is run and its final output, as follows (Figure 14-pml):

Figure 14-pml - Workflow output. Output image is in base64 code, therefore identified by Taverna as a text (string) output

The output is located in the lower right section. The presented result in this example is a base64 encoded image, hence the long char string (Figure 14-pml).

Taverna provides local services (located in the "Local Services" of the service panel) that can decode base64 into a binary format for display,this service in conjunction with another service that transforms geoTiff into PNG allow for the construction of the following workflow (Figure 15-pml):


Figure 15-pml - Extended workflow with image transformation (base64 encoding into binary)


That when run will have the following output (Figure 16-pml):

Figure 16-pml - Image PNG displayed in Taverna.

Taverna contains extensive tools and functionalities that help workflow development. It's adivsable to consult Taverna's web site and read the tutorials and examples[7].

WPS-Async call in Taverna

Async calls to WPS services return an URL pointing to a document with information concerning the service status and then it's final result.

The user would have to select the Asynchronous service, that is indicated with prefix "ExecuteProcessAsync_" and that outputs a "statusURLResult".

Figure 17-pml - Example of an asyn call to the "ultimatequestion process" that takes 25 seconds to complete.

The statusURLresult would be then passed to a local process (programmed in BeanShell) that would query the status URL until a output is present and passed to the following process (or output box) (Figure 17-pml).

For an extended tutorial and complete example please see [8]: 


myExperiment is a social web site that allows for researchers to share their scientifical workflows, any workflow developed in Taverna using WPS-WSDL can be uploaded to such platform and downloaded/consulted by other peers:

Figure 18-pml - Example of workflow search in myExperiment, using "r.los" as word query

Figure 18-pml, shows  a workflow search, in this search the keywork was "r.los" that is a GRASS module that generates a raster output map in which the cells that are visible from a user-specified observer position are marked with the vertical angle (in degrees) required to see those cells (viewshed).

The r.los workflow example can be downloaded directly into Taverna or explored as a PNG in the site (Figure 19-pml)

Figure 19-pml - Generic representation of a workflow containing the r.los web service




OpenLayers (OL) is an open source JavaScript API for displaying map data in web browsers. The API is extensive and offers the following functionalities:


  • Usage of external layers from 3rd source providers like Google Maps, Bing Maps, OpenStreet Maps;

  • Support of map servers like Geoserver or Mapserver

  • Support for OGC standards like WMS, WFS and SOS

  • Support for multiple raster and vector formats: GML,GeoTiff, KML


WPS support is part of OL version 2.11, and is under active development, and future developments concerning WPS integration and OL should use the API's native functionalities. Plymouth Marine Laboratory has developed some examples of WPS integration into OL, but based on the PyWPS javascript client that provides support for the 3 WPS operations, HTTP request/response and XML processing.

WPS implementation in OL has 2 strategies:

  • Toolbox, were user uses getCapabilities/describeProcess to determine which process to run;

  • Pre-defined process that will be run when necessary (e.g an event that triggers a JavaScript call to WPS).

The first strategy gives more freedom, it's more generalist and assumes that the user is familiar with GIS operations and that it has a clear idea what is necessary. The second strategy moves the responsibility of choosing the correct GIS operation to the developer. In any case, the WPS client has the ability to gather data from OL layers and to append it back to OpenLayer once the WPS process is finished.

PyWPS Javacript client is stored in the project's trunk, from where it can be fetched using SVN


:~/#>svn co webclient



The webclient folder contains basic example of usage and integration. The Javascript client is a simple file (WPS.js) that contains all the necessary classes to request and process WPS request

The client code (WPS.js) is appended after the OpenLayers API in the HTML e.g:

<script type="text/javascript" src="" />

<script type="text/javascript" src="WPS.js"/>


For extended documentation please consult [9].

PML has a demonstration page where OpenLayers calls a WPS instance containing the [10] process to find the shortest path on a vector network. The PML example [11] is based in a vector network over imposed major Iberian cities.


Figure 20-pml - Openlayers example of WPS usage. WPS instance with will determine the shortest path between two points (car and flat points).



The WPS process based on WPS-GRASS-Bridge can easely determine the shortest path between two points in vector network, the calculated result is returned as GML 2.1.2 and it is integrated into OL as an extra layer.





Figure 21-pml  - Shortest path result from a WPS service and appended to OpenLayers.


The OL example web page code is contained in a zip filed that can used as a study example [12].



Files (0)

You must login to post a comment.