The OPF Data Endpoint

The OPF Data Endpoint Install Guide

Core Software

• Apache2+
• PHP5+
• Curl
• Puelia-PHP 
• data.gov.uk extensions
• mysql-server
• Graphite RDF Library
• 4Store
• LDS3 SPARQL Code
• LDS3 System Code

Install Guide

Firstly you need a public domain which points at a shared web directory. We shall be installing the whole system in this location. 

First install the core software, for debian do the following:

Once you have this software, in the web directory create the following folders:

Now checkout the code to go into these folders

 Next we need to move some things around, thus breaking lots of svn links, but this get's us all set up quickly. From your web root dir:

OPTIONAL: Next we can download the example data file and test it all works at this point.

Lastly then we need the htaccess file in the web_root directory which activates puelia, this is first moved from the puelia directory

It now needs to be edited such that the path is correct, as follows:  

You will also need to ensure that AllowOverride ALL is set in your apache config for your VirtualHost docs directory and that the rewrite module has been enabled.

Now we need to update all the permissions so that the webserver group can write to the puelia (and other) directories.

At this point you should be able to go to the web root in a browser and view the example data. 
You will notice that you have the government banner along the top of the screen. This can be replaced with your own in the following way (NOTE: The Admin system is required to use the admin banner).

Installing the Admin System (April 2012)

The admin system is currently (April 2012) only available from the Open Planets GitHub @ https://github.com/openplanets/opf_lds3 and it is best to download the latest tag release from the tags tab. 

Once you have downloaded the latest tag you will need to extract the files and then copy the admin and sparql directory from the unzip'd folder into your web root. 

The puelia_patches directory is needed in the next stage but can be removed once this stage has been applied. 

Unless you want the unit_tests (which are available on the http://www.lds3.org website) then these and the rest of the downloaded files can be discarded at this point. 

Patching Puelia

This is necessary to enable the following:

• Fix a Puelia bug with url encoding unnecessarily
• Enabling Puelia to properly handle LDS3 documents. 

In the puelia_patches directory you will find several files (including a README) which repeats the following instructions:

To enable Puelia to handle LDS3 documents (and the admin system etc)

To fix the unl encoding issues:

Configuring the Admin System

In your admin directory you will find a config.php.example file that needs to be renamed:

You can then edit this file (as you will need to do throughout) to specify various parameters to make your system work. 

Most of the documentation for this file is contained within it. Note that we created the required doc directory earlier. 

Making the Admin System Work 

Once you have the admin system in /admin we need to get back the style, this can be done by linking to the puelia-php stylesheets

Registering with Google

To enable login using Google Accounts (which is the only supported mechanism the system comes with and why you need an externally accessible domain) you need to add your domain info to the API Access section of the Google Console @ https://code.google.com/apis/console

In the API Access section you need to create a new client id for a web application.

The domain should be set to the base of your domain (e.g. http://beta.lds3.org) and the redirect page MUST be set to this domain /admin/glogin.php. 

Once you have done this edit the config.php file in the admin directory and insert your google_client_id and google_client_secret.

You should now be able to log in to the system using a valid google account.

Setting up the Database

The database is used for user-records and storage of related key-pairs needed to upload data into the system via the LDS3.org API. 

First step is to create a mysql database, username and password, these need to be created manually. 

At this point you can edit the config.php file again and add the database details here. 

Then you will need to create the database tables from the schema file in database/database_schema.sql as follows:

In the config.php file you can also enable account moderation (i.e. people can't use their accounts until they have been approved) by changing the value of $moderate_accounts to true. 

Unit Test 1 - Authentication Test

Once the database is set up, you need to logout and login to create your account in the database.

From the Operations Panel you should select Manage Keys. 

On this page you can add a new key with a description on what you are going to use it for. The Key and Secret are auto-generated and cannot be changed.

With the key generated browse to http://www.lds3.org/unit_tests/authentication and enter in http://_your-domain_/admin/unit_tests/auth.php as the URL along with your key and secret to test this is working. 

If it is working try changing the key and secret to also check that it blocks access for invalid keys. 

Installing Graphite

Graphite needs to be installed in the admin/library path and can be installed easily on linux using the following commands:

Installing 4Store

Although other Quad/Triple stores are available, this LDS3 implementation was written specifically with 4Store in mind. Changing this to another product will require the replication and re-implemtation of the admin/library/4store_functions.inc.php file. 

To install 4Store on your platform it is best to follow the instructions on http://4store.org/. 

With these done you need to create a store, again this should be done by following the Using instructions on the http://4store.org/ website.

As a brief guide you should end up doing something like the following. 

This will give you a 4Store with an HTTP endpoint running (hopefully on a protected and private local port). 

The details of where the HTTP server is running will need to be added to your config file.

The .htaccess file

In order for the Document Management Unit Tests to work you need modify the previously created .htaccess file in the root web directory (e.g. alongside the doc, puelia and other directories).

It should now look like the following:

Note that if you want to use the unit_tests on the server then you will need to change the htaccess file such that POST requests can be made to these.

The Puelia api-config-files

Earlier we created a config directory in which our puelia api-config-files will reside. You should ensure that this is listed in your config file.

With this and the rest of the config done, this allows us to automatically generate the puelia api-config-files that allow us to actually browse the data. 

In the admin directory you will find a write_puelia_templates.php that can simply be executed to generate these files. 

Once done you should be able to simply browse to your data host domain (e.g. http://my-domain.org/) and then follow the links to the various datasets.

NOTE: Initially all of these should be empty until you populate them, either with Live data or the Unit-Test data.

Document Management Unit Tests

Now that you have your system working, it should be possible to run all other unit tests from http://www.lds3.org/unit_tests/ with successful outcomes. 

In most of the tests input fields such as Access Key and Secret should be obvious as these are the keys you should have created in the admin system by now. 

The Data Submission URI will be:

And the IRI-Prefix will thus be: