The OPF Data Endpoint

h2. The OPF Data Endpoint Install Guide

{note}
Following this document requires a high level of technical knowledge of web based systems for linked data, this is a complex and very modular process.
{note}

h2. Core Software

* Apache2\+
* PHP5\+
* Curl
* Puelia-PHP 
* data.gov.uk extensions
* mysql-server
* Graphite RDF Library
* 4Store
* LDS3 SPARQL Code
* LDS3 System Code

h2. Install Guide

Firstly you need a public domain which points at a shared web directory. We shall be installing the whole system in this location. 

First install the core software, for debian do the following:

{code}
# apt-get install mysql-server libapache2-mod-php5 php5-mysql php5-memcached php5-memcache memcached php5-xsl php5-curl subversion curl
{code}
Once you have this software, in the web directory create the following folders:

{code}
# mkdir puelia data.gov.uk config doc
{code}
Now checkout the code to go into these folders

{code}
# cd puelia
# svn co http://puelia-php.googlecode.com/svn/trunk/ .

# cd ../data.gov.uk
# svn co http://linked-data-api.googlecode.com/svn/trunk/examples/ .
{code}
 Next we need to move some things around, thus breaking lots of svn links, but this get's us all set up quickly. From your web root dir:

{code}
# rm -fR puelia/api-config-files
# cd puelia && ln -s ../config api-config-files && cd .. (THIS PATH NEEDS TO GO IN OUR CONFIG FILE LATER)
# sudo cp -r data.gov.uk/css/* puelia/css/
# cd puelia && ln -s ../data.gov.uk/xslt/ && cd ..
# ln -s puelia/images/
# ln -s puelia/scripts/
# ln -s puelia/css/
{code}OPTIONAL: Next we can download the example data file and test it all works at this point.


{code}
# cd config
# wget http://beta.lds3.org/puelia/api-config-files/data.openplanetsfoundation.org.ttl
{code}
Lastly then we need the htaccess file in the web_root directory which activates puelia, this is first moved from the puelia directory

{code}
mv puelia/.htaccess .
{code}
It now needs to be edited such that the path is correct, as follows:  

{code}
  RewriteEngine on
  
  RewriteCond % {REQUEST_FILENAME}  -f

{code}
# sudo chgrp www-data * -R
# sudo chmod g+w * -R
{code}
At this point you should be able to go to the web root in a browser and view the example data. 
You will notice that you have the government banner along the top of the screen. This can be replaced with your own in the following way (*NOTE:* The Admin system is required to use the admin banner).
{code}
# cd images
# rm -f hmg-banner.png
# ln -s ../../admin/lds3_images/banner.png hmg-banner.png
{code}

h2. Installing the Admin System (April 2012)

The admin system is currently (April 2012) only available from the Open Planets GitHub @ [https://github.com/openplanets/opf_lds3|https://github.com/openplanets/opf_lds3] and it is best to download the latest tag release from the tags tab. 

Once you have downloaded the latest tag you will need to extract the files and then copy the *admin* and *sparql* directory from the unzip'd folder into your web root. 

The *puelia_patches* directory is needed in the next stage but can be removed once this stage has been applied. 

Unless you want the *unit_tests* (which are available on the [http://www.lds3.org] website) then these and the rest of the downloaded files can be discarded at this point. 

h2. Patching Puelia

This is necessary to enable the following:

* Fix a Puelia bug with url encoding unnecessarily
* Enabling Puelia to properly handle LDS3 documents. 

In the *puelia_patches* directory you will find several files (including a README) which repeats the following instructions:

To enable Puelia to handle LDS3 documents (and the admin system etc)


{code}
# cd puelia_patches
# cp pre_processor.php ../puelia/
# patch ../puelia/index.php pre_processor.patch
{code}
To fix the unl encoding issues:


{code}
# patch ../puelia/graphs/configgraph.class.php puelia_graph_patch.patch
{code}

h2. Configuring the Admin System

In your *admin* directory you will find a *config.php.example* file that needs to be renamed:

{code}
 mv config.php.example config.php 
{code}
You can then edit this file (as you will need to do throughout) to specify various parameters to make your system work. 

Most of the documentation for this file is contained within it. Note that we created the required doc directory earlier. 

h2. Making the Admin System Work 

Once you have the admin system in /admin we need to get back the style, this can be done by linking to the puelia-php stylesheets
{code}
# cd admin
# ln -s ../puelia/css
# ln -s ../puelia/images
# ln -s ../site_images
# cd ..
# cd sparql
# ln -s ../admin/config.php
{code}

h2. Registering with Google

To enable login using Google Accounts (which is the only supported mechanism the system comes with and why you need an externally accessible domain) you need to add your domain info to the *API Access* section of the *Google Console* @ [https://code.google.com/apis/console]

In the API Access section you need to create a new *client id* for a *web application.*

The domain should be set to the base of your domain (e.g. [http://beta.lds3.org]) and the redirect page *MUST* be set to this domain */admin/glogin.php*.* *

Once you have done this edit the *config.php* file in the *admin* directory and insert your *google_client_id* and *google_client_secret*.

You should now be able to log in to the system using a valid google account.

h2. Setting up the Database

The database is used for user-records and storage of related key-pairs needed to upload data into the system via the LDS3.org API. 

First step is to *create a mysql database, username and password,* these need to be created manually. 

At this point you can edit the *config.php* file again and add the database details here. 

Then you will need to create the database tables from the schema file in *database/database_schema.sql* as follows:


{code}
mysql -u username -p password -h host database_name < database/database_schema.sql
{code}
In the *config.php*&nbsp;file you can also enable account moderation (i.e. people can't use their accounts until they have been approved) by changing the value of&nbsp;*$moderate_accounts* to *true*.&nbsp;

{note}
At the time of publishing enabling of accounts in moderated mode has to be done by hand with a database query by changing the value of the enabled field in the Users table to 1 (-1 = banned, 0 = awaiting moderation, 1 = active)
{note}

h2. Unit Test 1 - Authentication Test

Once the database is set up, you need to logout and login to create your account in the database.

From the *Operations Panel* you should select *Manage Keys*.&nbsp;

On this page you can add a new key with a description on what you are going to use it for. The Key and Secret are auto-generated and&nbsp;*cannot*&nbsp;be changed.

With the key generated browse to&nbsp;[http://www.lds3.org/unit_tests/authentication|http://www.lds3.org/unit_tests/authentication]&nbsp;and enter in [http://_your-domain_/admin/unit_tests/auth.php] as the URL along with your key and secret to test this is working.&nbsp;

If it is working try changing the key and secret to also check that it blocks access for invalid keys.&nbsp;

h2. Installing Graphite

Graphite needs to be installed in the *admin/library* path and can be installed easily on linux using the following commands:

{code}
cd admin/library
mkdir graphite
cd graphite
curl -s http://graphite.ecs.soton.ac.uk/download.php/Graphite_and_ARC2.tgz | tar xzvf -
{code}

h2. Installing 4Store

Although other Quad/Triple stores are available, this LDS3 implementation was written specifically with 4Store in mind. Changing this to another product will require the replication and re-implemtation of the *admin/library/4store_functions.inc.php*&nbsp;file.&nbsp;

To install 4Store on your platform it is best to follow the instructions on&nbsp;[http://4store.org/|http://4store.org/].&nbsp;

With these done you need to create a store, again this should be done by following the *Using*&nbsp;instructions on the&nbsp;[http://4store.org/|http://4store.org/]&nbsp;website.

As a brief guide you should end up doing something like the following.&nbsp;

{code}
4s-backend-setup lds
4s-backend lds
4s-httpd -p 5052 lds
{code}
This will give you a 4Store with an HTTP endpoint running (hopefully on a protected and private local port).&nbsp;

The details of where the HTTP server is running will need to be added to your *config* file.

h2. The .htaccess file

In order for the *Document Management Unit Tests*&nbsp;to work you need modify the previously created&nbsp;*.htaccess* file in the root web directory (e.g. alongside the doc, puelia and other directories).

It should now look like the following:

{code}
RewriteEngine on

DirectorySlash Off

RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} !/sparql/index.php
RewriteCond %{REQUEST_URI} !/admin/manage_keys.php
RewriteRule ^.*$ admin/handler.php

RewriteCond %{REQUEST_METHOD} DELETE
RewriteRule ^.*$ admin/handler.php

RewriteCond %{REQUEST_URI} ^/doc/.*$
RewriteRule ^.*$ puelia/index.php [L]

RewriteCond %{REQUEST_FILENAME}  -f
RewriteRule  ^.*$  -  [L]
RewriteRule ^.*$  puelia/index.php [L]
{code}

Note that if you want to use the unit_tests on the server then you will need to change the htaccess file such that&nbsp;*POST*&nbsp;requests can be made to these.

h2. The Puelia api-config-files

Earlier we created a *config* directory in which our puelia api-config-files will reside. You should ensure that this is listed in your *config* file.

With this and the rest of the *config* done, this allows us to automatically generate the puelia api-config-files that allow us to actually browse the data.&nbsp;

In the *admin* directory you will find a *write_puelia_templates.php*&nbsp;that can simply be executed to generate these files.&nbsp;

{code}
cd admin
php write_puelia_templates.php
{code}
Once done you should be able to simply browse to your data host domain (e.g. [http://my-domain.org/]) and then follow the links to the various datasets.

NOTE: Initially all of these should be empty until you populate them, either with Live data or the Unit-Test data.

h2. Document Management Unit Tests

Now that you have your system working, it should be possible to run all other unit tests from&nbsp;[http://www.lds3.org/unit_tests/] with successful outcomes.&nbsp;

In most of the tests input fields such as *Access Key*&nbsp;and *Secret*&nbsp;should be obvious as these are the keys you should have created in the admin system by now.&nbsp;

The *Data Submission URI*&nbsp;will be:

{code}
http://dev.my-company.org/admin/handler.php
{code}
And the *IRI-Prefix*&nbsp;will thus be:

{code}
http://dev.my-company.org
{code}