Skip to end of metadata
Go to start of metadata

Contents

Task List

46%

SCAPE components TODO list

  1. handler

    Support annotation extraction and query in myExperiment (MAN)

    HIGH
    mferreira
    Dec 03, 2013
    Dec 03, 2013
  2. handler

    Implement, test and release extended myExperiment API – working on component plugin, myE API to go live this week

    HIGH
    dkf
    Oct 22, 2013
    Nov 21, 2013
  3. handler

    Generate Components for existings tools and publish to MyExperiment (PC developers)

    HIGH
    mferreira
    Dec 03, 2013
  4. handler

    Finalize component profile specification

    HIGH
    duretec
    Nov 21, 2013
  5. handler

    Validate workflow according to profile

    MEDIUM
    dkf
    Nov 21, 2013
  6. handler

    Review component end-2-end workflow whitepaper comments and update

    MEDIUM
    pmay
    Oct 22, 2013
    Jan 28, 2014
  7. handler

    Report status of support for complex annotations to Markus P

    MEDIUM
    pmay
    Nov 21, 2013
    Jan 28, 2014
  8. handler

    Extract and validate dependencies on the Platform (AIT)

    MEDIUM
    mferreira
    Dec 03, 2013
  9. handler

    Enhance toolwrapper (KEEPS, started)

    MEDIUM
    pmay
    Oct 22, 2013
  10. handler

    Component Validation on Taverna (MAN)

    MEDIUM
    mferreira
    Dec 03, 2013
  11. handler

    Clarification from Per about external post-processing step for command line tool components – Miguel talked to Per and point was taken. Post-processing will happen within the component.

    MEDIUM
    mferreira
    Oct 22, 2013
    Oct 22, 2013
  12. handler

    Check status of tasks in Preservation Components, Workflows, Planning and Platform integration – outdated page

    MEDIUM
    pmay
    Oct 22, 2013
    Dec 03, 2013
  13. handler

    [All] Update "developments" as per Toolwrapper example

    MEDIUM
    all
    Oct 22, 2013

"Things" under development

Components specs

Examples of SCAPE components: https://github.com/openplanets/scape-component-profiles/tree/master/examples

Component profiles spec

The component profiles are available in the component profile repository.

The used ontology is also developed in this repository and published on http://purl.org/DP/components alongside with documentation.

MyExperiment API 

(supports creating, searching and managing components)

This is anticipated to go live this week (by 25/10/13). (Documentation link.)

Component plugin for taverna

Currently testing version 1.1.0 (will release once it works!) (Browse source.)

Toolwrapper - Wrapped tools from PC

Module name Toolwrapper
Description The toolwrapper is a tool that, based on a small specification file, is able to produce the following outputs:
  • Bash scripts that standardizes the invocation of Preservation tools
  • Debian packages for easy installation

    The toolwrapper will be enhanced to also support the following features:

  • Generate SCAPE components according to a Component Profile (Characterization, QA, and Actions)
  • Publish generated components to MyExperiment
WP PC.WP.2 - Action Services
Responsible Miguel Ferreira (WP lead)

Tasks

Task Depends of Status Responsible Due date
Development of new version of the toolwrapper Component examples DONE (see github) Hélder Silva (KEEPS) December
Development of new tool that publishes SCAPE components to MyExperiment MyExperiment API UNDER DEVELOPMENT
Hélder Silva (KEEPS)
December

References

Useful information Description URL
SCAPE Component examples Examples of how components should look like. These will be used as templates
by the new version of the ToolWrapper so that Components are produced automatically
by this tool
Github: SCAPE Component Profiles examples
SCAPE Component profiles Profiles that validate that a SCAPE component is according to the right specification. 
There are 3 types of profiles.
Github: SCAPE Component Profiles
MyExperiment API documentation Documentation of the API provided by MyExperiment to support the publication of 
Components by the new tool to be developed
MyExperiment: Components API

Issues

Issue description Reported by Caused by Solution owner
       
       
       

Execution module to run components on the platform

We need feedback from Rainer. 

Main question: Will we be able to submit a SCAPE component to the Job execution service and have it run on the platform.

Integration of Plato and Components

Module name Plato
Description Preservation planning currently does
  • Query myExperiment for migration components (based on samples, user input)
  • Configure migration components based on parameter ports
  • Run selected migration components
  • Query CC/QA components defined on decision tree and run them

    Enhancements 

  • Allow better selection of CC/QA components
  • Create an executable plan template containing input/output ports and migration components
  • Allow configuration of CC/QA components
  • Add CC/QA components to template
WP PW.WP.3 Automated Planning
Responsible Kresimir Duretec (WP lead)

Tasks

Task Depends on Status Responsible Due date
Executable Plan Template MyExperiment API, Component Profiles PENDING Markus Plangg (TUW) December

References

Useful information Description URL
     

Issues

Issue description Reported by Caused by Solution owner
       
       
       

Integration issues

PC Requirements

On a previous PC meeting held in Paris the following whitepaper has been drafted according to PC members perspective. SCAPE_Preservation Compoments Profiles_KEEPS_V0.1.docx

The document has been shared with PW at the time.

PC/PT

How to handle non-1-to-1 migrations? (Current migration component profile handles this correctly; output files are presented component outputs, and both inputs and outputs can be singletons (=filename) or lists of singletons, so 1-many, many-1 and many-many transformations can be represented.)

PT/PW

How to express measures so that they can be assessed ("do we write this object back to the repository?")

  • Proposal: Produce a common format to contain the measures. This common container format can then be combined (using some sort of standardised combiner component) so that a single document is presented to an assessment component; the boolean output of that assessment component is then used to guide what happens next (i.e., “write back to repository” or “complain to user about a failed migration”.)
    • Needs basic worker components for container document construction, container document combination, container document assessment
    • Container format needs to be logically a collection of triples: subject, property type, measured value
      • Having an explicit subject allows reference to the input object, the output object, the comparison of the two, or anything else (e.g., intermediate transformations)
      • Property types should be mentioned in relevant ontology; not sure that subjects can be in general (too many possibilities, too dependent on the structure of the overall migration workflow)
      • Measured values should be simple strings; meaning determined by property type
    • Container format/assessment suggestion: make it be XML and use something like Schematron to do the assessment.
  • Specific format example (proposed schema):
    • Note: no caution was taken over the subject and type attribute values.
  • Specific suggested operations:
  • Consequence: The characterisation and QA components have to change to (also?) produce a metric document output.

How to get it so that PW can ask the platform to perform a migration?

  • Plan, part A: It is currently planned (check with Matthias Hahn) to have a standardised migration execution service interface that will allow a migration to be requested (probably by POSTing a description document to a service endpoint). The document will contain references to any subordinate resources required, such as:
    • the migration plan to enact,
    • the collection of digital objects to apply the migration plan to, and
    • the acceptance criteria for judging that the plan successfully applied to a particular digital object.
  • Plan, part B: It is currently planned to enhance Taverna Server to support that standardised migration execution service interface (in addition to the current service interfaces it has).
    • Note that it's not currently decided what the relationship of the requested plan is to the actual workflow to be executed. Does the plan in/referred-to the request include the staging and assessment steps?

PC/PW

How to handle similarity measures?

A question is if the two ontologies/vocabularies that being developed in SCAPE are compatible or even the same:

1. The Component measures vocabulary (http://ifs.tuwien.ac.at/dp/vocabulary/quality/measures)
2. The Policies representation ontology developed in WP13.

The question is... are both of these ontology structures compatible? Let me put it in another way, are the measures defined by the first ontology compatible with the lower nodes of the second ontology (i.e. the nodes that represent operational measures of the policy representation)?

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.