About these pages

The pages referenced below form a network of Datasets, preservation and curation Issues with those Datasets, and Solutions to those Issues. As such, these pages capture information and requirements about concrete digital preservation and curation challenges, that are present in specific datasets and collections. The experiences of solving these Issues are written up on Solution pages. These in turn link to pages in the OPF Tool Registry, and to actual code that can be downloaded and re-used.

The purpose of these pages is to share experiences in solving preservation and curation problems, so we can learn from each other, and to articulate practitioners needs and requirements to those who are in a position to produce practical solutions to their problems.

Support

The work collated on this page is supported by: Open Preservation Foundation, Jisc, European Commission, Digital Preservation Coalition, SPRUCE Project, AQuA Project, SCAPE Project, and you!

Practitioners need better characterisation tools

Analysis of the Datasets, Issues and Solutions collated on this page indicated a broad cross section of preservation requirements, but an overriding need for more effective characterisation. Practitioners need to understand more about their data and it's condition, typically for quality assurance, appraisal and assessment and for identifying preservation risks. This analysis and details of these conclusions are described in this poster, published at the 8th International Digital Curation Conference, Amsterdam, January 2013.

Get involved

Anyone can contribute to these pages. All you need to do is register for an OPF account (its quick, free, and anyone can do it), and then start adding comments, adding value to existing pages, or contributing new ones. Please help us make this a valuable resource for all!

Datasets

These are the Datasets or collections that relate to specific preservation Issues which in turn (may) have Solutions developed for them. The Datasets are categorised by their media type. Click this link to create a new Dataset, then edit the italicised text.

Audio datasets

Label: audio

Disk image datasets

Label: disk_image

Document datasets

Label: document

Email datasets

Label: email

Geodatasets

Label: geodatasets

Image datasets

Label: image

Mixed/Misc datasets

Label: mixed_misc

Research datasets

Label: researchdata

Software datasets

Label: software

Web datasets

Label: web

Video datasets

Label: video

Other / untagged

Issues

These are the preservation or other business driven Issues that relate to a specific Dataset and may have one or more specific Solutions developed to solve them. The aim of an Issue page is to provide a detailed description of the preservation challenge and the requirements of the Issue Owner that will help to inform development of a Solution that solves the Issue.
Click this link to create a new Issue, then edit the italicised text.

Unsolved issues

Issues that do not have linked solutions. Why not suggest or contribute a solution?

Label: unsolved_issue

Appraisal and assessment issues

Issues related to appraising or assessing digital content as the first step in deciding how to proceed with preservation activities.

Label: appraisal_assessment

Bit rot issues

Issues related to Datasets that exhibit bit rot (files damaged by imperfect storage, failed write operations or software/processing errors) and require a Solution to identify, and if possible repair, problematic files.

Label: bit_rot

Conformance issues

Issues where Dataset content does not match a required profile, or needs to be checked or validated against a particular profile. These profiles are typically determined by an organisation's collection or preservation policy.

Label: conformance

Contextual issues

Issues related to the wider context of a particular Dataset.

Label: context

Data capture issues

Issues related to the capture, harvesting or extraction of data in order to facilitate effective preservation and access.

Label: data_capture

Duplication issues

Duplicated files can arise from a number of causes. Identical duplicates are relatively easy to detect. Similar duplicates (eg. one file processed from another, or the same item scanned on a different device) can require much more complicated Solutions.

Label: duplication

Embedded objects issues

Objects embedded within other objects (such as OLE, OOXML, PDF, ZIP) can pose identification, appraisal or risk assessment challenges.

Label: embedded_objects

External dependency issues

Issues relating to digital objects that have dependencies on other objects or content on the web.

Label: dependency

Integrity issues

Issues relating to ensuring the integrity or fixity of Datasets.

Label: integrity

Obsolescence, preservation risk and business constraint issues

Issues that relate to the obsolescence of Datasets, preservation risk or business constraints placed on the way that Datasets are managed.

Label: obsolescence

Planning and management issues

Issues that relate to the general planning and management of digital preservation.

Label: planning_management 

Quality issues

Issues relating to Datasets containing quality issues caused by digitisation, processing or format migration.

Label: qa

Retention/disposal issues

Issues relating to the retention, disposal and/or deletion of digital objects.

Label: retention

Rights issues

Issues related to rights or permissions that cause difficulties in managing or preserving Datasets.

Label: rights

Structural relationship issues

Digital entities can be made up of a number of objects (eg. masters, services copies, metadata). Structural relationships are important to understand which objects are part of an entity and what they for.

Label: structural_relationships

System obsolescence issues

Issues related to the obsolescence of software or other systems that manage Datasets.

Label: system_obsolescence

Unknown characteristics issues

Issues related to Datasets with unknown characteristics that are necessary for a preservation, management or other business need.

Label: unknown_characteristics

Unknown file formats issues

Datasets containing unknown file formats tend to pose a preservation risk and make management of them difficult.

Label: unknown_file_formats

Value and cost issues

Issues relating to the cost of Dataset management or the Value of the Dataset to its owners and users.

Label: value_cost

Other / untagged

Issues that haven't been tagged with any of the labels listed in this column. This provides a useful mechanism for catching Issues that have not been tagged with sufficient detail, or identifying the need to add new labels to this page.

Solutions

These are Solutions that address specific Issues encountered in particular Datasets. Solutions are typically quite specific to a particular Issue and Dataset but many will have a wider application. For details of tools utilised in a Solution, either follow links from individual Solution pages or see the Tool Registry.
Click this link to create a new Solution, then edit the italicised text.

Appraisal and assessment solutions

Solutions for assessing or appraising datasets.

Label: appraisal_assessment

Bit rot detection and repair solutions

Solutions for detecting and possibly repairing bit rot Issues in Datasets.

Label: bit_rot_detection

Characterisation solutions

Solutions for characterising content.

Label: characterisation

Data capture solutions

Solutions for capturing data from an external source, or imaging data from hand held media.

Label: data_capture

De-duplication solutions

Solutions for detecting and managing duplicated digital objects or datasets.

Label: de-duplication

Embedded object solutions

Solutions for managing and preserving embedded digital objects.

Label: embedded_objects

Emulation solutions

Solutions utilising emulation or virtualisation technologies.

Label: emulation

File format identification solutions

Solutions for identifying file formats.

Label: identification

Fixity solutions

Solutions for addressing integrity issues using approaches for generating and verifying fixity information such as manifests and checksums.

Label: fixity

Migration solutions

Solutions for migrating data from one format to another.

Label: migration

Miscellaneous solutions

Solutions for miscellaneous topics.

Label: miscellaneous

Quality assurance solutions

Solutions for assessing or identifying quality Issues in Datasets.

Label: quality_assurance

Rights managment solutions

Solutions for managing permissions and rights Issues.

Label: rights

Structural relationship solutions

Solutions for preserving or checking the structural relationships between digital objects belonging to a particular entity.

Label: structural_relationships

Validation solutions

Solutions for validating the conformance of digital objects to file format specifications or institutional profiles.

Label: validation

Other / untagged

Solutions that haven't been tagged with any of the labels listed in this column. This provides a useful mechanism for catching Issues that have not been tagged with sufficient detail, or identifying the need to add new labels to this page.