Collection:

Unable to render {include} Couldn't find a page to include called: mpeg video with Danish TV broadcasts

Issue:

Title	IS22 Characterise and Validate very large mpeg-1 and mpeg-2 files
Detailed description	Collections of very large videofiles (50Gb+ each) are hard to handle when it comes to characterisation and validation. Known characterisation tools do not nessecarily like very large files. Not all needed formats are well supported (if supported at all) in known tools (JHove, JHove2, FITS, XC*L)
Scalability Challenge	Tools need to be able to work on very large files (50Gb+) and in a distributed environment to scale (SB holds more than 400Tbytes mpeg-1/2)
Issue champion	Gry Elstrøm (SB)
Other interested parties
Possible Solution approaches	1. Survey and test existing tools for scalability when it comes to file size 2. Survey and test existing tools for support for mpeg-1 and mpeg-2 3. Adapt tools to support large files and/or extend format support in tools
Context
Lessons Learned
Training Needs
Datasets	Danish TV broadcasts, mpeg videos
Solutions	Optional solution - SO25 Rosetta v3.0 Implementation Integrated with DROID 6 Characterise (not validate) solution - SO06 FFprobe

Objectives	This is about robustness and scaleability as well as advanced functionality. The corresponding collection is currently over 700Tbytes
Success criteria	We will have a workflow that gives technical output of all the mpeg-files and validates the files to identify problematic files that current or future tools might have problems with
Automatic measures	1. Tool support for very large files (75Gb) 2. Process 2Tbytes of sample content in less than 24 hours
Manual assessment	1. The workflow gives useful output. Should be understandable by curators 2. The majority of the files deemed NOT VALID have a human understandable problem
Actual evaluations	links to acutual evaluations of this Issue/Scenario

These solutions have proven to work

Title	SO06 Use Ffprobe to characterise audio/video
Detailed description	A detailed description of the Solution. Feel free to include links to further information (eg. OPF blog posts!). Note that a Solution is a specific digital preservation application of a software tool or tools. It might for example be a scripted tool, or a myExperiment workflow.
Solution Champion	Asger Askov Blekinge
Corresponding Issue(s)	IS24 Characterisation of large amounts of wav audio IS22 Characterise and Validate very large mpeg-1 and mpeg-2 files
myExperiment Link	A link to a corresponding workflow on myExperiment
Tool Registry Link	http://wiki.opf-labs.org/display/TR/Ffprobe
Evaluation	Any notes or links on how the solution performed. This will be developed and formalised by the Testbed SP.

and

Title	SO34 Use Manzanita Crosscheck to validate mpeg transport streams
Detailed description	A detailed description of the Solution. Feel free to include links to further information (eg. OPF blog posts!). Note that a Solution is a specific digital preservation application of a software tool or tools. It might for example be a scripted tool, or a myExperiment workflow.
Solution Champion	Who owns the Solution? Identify the owner with a link to their contact page on the SCAPE Sharepoint site, as well as identifying their institution in brackets. Eg: Schlarb Sven (ONB)
Corresponding Issue(s)	A bulletted list of links to Issues to which this provides a Solution
myExperiment Link	A link to a corresponding workflow on myExperiment
Tool Registry Link	A link to information about the tool itself. Ideally this should point to an entry in the OPF Tool Registry Click here if you need to create a new Tool Registry entry
Evaluation	Any notes or links on how the solution performed. This will be developed and formalised by the Testbed SP.