Below is an e-mail exchange between Christy Henshaw (Wellcome Trust) and Johan van der Knijff (most recent message at the top). Relevance: in addition to format conformance the Wellcome Trust wants to validate their JP2s against their institute-specific profile, which is something that is on the wish list for SCAPE as well.
From: Johan van der Knijff
Sent: 22 December 2011 13:35
To: 'Henshaw, Christy'
Cc: Thompson, Dave
Subject: RE: JP2 validator / properties extractor
Hi Christy,
Great to hear about that!
As for the compression ratio: this isn’t actually an extractable property, but jpylyzer calculates it from the number of components, the number of bits per component, the height and the width (which, combined, give you the size of the uncompressed image data) and the file size (which is the size of the compressed image data). Then the compression ratio is simply:
(size_uncompressed) / (size_compressed)
I would be very interested to hear about any progress on the schemas. Eventually we want to do something pretty similar in SCAPE (validation against institute-specific profile). Ideally we would probably want to do this using schemas as well, as this is something that doesn’t involve any additional development effort. Not quite sure if schemas will be able to cover all possible needs, but we’ll have to investigate that.
Also, keep in mind that, as this is a first prototype, there will probably some slight changes in jpylyzer’s output format in the future. I expect that this will mainly affect the stuff that’s now under ‘tests’; ‘properties’ will probably remain pretty much unchanged (although a couple of JP2 boxes are unsupported at this stage, and they will be added to upcoming versions of the tool).
Anyway, more about this next year. Meanwhile have a great Christmas, and all the best for 2012!
Cheers,
Johan
From: Henshaw, Christy
Sent: 22 December 2011 11:14
To: Johan van der Knijff
Cc: Thompson, Dave
Subject: RE: JP2 validator / properties extractor
Hi Johan,
Well, I have tried the jpylyzer and it is as EASY AS PIE. Yay!
I don’t know how to use DOS very well, so I just put sample JP2s in the same folder as the application files and deposited the XML in the same folder as well so I could use the exact command line you gave me.
I notice that <compressionRatio> is one parameter that is output – and it was correct at 6.0. I didn’t realise that this was an extractable property, so that is great.
I’m talking to someone at King’s Digital Humanities dept about the possibility of getting a set of schemas together to validate against (we might want one for lossless, and then a set for lossy corresponding to the different compression ratios we’re using (6:1, 10:1 etc.).
Happy Christmas Johan, and best of luck getting the documentation written in the New Year.
Christy
From: Johan van der Knijff
Sent: 21 December 2011 12:16
To: Henshaw, Christy
Cc: Thompson, Dave
Subject: RE: JP2 validator / properties extractor
All makes perfect sense to me. Some remarks:
- Yes
- Yes (’False’ actually)
- This is all under the ‘properties’ section (not tests!). All the properties you mention are included (mappings below):
a. Number of layers: properties/contiguousCodestreamBox/cod/layers
b. Number of levels: properties/contiguousCodestreamBox/cod/levels
c. Lossy v lossless: properties/contiguousCodestreamBox/cod/transformation (9-7 irreversible = lossy; 5-3 reversible=lossless)
d. Tiling: properties/contiguousCodestreamBox/siz/xTsiz and properties/contiguousCodestreamBox/siz/yTsiz (horizontal and vertical tile size)
e. Progression order: properties/contiguousCodestreamBox/cod/order
- Yes. You have to make the schema yourself though (haven’t tried this myself yet and I know next to nothing of XML schemas, but I think this should be pretty straightforward)
- Yes
- Yes
Johan
From: Henshaw, Christy
Sent: 21 December 2011 12:53
To: Johan van der Knijff
Cc: Thompson, Dave
Subject: RE: JP2 validator / properties extractor
Hi Johan,
Okay, I’ve read everything now, I think (hope). Before I try anything, can you let me know if I understand this right:
We could implemente the jpylyzer in this way:
Run the jpylyzer on jp2 images and look for <isValidJP2>
Where this is “no” we stop and evaluate our JP2s
Where this is “yes” we look at specific properties in the <tests> section that we need to conform to a profile (are these properties included?)
Number of layers
Number of levels
Lossy v lossless (?)
Tiling
Progression order
These properties are compared against the appropriate profile schema
Where the properties deviate we stop and investigate, re-convert images as required
Where the properties validate we carry on with ingest.
Does this make sense?
If so, we need to test that the tool will give us the parameters in point 3 in such a way that we can validate them – once you’ve settled on an output I guess we can set up some kind of validation script for the XML outputs. Hopefully all this can be automated by our workflow system, Goobi. So Goobi can trigger jpylyzer once images are in the right state/location, trigger profile comparison, and flag any errors to a human.
Christy