If a certain experiment results in the conclusion that, using a certain migration tool, the height of the migrated image is “quite similar” to the height of the original image, when migrating an image from a .bmp to a .jpeg-file, the assessment “quite similar” should be objectively measurable. It should be avoided that, for instance, institution A rates the quality of migration “quite similar” and institution rates it “very different”, because they use different ways of evaluation.
For instance, imagine a text object with font size 12pt. A migration tool X migrates this document to another file format, and in this process changes the font size to 11pt. Institution A concludes that the image heights of the original and migrated image are “quite similar” – ideally, this comparing and assessing a measure of similarity is done automatically. Institution B agrees with this assessment: the Tested result is usable for B, too. However, when it comes to deciding which tools are usable in practice, for institution A migration tool X might be good enough – because the text only has to be readable – but this might be totally unacceptable for institution B. In other words, A and B will assign a different importance to the exact preservation of the font size.
The institution specific information (e.g.: how bad is it if the font size has changed but the document is still readable) can be introduced in the form of weigh factors, when compiling a preservation plan that states when to deploy which tools, depending on the institution-specific characteristics.
However, as explained above, this last step is not part of the Testbed application. Testbed results only have to do with the performance of tools on different types of objects, but leave out any institution specific factors.