Skip to end of metadata
Go to start of metadata
IS18 Verify bitstream integrity
Detailed description Problem: bit rot occurs as a result of the (gradual) deterioration of storage media, leading to damage to digital file objects at the bitstream level. Depending on the extent and position of the damaged bits, files affected by bit rot may end up being completely unreadable. Unlike format obsolescence (where a file cannot be read because no readers are available for the used file format), bit rot results in a loss of information that is irreversible. If multiple copies of an object exist, data losses due to bit rot can be recovered by using the intact copies. However, this requires that archived materials are continuously monitored for signs of bit rot (preservation watch), and that affected materials are recovered upon detection (action services).
Scalability Challenge
Solution should be applicable to very large volumes of data
Issue champion Johan van der Knijff (KB)
Other interested parties
STFC created an issue that is very simuilar to this one: IS30 Fixity capturing and checking of very large data files. Maybe these issues should be merged.
Possible Solution approaches The occurrence of bit rot can be monitored by regular checksum checks.  Many well-established and stable tools exist for this. The challenge of this issue mainly lies in the effective and efficient deployment of such tools for (very) large digital collections. The scope of this issue may also include guidelines on backup strategies that eliminate the risk of files that are affected by bit rot being unintentionally transferred to backups (thereby overwriting the original, undamaged objects).
Context Details of the institutional context to the Issue. (May be expanded at a later date)
Lessons Learned Notes on Lessons Learned from tackling this Issue that might be useful to inform the development of Future Additional Best Practices, Task 8 (SCAPE TU.WP.1 Dissemination and Promotion of Best Practices)
Training Needs Is there a need for providing training for the Solution(s) associated with this Issue? Notes added here will provide guidance to the SCAPE TU.WP.3 Sustainability WP.
Datasets To be confirmed.
Solutions Reference to the appropriate Solution page(s), by hyperlink


Objectives Reliability, scalability, robustness
Success criteria Describe the success criteria for solving this issue - what are you able to do? - what does the world look like?
Automatic measures Process ## files per second / process ## MB per second
Manual assessment Apart from automated measures that you would like to get do you foresee any necessary manual assessment to evaluate the solution of this issue?
If possible specify measures and your goal - e.g.
 * Solution installable with basic linux system administration skills
 * User interface understandable by non developer curators
Actual evaluations links to acutual evaluations of this Issue/Scenario
lsdr lsdr Delete
fixity fixity Delete
issue issue Delete
integrity integrity Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.