File format management utilities
- DROID (Digital Record Object Identification)
: Automated identification of file formats. - JHOVE (JSTOR/Harvard Object Validation Environment) & JHOVE2: Provides functions to perform format-specific identification, validation, and characterization of digital objects.
- XENA (XML Electronic Normalising for Archives): Detects the file formats of digital objects; converts digital objects into open formats for preservation. Part of NAA DSPS tool suite.
- Fido: A python implementation of the DROID identification system, using regular expressions.
- Fine Free File Command: Identifies a very wide range of files, although not focused on preservation issues.
- FITS: The File Information Tool Set (FITS) identifies, validates, and extracts technical metadata for various file formats. Aggregates results from other tools.
- NZNL Metadata Extraction Tool: See here for further information.
- JMimeMagic: Although not in development at present, this open source identifier is very close to 'file' in structure, and may provide a useful base for new developments.
File integrity utilities
- ACE (Audit Control Environment): Validates the integrity of digital files through mathematical techniques.
- Note that
Andrew Jackson uses ACE-AM to manage multi-Terabyte data prior to curation and later ingest into our digital library system.
- Note that
- Checksum Checker: Monitors the contents of a digital archive for data loss or corruption. Part of NAA DSPS tool suite.
- FastSum: Manifest creation and checking with a reasonably usable GUI.
- Manifest Maker: Supports the transfer of data objects by producing a manifest file which satisfies the requirements for a digital transfer. Part of NAA DSPS tool suite.
- JackSum: Support 58 popular standard algorithms. Integrated with File Browser.
File transfer utilities
- BagIt Library: Java software library that supports the creation, manipulation and validation of BagIt bags.
- BagIt Transfer Utilities: Collection of tools for validation and transfer of BagIt bags.
- GNU Wget: File transfer utility: Permits retrieval of files using HTTP, HTTPS and FTP
- rsync: Checksum-validated robust re-startable transfers of all sorts. Useful for cloning large collections reliably over a flaky connection.
Conversion utilities
- NCSA Polyglot : Distributed file format conversion service.
Labels:
None