You are viewing an old version of this page. View the current version.
Compare with Current |
View Page History
| Title | SO12 Tool testing framework |
| Detailed description | A testing framework, to ease the evaluation and measurement of tools. Furthermore, it allows us to test the effects of new versions and modifications |
| Solution Champion |
Asger Askov Blekinge (SB) |
| Corresponding Issue(s) |
|
| myExperiment Link |
|
| Tool Registry Link |
|
| Evaluation |
|
Currently, the framework consist of two scripts for running the tools recursively
Invoking - Tools that read the datafile themselves
Datadir is the directory where the datafiles reside.
Report dir is where the output reports of the tool should be dumped. Each file will produce an output file, so the directory structure in REPORTDIR will come to resemble the one in DATADIR.
PreCommand is the bash command the place before the file. Postcommand is the bit to place afterwards.
To run Tika (java -jar tika-app-1.0.jar <datafile>) on every file in the folder ~/testdata and have it report to ~/testdatareports, this would be the command
Since performance information is relevant, this is recorded in a CSV file in the REPORTDIR by the name of time.log
Piping - Tools that read stdin
Functions almost like recursive_invoke.sh
To run Tika (cat <datafile> | java -jar tika-app-1.0.jar -) on every file in the folder ~/testdata and have it report to ~/testdatareports, this would be the command
Since performance information is relevant, this is recorded in a CSV file in the REPORTDIR by the name of time.log