h1. Evaluation specs platform/system level
|| Field \\ || Datatype \\ || Value \\ || Description \\ ||
| Evaluation seq. num. \\ | int \\ | 1 \\ | Use only of sub-sequent evaluations of the same evaluation is done by another person than a previous one. \\
In that case copy the Evaluation specs table and fill out a new one with a new sequence number. \\
For the first evaluation leave this field at "1" \\ |
| Evaluator-ID | email | [email protected] \\ | Unique ID of the evaluator that carried out this specific evaluator. \\ |
| Evaluation describtion | text | Evaluating overall performance of running FITS on arbitrary web content \\ | Textual description of the evaluation and the overall goals \\ |
| Evaluation-Date | DD/MM/YY | 19/07/2012 \\ | Date of evaluation \\ |
| Platform-ID | string \\ | IMF 1 \\ | Unique ID of the platform involved in the particular evaluation - see Platform page included below \\ |
| Dataset(s) | string \\ | [Statsbiblioteket Web Archive Collection|SP:State and University Library Denmark - Web Archive Data]\\ | Link to dataset page(s) on WIKI \\
For each dataset that is a part of an evaluation \\
make sure that the dataset is described here: [SP:Datasets]\\ | |
| Workflow method | string \\ | Commandline \\ | Taverna / Commandline / Direct hadoop etc... \\ | |
| Workflow(s) involved \\ | URL(s) \\ | N/A \\ | Link(s) to MyExperiment *if applicable* \\ |
| Tool(s) involved \\ | URL(s) | FITS-0.6.0 | Link(s) to distinct versions of specific components/tools in the component registry *if applicable* \\ |
| Link(s) to Scenario(s) | URL(s) \\ | [WCT4|SP:WCT4 Web Archive Mime-Type detection at Austrian National Library]\\ | Link(s) to scenario(s) *if applicable* \\ |
\\
{include:Platform IMF 1}
h1. Evaluation points
metrics must come from / be registered in the [metrics catalogue|Metrics Catalogue]
|| Metric || Baseline definition || Baseline value \\ || Goal || Evaluation 1 (19/07/2012) \\ || Evaluation 2 (15/12/2012) \\ || Evaluation 3 (20/10/2013) \\ ||
| NumberOfObjectsPerHour | On a single machine from commandline \\ | 3000 | 50000 \\ | 15000 \\ | 30000 \\ | 75000 \\ |
| | | | | | | |
| | | | | | | |
|| Field \\ || Datatype \\ || Value \\ || Description \\ ||
| Evaluation seq. num. \\ | int \\ | 1 \\ | Use only of sub-sequent evaluations of the same evaluation is done by another person than a previous one. \\
In that case copy the Evaluation specs table and fill out a new one with a new sequence number. \\
For the first evaluation leave this field at "1" \\ |
| Evaluator-ID | email | [email protected] \\ | Unique ID of the evaluator that carried out this specific evaluator. \\ |
| Evaluation describtion | text | Evaluating overall performance of running FITS on arbitrary web content \\ | Textual description of the evaluation and the overall goals \\ |
| Evaluation-Date | DD/MM/YY | 19/07/2012 \\ | Date of evaluation \\ |
| Platform-ID | string \\ | IMF 1 \\ | Unique ID of the platform involved in the particular evaluation - see Platform page included below \\ |
| Dataset(s) | string \\ | [Statsbiblioteket Web Archive Collection|SP:State and University Library Denmark - Web Archive Data]\\ | Link to dataset page(s) on WIKI \\
For each dataset that is a part of an evaluation \\
make sure that the dataset is described here: [SP:Datasets]\\ | |
| Workflow method | string \\ | Commandline \\ | Taverna / Commandline / Direct hadoop etc... \\ | |
| Workflow(s) involved \\ | URL(s) \\ | N/A \\ | Link(s) to MyExperiment *if applicable* \\ |
| Tool(s) involved \\ | URL(s) | FITS-0.6.0 | Link(s) to distinct versions of specific components/tools in the component registry *if applicable* \\ |
| Link(s) to Scenario(s) | URL(s) \\ | [WCT4|SP:WCT4 Web Archive Mime-Type detection at Austrian National Library]\\ | Link(s) to scenario(s) *if applicable* \\ |
\\
{include:Platform IMF 1}
h1. Evaluation points
metrics must come from / be registered in the [metrics catalogue|Metrics Catalogue]
|| Metric || Baseline definition || Baseline value \\ || Goal || Evaluation 1 (19/07/2012) \\ || Evaluation 2 (15/12/2012) \\ || Evaluation 3 (20/10/2013) \\ ||
| NumberOfObjectsPerHour | On a single machine from commandline \\ | 3000 | 50000 \\ | 15000 \\ | 30000 \\ | 75000 \\ |
| | | | | | | |
| | | | | | | |