Skip to end of metadata
Go to start of metadata

Evaluation specs component level

Field
Datatype
Value
Description
Evaluation seq. num.
int
1
Use only of sub-sequent evaluations of the same evaluation is done in another setup than a previous one.
In that case copy the Evaluation specs table and fill out a new one with a new sequence number.
For the first evaluation leave this field at "1"
Evaluator-ID email bja@statsbiblioteket.dk
Unique ID of the evaluator that carried out this specific evaluator.
Evaluation describtion text Evaluating correctness of mimetype identification of Web Content on a controlled annotated corpora
Textual description of the evaluation and the overall goals
Evaluation-Date DD/MM/YY 20/07/2012
Date of evaluation
Dataset(s) string
Govdocs1 Open Corpus
Link to dataset page(s) on WIKI
For each dataset that is a part of an evaluation
make sure that the dataset is described here: Datasets
 
Workflow method string
commandline
Taverna / Commandline / Direct hadoop etc...
 
Workflow(s) involved
URL(s)
N/A
Link(s) to MyExperiment if applicable
Tool(s) involved
URL(s) Apache TIKA 0.7 Link(s) to distinct versions of specific components/tools in the component registry if applicable
Link(s) to Scenario(s) URL(s)
WCT4
Link(s) to scenario(s) if applicable
Link(s) to evaluation blog
URL(s)   Link(s) to evaluation blog if applicable

Technical setup

Field
Datatype
Value
Description
Description String iapetus.statsbiblioteket.dk
Human readable description of the "platform" - e.g. Bjarnes Linux PC
Total number of physical CPUs integer 2
Number of CPU's involved
CPU specs string Intel® Xeon® Processor X5670 
(12M Cache, 2.93 GHz, 6.40 GT/s Intel® QPI)
Specification of CPUs
Total number of CPU-cores integer 12
Number of CPU-cores involved
Total amount of RAM in Gbytes
integer 96
Total amount of RAM on all nodes
Operating System
String Linux
Linux (specific distribution), Windows (specific distribution), other?
Storage system/layer String NFS mounted files
NFS, HDFS, local files, ?
       

   

Evaluation points

metrics must come from / be registered in the metrics catalogue

Metric Baseline definition
Baseline value Goal Evaluation 1 (20/07/2012)
Evaluation 2 (10/06/2013)
Evaluation 3 (01/03/2014)
IdentificationCorrectnessInPercent
Whats possible with Jhove2 default distribution
85 %
99 %
94 %
97 %
99 %
             
             
Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.