Skip to end of metadata
Go to start of metadata

Evaluation specs platform/system level

Field
Datatype
Value
Description
Evaluation seq. num.
int
1
Use only of sub-sequent evaluations of the same evaluation is done by another person than a previous one.
In that case copy the Evaluation specs table and fill out a new one with a new sequence number.
For the first evaluation leave this field at "1"
Evaluator-ID email bja@statsbiblitoeket.dk
Unique ID of the evaluator that carried out this specific evaluator.
Evaluation describtion text Evaluating overall performance of running FITS on arbitrary web content
Textual description of the evaluation and the overall goals
Evaluation-Date DD/MM/YY 19/07/2012
Date of evaluation
Platform-ID string
IMF 1
Unique ID of the platform involved in the particular evaluation - see Platform page included below
Dataset(s) string
Statsbiblioteket Web Archive Collection
Link to dataset page(s) on WIKI
For each dataset that is a part of an evaluation
make sure that the dataset is described here: Datasets
 
Workflow method string
Commandline
Taverna / Commandline / Direct hadoop etc...
 
Workflow(s) involved
URL(s)
N/A
Link(s) to MyExperiment if applicable
SCAPE Tools/API URL(s) or text   Link(s) to distinct versions of specific components/tools in the component registry if applicable
Tool(s) involved
URL(s) FITS-0.6.0 Link(s) to distinct versions of specific components/tools in the component registry if applicable
Link(s) to Scenario(s) URL(s)
WCT4
Link(s) to scenario(s) if applicable
       


Platform IMF 1

Field
Datatype
Value
Description
Platform-ID String IMF Cluster Unique string that identifies this specific platform.
Use the platform name
Platform description String Cloudera CDH3u2.
3 dual-core low consumption nodes
Human readable description of the platform. Where is it located, contact info, etc.
Number of nodes integer 3
Number of hosts involved - could be both physical hosts as well as virtual hosts
Total number of physical CPUs integer 3
Number of CPU's involved
CPU specs string Dual core AMD G-T56N on 1600MHz Specification of CPUs
Total number of CPU-cores integer 6 Cores (3 * 2 Cores) Number of CPU-cores involved
Total amount of RAM in Gbytes
integer 24GB (3 * 8GB) Total amount of RAM on all nodes
average CPU-cores for nodes
integer 2
Number of CPU-cores in average across all nodes
avarage RAM in Gbytes for nodes
integer 8
Amount of memory in average across all nodes
Operating System on nodes
String Debian 6 squeeze (64bit) Linux (specific distribution), Windows (specific distribution), other?
Storage system/layer String HDFS NFS, HDFS, local files, ?
Network layer between nodes String Local copy between two nodes : 80 MB/s 640 Mbps Speed of network interfaces, general network speed
       

   

Evaluation points

metrics must come from / be registered in the metrics catalogue

Metric Baseline definition Baseline value
Goal Evaluation 1 (19/07/2012)
Evaluation 2 (15/12/2012)
Evaluation 3 (20/10/2013)
NumberOfObjectsPerHour On a single machine from commandline
3000 50000
15000
30000
75000
             
             

Evaluation notes for SCAPE tools

Field note
Issues idenitifed  
Workarounds  
Data preparations  
Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.