Evaluation specs platform/system level
| Field |
Datatype |
Value |
Description |
|
|---|---|---|---|---|
| Evaluation seq. num. |
int |
1 |
Use only of sub-sequent evaluations of the same evaluation is done by another person than a previous one. In that case copy the Evaluation specs table and fill out a new one with a new sequence number. For the first evaluation leave this field at "1" |
|
| Evaluator-ID | [email protected] |
Unique ID of the evaluator that carried out this specific evaluator. |
||
| Evaluation describtion | text | Evaluating overall performance of running FITS on arbitrary web content |
Textual description of the evaluation and the overall goals |
|
| Evaluation-Date | DD/MM/YY | 19/07/2012 |
Date of evaluation |
|
| Platform-ID | string |
IMF 1 |
Unique ID of the platform involved in the particular evaluation - see Platform page included below |
|
| Dataset(s) | string |
Statsbiblioteket Web Archive Collection |
Link to dataset page(s) on WIKI For each dataset that is a part of an evaluation make sure that the dataset is described here: Datasets |
|
| Workflow method | string |
Commandline |
Taverna / Commandline / Direct hadoop etc... |
|
| Workflow(s) involved |
URL(s) |
N/A |
Link(s) to MyExperiment if applicable |
|
| Tool(s) involved |
URL(s) | FITS-0.6.0 | Link(s) to distinct versions of specific components/tools in the component registry if applicable |
|
| Link(s) to Scenario(s) | URL(s) |
WCT4 |
Link(s) to scenario(s) if applicable |
Platform IMF 1
| Field |
Datatype |
Value |
Description |
|---|---|---|---|
| Platform-ID | String | IMF Cluster | Unique string that identifies this specific platform. Use the platform name |
| Platform description | String | Cloudera CDH3u2. 3 dual-core low consumption nodes |
Human readable description of the platform. Where is it located, contact info, etc. |
| Number of nodes | integer | 3 |
Number of hosts involved - could be both physical hosts as well as virtual hosts |
| Total number of physical CPUs | integer | 3 |
Number of CPU's involved |
| CPU specs | string | Dual core AMD G-T56N on 1600MHz | Specification of CPUs |
| Total number of CPU-cores | integer | 6 Cores (3 * 2 Cores) | Number of CPU-cores involved |
| Total amount of RAM in Gbytes |
integer | 24GB (3 * 8GB) | Total amount of RAM on all nodes |
| average CPU-cores for nodes |
integer | 2 |
Number of CPU-cores in average across all nodes |
| avarage RAM in Gbytes for nodes |
integer | 8 |
Amount of memory in average across all nodes |
| Operating System on nodes |
String | Debian 6 squeeze (64bit) | Linux (specific distribution), Windows (specific distribution), other? |
| Storage system/layer | String | HDFS | NFS, HDFS, local files, ? |
| Network layer between nodes | String | Local copy between two nodes : 80 MB/s 640 Mbps | Speed of network interfaces, general network speed |
| |
|
Evaluation points
metrics must come from / be registered in the metrics catalogue
| Metric | Baseline definition | Baseline value |
Goal | Evaluation 1 (19/07/2012) |
Evaluation 2 (15/12/2012) |
Evaluation 3 (20/10/2013) |
|---|---|---|---|---|---|---|
| NumberOfObjectsPerHour | On a single machine from commandline |
3000 | 50000 |
15000 |
30000 |
75000 |
Labels:
None