Evaluation specs platform/system level
Field |
Datatype |
Value |
Description |
|
---|---|---|---|---|
Evaluation seq. num. |
int |
1 |
Use only of sub-sequent evaluations of the same evaluation is done in another setup than a previous one. In that case copy the Evaluation specs table and fill out a new one with a new sequence number. For the first evaluation leave this field at "1" |
|
Evaluator-ID | [email protected] | Unique ID of the evaluator that carried out this specific evaluator. |
||
Evaluation describtion | text | Since November 2011 we have been running FITS on a selection of our web content spread over the years from 2005 up till 2011. The data is stored in ARC files on a SAN. These ARC files are fetched from this SAN, unpacked and the FITS are run on each ARC record. Running FITS on a ARC record produces an XML file. These XML files from a single ARC are packed into TGZ files and made available to the Planning and Watch subproject. To evaluate this job we extract information on the timing of the FITS jobs together with information from the ARC files. |
Textual description of the evaluation and the overall goals |
|
Evaluation-Date | DD/MM/YY | 25th of November 2011 till 8th of November 2012 | Date of evaluation |
|
Platform-ID | string |
SB Test Platform | Unique ID of the platform involved in the particular evaluation - see Platform page included below |
|
Dataset(s) | string |
http://wiki.opf-labs.org/display/SP/State+and+University+Library+Denmark+-+Web+Archive+Data![]() |
Link to dataset page(s) on WIKI For each dataset that is a part of an evaluation make sure that the dataset is described here: Datasets |
|
Workflow method | string |
Commandline | Taverna / Commandline / Direct hadoop etc… |
|
Workflow(s) involved |
URL(s) |
None | Link(s) to MyExperiment if applicable |
|
Tool(s) involved |
URL(s) | fits 0.6.0, arc-unpacker 0.2![]() |
Link(s) to distinct versions of specific components/tools in the component registry if applicable |
|
Link(s) to Scenario(s) | URL(s) |
WCT3![]() |
Link(s) to scenario(s) if applicable |
SB Test Platform
Field |
Datatype |
Value |
Description |
---|---|---|---|
Platform-ID | String | Platform SB 1 | Unique string that identifies this specific platform. Use the platform name |
Platform description | String | We have five Blade servers located at SB | Human readable description of the platform. Where is it located, contact info, etc. |
Number of nodes | integer | 5 physical servers | Number of hosts involved - could be both physical hosts as well as virtual hosts |
Total number of physical CPUs | integer | 10 | Number of CPU's involved |
CPU specs | string | Intel® Xeon® Processor X5670 (12M Cache, 2.93 GHz, 6.40 GT/s Intel® QPI) |
Specification of CPUs |
Total number of CPU-cores | integer | 60 | Number of CPU-cores involved |
Total amount of RAM in Gbytes |
integer | 288 GB | Total amount of RAM on all nodes |
average CPU-cores for nodes |
integer | 6 | Number of CPU-cores in average across all nodes |
avarage RAM in Gbytes for nodes |
integer | 96 GB | Amount of memory in average across all nodes |
Operating System on nodes |
String | Red Hat based Linux | Linux (specific distribution), Windows (specific distribution), other? |
Storage system/layer | String | SAN and EMC Isilon | NFS & HDFS |
Network layer between nodes | String | 2 GB ethernet | Speed of network interfaces, general network speed |
|
|
Evaluation points
metrics must come from / be registered in the metrics catalogue
The motivation behind the goal is as follows: we want to be able to run a FITS-like characterisation on a complete snap-shot of the Danish TLD within weeks. Such a snap-shot harvest amounts to 25 TB. This gives a throughput in the order of 1GB/minute. "FITS-like" is here defined as a characterisation using multiple tools combined with a comparison of the output of these tools.
Even though the base line is calculated based on one thread on one CPU, we did the actual assessment on a five machine cluster where each process was allowed to use up to 4 threads. This experiment is our first evaluation.
Metric | Baseline definition | Baseline value | Goal | Evaluation 1 (8/11 2012) |
Evaluation 2 (date) |
Evaluation 3 (date) |
---|---|---|---|---|---|---|
ThroughputGbytesPerHour | Measurement of the running time of the FITS jobs assuming one thread on one machine. During the last year the job has actual run on one to five servers using one to four threads but that job distribution is not represented in the metadata. | 0.162 | 60 | 1.32 |
||
OrganisationalFit |
N/A |
true |
true |
|||