Evaluation specs platform/system level

Field	Datatype	Value	Description
Evaluation seq. num.	int	1	Use only of sub-sequent evaluations of the same evaluation is done in another setup than a previous one. In that case copy the Evaluation specs table and fill out a new one with a new sequence number. For the first evaluation leave this field at "1"
Evaluator-ID	email	[email protected]	Unique ID of the evaluator that carried out this specific evaluator.
Evaluation describtion	text	Since November 2011 we have been running FITS on a selection of our web content spread over the years from 2005 up till 2011. The data is stored in ARC files on a SAN. These ARC files are fetched from this SAN, unpacked and the FITS are run on each ARC record. Running FITS on a ARC record produces an XML file. These XML files from a single ARC are packed into TGZ files and made available to the Planning and Watch subproject. To evaluate this job we extract information on the timing of the FITS jobs together with information from the ARC files.	Textual description of the evaluation and the overall goals
Evaluation-Date	DD/MM/YY	25th of November 2011 till 8th of November 2012	Date of evaluation
Platform-ID	string	SB Test Platform	Unique ID of the platform involved in the particular evaluation - see Platform page included below
Dataset(s)	string	http://wiki.opf-labs.org/display/SP/State+and+University+Library+Denmark+-+Web+Archive+Data	Link to dataset page(s) on WIKI For each dataset that is a part of an evaluation make sure that the dataset is described here: Datasets
Workflow method	string	Commandline	Taverna / Commandline / Direct hadoop etc…
Workflow(s) involved	URL(s)	None	Link(s) to MyExperiment if applicable
Tool(s) involved	URL(s)	fits 0.6.0, arc-unpacker 0.2	Link(s) to distinct versions of specific components/tools in the component registry if applicable
Link(s) to Scenario(s)	URL(s)	WCT3	Link(s) to scenario(s) if applicable

SB Test Platform

Evaluation points

metrics must come from / be registered in the metrics catalogue

The motivation behind the goal is as follows: we want to be able to run a FITS-like characterisation on a complete snap-shot of the Danish TLD within weeks. Such a snap-shot harvest amounts to 25 TB. This gives a throughput in the order of 1GB/minute. "FITS-like" is here defined as a characterisation using multiple tools combined with a comparison of the output of these tools.

Even though the base line is calculated based on one thread on one CPU, we did the actual assessment on a five machine cluster where each process was allowed to use up to 4 threads. This experiment is our first evaluation.