Skip to end of metadata
Go to start of metadata

Investigator(s)

Frank Asseg, frank.asseg(at)fiz-karlsruhe.de
Matthias Hahn, matthias.hahn(at)fit-karlsruhe.de

Dataset

The Dataset used was a random set with different file sizes

Platform

We tested on diverse Cluster at the Steinbuch Center for Computing SCC, on a Amazon AWS instance, on a local Cluster at FIZ Karlsruhe and on a Cluster at Timisoara (a SCAPE Partner)

Workflow

We first deployed Fedora 4 in clustered mode on the Cluster Nodes with the help of Shell- and Puppet scripts to do this efficiently. We used a benchtool to measure the ingest performance of the datasets.

Evaluation Summaries

Links to results of the experiment using the evaluation template.

Experiment Fedora 4 at SCC Cluster: https://wiki.duraspace.org/display/FF/Performance+evaluation+on+the+SCC+Cluster

Experiment Fedora 4 at AWS Cluster: https://wiki.duraspace.org/display/FF/Performance+evaluation+on+AWS

Experiment Fedora 4 at FIZ Cluster: https://wiki.duraspace.org/display/FF/Performance+evaluation+on+the+FIZ+cluster

Experiment Fedora 4 at Timisoar Cluster: for this test we do not have a Wiki Page at Duraspace, instead we provide the numbers here:

The cluster is currently running on these three nodes:
   http://scape-fiz-1.info.uvt.ro:8080/fcrepo/rest
   http://scape-fiz-3.info.uvt.ro:8080/fcrepo/rest
   http://scape-fiz-4.info.uvt.ro:8080/fcrepo/rest

The first small ingest test of a single file with 10m xielded this result:
17:15  INFO  Found Fedora 4 at http://localhost:8080/fcrepo
16:17:15  INFO  Running 1 INGEST action(s) against FCREPO4 with a binary size of 10.0 MB using 1 thread(s)
16:17:16  INFO  The Fedora cluster has 3 node(s) before the benchmark
16:17:16  INFO  preparing 1 objects
16:17:16  INFO  creating 1 objects took 57 ms
16:37:16  INFO  scheduling 1 actions
17:08:55  INFO  purging 1 objects and datastreams
17:08:55  INFO  Completed 1 INGEST action(s) executed in 1898784 ms
17:08:55  INFO  The Fedora cluster has 3 node(s) after the benchmark
17:08:55  INFO  Throughput was 0.01 MB/sec
17:08:55  INFO  Condensed results:
17:08:55  INFO  1 10485760 1 INGEST 1898784 0.0052665286 no-tx
17:08:55  INFO  All operations completed in 3099875 ms

So it did work but the throughput with one file was only 0.1mb/s.
Running a larger test ingesting 100 Objects with 10mb each gives the same result.

The profile of the cluster is available here:
https://wiki.duraspace.org/pages/viewpage.action?pageId=50528479

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.