Frank Asseg, frank.asseg(at)fiz-karlsruhe.de
Matthias Hahn, matthias.hahn(at)fit-karlsruhe.de
The Dataset used was a random set with different file sizes
We tested on diverse Cluster at the Steinbuch Center for Computing SCC, on a Amazon AWS instance, on a local Cluster at FIZ Karlsruhe and on a Cluster at Timisoara (a SCAPE Partner)
We first deployed Fedora 4 in clustered mode on the Cluster Nodes with the help of Shell- and Puppet scripts to do this efficiently. We used a benchtool to measure the ingest performance of the datasets.
Links to results of the experiment using the evaluation template.
Experiment Fedora 4 at SCC Cluster: https://wiki.duraspace.org/display/FF/Performance+evaluation+on+the+SCC+Cluster
Experiment Fedora 4 at AWS Cluster: https://wiki.duraspace.org/display/FF/Performance+evaluation+on+AWS
Experiment Fedora 4 at FIZ Cluster: https://wiki.duraspace.org/display/FF/Performance+evaluation+on+the+FIZ+cluster
Experiment Fedora 4 at Timisoar Cluster: for this test we do not have a Wiki Page at Duraspace, instead we provide the numbers here:
The first small ingest test of a single file with 10m xielded this result:
17:15 INFO Found Fedora 4 at http://localhost:8080/fcrepo
16:17:15 INFO Running 1 INGEST action(s) against FCREPO4 with a binary size of 10.0 MB using 1 thread(s)
16:17:16 INFO The Fedora cluster has 3 node(s) before the benchmark
16:17:16 INFO preparing 1 objects
16:17:16 INFO creating 1 objects took 57 ms
16:37:16 INFO scheduling 1 actions
17:08:55 INFO purging 1 objects and datastreams
17:08:55 INFO Completed 1 INGEST action(s) executed in 1898784 ms
17:08:55 INFO The Fedora cluster has 3 node(s) after the benchmark
17:08:55 INFO Throughput was 0.01 MB/sec
17:08:55 INFO Condensed results:
17:08:55 INFO 1 10485760 1 INGEST 1898784 0.0052665286 no-tx
17:08:55 INFO All operations completed in 3099875 ms
So it did work but the throughput with one file was only 0.1mb/s.
Running a larger test ingesting 100 Objects with 10mb each gives the same result.
The profile of the cluster is available here: