h1.
h1. SB Hadoop Platform
|| Field \\ || Datatype \\ || Value \\ || Description \\ ||
| Platform-ID | String | Platform SB Hadoop | Unique string that identifies this specific platform. \\
Use the platform name |
| Platform description | String | We have five Blade servers located at SB | Human readable description of the platform. Where is it located, contact info, etc. |
| Number of nodes | integer | 4 physical servers | Number of hosts involved - could be both physical hosts as well as virtual hosts \\ |
| Total number of physical CPUs | integer | 8 | Number of CPU's involved \\ |
| CPU specs | string | Intel® Xeon® Processor X5670 \\
(12M Cache, 2.93 GHz, 6.40 GT/s Intel® QPI) | Specification of CPUs \\ |
| Total number of CPU-cores | integer | 48 | Number of CPU-cores involved \\
Each physical CPU consists of 6 cores, and 12 threads \\ |
| Total amount of RAM in Gbytes \\ | integer | 384 GB | Total amount of RAM on all nodes \\ |
| -average- CPU-cores for nodes \\ | integer | 12 | Number of CPU-cores in average across all nodes \\ |
| -average- RAM in Gbytes for nodes \\ | integer | 96 GB | Amount of memory in average across all nodes \\ |
| Operating System on nodes \\ | String | Red Hat based Linux | Linux (specific distribution), Windows (specific distribution), other? |
| Storage system/layer | String | mounted NFS | NFS, HDFS, local files, ? |
| Network layer between nodes | String | 2 GB ethernet | Speed of network interfaces, general network speed |
h1. Parallel Execution System
|| Field || Value ||
| Installation description | Cloudera CDH4, Hadoop 2.0.0-cdh4.5.0 \\ |
| Configuration notes | Replicationfactor is n/a. Using HDFS protocol for EMC Isilon Storage System \\
1 x Namenode (JobTracker) \\
4 x Datanode (TaskTracker) |
h1. Benchmark of Parallel Execution System
|| Metric || Benchmark type || Value || Comment ||
| Concurrent write throughput | TestDFSIO \\ | 2.6 Gb/s | |
| Concurrent read throughput \\ | TestDFSIO \\ | 4.4 Gb/s | |
| Execution time | TeraSort of one billion 100 bytes index's (100GB) | 10,07 minutes | |
h1. SB Hadoop Platform
|| Field \\ || Datatype \\ || Value \\ || Description \\ ||
| Platform-ID | String | Platform SB Hadoop | Unique string that identifies this specific platform. \\
Use the platform name |
| Platform description | String | We have five Blade servers located at SB | Human readable description of the platform. Where is it located, contact info, etc. |
| Number of nodes | integer | 4 physical servers | Number of hosts involved - could be both physical hosts as well as virtual hosts \\ |
| Total number of physical CPUs | integer | 8 | Number of CPU's involved \\ |
| CPU specs | string | Intel® Xeon® Processor X5670 \\
(12M Cache, 2.93 GHz, 6.40 GT/s Intel® QPI) | Specification of CPUs \\ |
| Total number of CPU-cores | integer | 48 | Number of CPU-cores involved \\
Each physical CPU consists of 6 cores, and 12 threads \\ |
| Total amount of RAM in Gbytes \\ | integer | 384 GB | Total amount of RAM on all nodes \\ |
| -average- CPU-cores for nodes \\ | integer | 12 | Number of CPU-cores in average across all nodes \\ |
| -average- RAM in Gbytes for nodes \\ | integer | 96 GB | Amount of memory in average across all nodes \\ |
| Operating System on nodes \\ | String | Red Hat based Linux | Linux (specific distribution), Windows (specific distribution), other? |
| Storage system/layer | String | mounted NFS | NFS, HDFS, local files, ? |
| Network layer between nodes | String | 2 GB ethernet | Speed of network interfaces, general network speed |
h1. Parallel Execution System
|| Field || Value ||
| Installation description | Cloudera CDH4, Hadoop 2.0.0-cdh4.5.0 \\ |
| Configuration notes | Replicationfactor is n/a. Using HDFS protocol for EMC Isilon Storage System \\
1 x Namenode (JobTracker) \\
4 x Datanode (TaskTracker) |
h1. Benchmark of Parallel Execution System
|| Metric || Benchmark type || Value || Comment ||
| Concurrent write throughput | TestDFSIO \\ | 2.6 Gb/s | |
| Concurrent read throughput \\ | TestDFSIO \\ | 4.4 Gb/s | |
| Execution time | TeraSort of one billion 100 bytes index's (100GB) | 10,07 minutes | |