Skip to end of metadata
Go to start of metadata
You are viewing an old version of this page. View the current version. Compare with Current  |   View Page History

Basic cluster software

Scientific Linux 2.6.18-238.9.1.el5 #1 SMP (Symmetric Multi Processors) Tue Apr 12 18:53:46 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

OpenSSH as part of the OS installation

Hadoop 0.20.2
Subversion  -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14

Cloudera CDH3u0

java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

Cluster hardware

NETWORK infrastructure

The CONTROLLER and the NODEs are connected to a 1GBit high performance network switch (guarantees the full GBit performance for each port) to the STFC internal network infrastructure.

Since all machines are connected to ports on the same physical network switch, the traffic between the cluster machines does not affect the traffic on the STFC network.

CONTROLLER

Total disk capacity: 70TBs

=== Yet to confirm

Dell Poweredge R510
CPU: 2 x Xeon [email protected] Quadcore CPU (16 HyperThreading cores)
RAM: 24GB
NIC: 2 x GBit Ethernet (1 used)
DISK: 3 x 1TB DISKs; configured as RAID5 (redundancy) => 2TB effective disk space

NODE (worker nodes 1 to 5)

=== Yet to confirm

Dell Poweredge R310
CPU: 1 x Xeon [email protected] Quadcore CPU (8 HyperThreading cores)
RAM: 16GB
NIC: 2 x GBit Ethernet (1 used)
DISK: 2 x 1TB DISKs; configured as RAID0 (performance) => 2TB effective disk space

Comment: Some hadoop documentation recommends JBOD (Just a Bunch Of Disks) instead of RAID0 (stripe set). Unfortunately the RAID controller in the R310 does not offer JBOD functionality. If that impacts performance at all, the impact might be very, very small in our usage scenario.

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.