Version 3 by Erica Yang
on Aug 15, 2012 13:52.

compared with
Current by Erica Yang
on Aug 15, 2012 15:12.

This line was removed.
This word was removed. This word was added.
This line was added.

Changes (21)

View Page History
h2. NETWORK infrastructure

The CONTROLLER and the NODEs are connected to a 1GBit high performance network switch (guarantees the full GBit performance for each port) to the STFC internal network infrastructure.
The head nodes and the slaves are connected to the Tier1's core network at 1Gbps, they are connected to several switch stacks with 80Gbps backplanes.

Since all machines are connected to ports on the same physical network switch, the traffic between the cluster machines does not affect the traffic on the STFC network.
The switch stacks are in turn connected by 20Gbps backbone links to a core non-blocking switch.

h2. NameNode and JobTracker

Total disk capacity: 70TBs
  Total disk capacity: 70+TBs
  CPU: 2 x Xeon E5130@2.00GHz Dualcore CPU ( 4 cores )
  RAM: 8GB
  NIC: 2 x GbE (1 used)
  DISK: 1 x 250GB DISK

=== Yet to confirm
h2. Slaves 1 - 4

  Supermicro/Boston custom storage solution
  CPU: 2 x Dual-Core AMD Opteron(tm) Processor 2220
  RAM: 8GB
  NIC: 2 x GbE (1 used)
  DISK: 16 x 750GB disks as JBOD

Dell Poweredge R510
CPU: 2 x Xeon E5620@2.40GHz Quadcore CPU (16 HyperThreading cores)
NIC: 2 x GBit Ethernet (1 used)
DISK: 3 x 1TB DISKs; configured as RAID5 (redundancy) => 2TB effective disk space
h2. Slaves 5 - 6

h2. NODE (worker nodes 1 to 6)
Supermicro/Boston custom storage solution

=== Yet to confirm

Dell Poweredge R310
CPU: 1 x Xeon X3440@2.53GHz Quadcore CPU (8 HyperThreading cores)
CPU: 2 x  Intel(R) Xeon(R) CPU E5405 @ 2.00GHz ( 8 cores )
NIC: 2 x GBit Ethernet GbE (1 used)
DISK: 2 x 1TB DISKs; configured as RAID0 (performance) => 2TB effective disk space

Comment: Some hadoop documentation recommends JBOD (Just a Bunch Of Disks) instead of RAID0 (stripe set). Unfortunately the RAID controller in the R310 does not offer JBOD functionality. If that impacts performance at all, the impact might be very, very small in our usage scenario.
DISK: 24 x 1TB disks as JBOD