Implementation

Directly derived from the initial technical specifications, a core system was built up since 1994 and is growing since. NICE consists of the following sub-systems:

• The Gigabit backbone network
• The storage servers, the RAID storage systems and their associated backup units
• The compute clusters
• The I/O workstations

The Gigabit backbone network

The traditional Ethernet network, with 10 Mbps shared by many nodes, was saturated already in the early 1990s. The ATM network helped to set up NICE, but it proved essential to find a migration path for all the remaining network to allow for almost unlimited and fast data transfer.

This is why we started in 1999 to implement 100bT switches with Gigabit interswitch communication, starting from the beamline backbone network, and gradually spreading to all the ESRF buildings. Since 2005, 1000bT switches are operating on the Beamlines, and large multi-10Gbe (10Gigabit) highways interconnect the core swiitches.The first private 10Gigabit link from a Beamline to the storage servers has been successfully tested in early 2008.

Core device in the main network room

The NAS storage systems and their associated backup units

The very first storage systems in NICE were RAID7 units from a small American company called Storage Computer. Their capacities varied from 100 to 320 GigaBytes per system. All RAID7 systems were de-commissioned in the course of 2000.

The next generation of storage systems were based on SUN multiprocessor computers. They provided an excellent load stability, and offered a high degree of flexibility. All SUN storage systems were de-commissioned by the beginning of 2006

Since 2002 we have switched to Network Appliance FAS940C NAS systems, then to FAS3050C models and recently to Network Appliance FAS6000 series with 10Gbe attachments. The total disk capacity of NICE will exceed 410 TeraBytes at the end of 2008.

Network Appliance storage server

The backup systems are based on Linear Tape Open (LTO) technology, from LTO-1 to the latest LTO-4. Two different generations of tape libraries are in currently in operation: SUN's 700 cartridge L700 and the much larger L8500 (see below). The fact that the backups are performed locally on each storage server frees the network for other purposes and increases substantially the overall throughput. The backup software in use is Atempo's Time Navigator software.

The compute clusters

Three different compute clusters are currently available for our users. They can be accessed either via the network individually  (logical names like: ssh coral10) or by generic names (e.g. nice). When the generic name is used, the connection will be established with the system having the lowest load.

As of 2008, fifty HP Proliant 145G3 (dual quad-core AMD) and fortySun V20Z, X2200 and X4100 participate in the compute clusters, with a large variety of scientific software.

I/O workstations

What we call I/O workstations are UNIX workstations based on LINUX PCs. Their main purpose is to allow our visiting scientists to write their data to DAT, CDs, DVDs,  ExaByte, LTO or DLT tape and/or to start the data analysis.

Onsite information:

Please see the link in the left-hand navigation column concerning I/O workstation locations (visible to onsite visitors only).