Sunday, May 18, 2014

BSC's MareNostrum (2013)

Recently i visited Barcelona Supercomputing Center and UPC for the Computing Systems Week and the Block Review of Euroserver project.

During my visit, I had the opportunity to visit the MareNostrum supercomputer. MareNostrum is a supercomputer in the Barcelona Supercomputing Center initially build in 2004. The initial setup was containing around 3564 cores based on BM 64-bit PowerPC 970MP processors running at 2.3 GHz. In November 2006 its capacity was increased due to the large demand of scientific projects. MareNostrum increased the calculation capacity until reaching 94.21 Teraflops. The system was based on 2560 JS21 blade computing nodes, based on the same processor for 10,240 CPUs in total. The machine is using Myrinet interconnect for communication.

In 2013 MareNostrum upgraded and changed the CPU architecture. With the last upgrade, MareNostrum has a peak performance of 1,1 Petaflops, with 48896 Intel Sandy Bridge processors in 3056 nodes, and 84 Xeon Phi 5110P in 42 nodes, with more than 100.8 TB of main memory and 2 PB of GPFS disk storage. At  May 2014, MareNostrum was positioned at the 34th place in the TOP500 list of fastest supercomputers in the world. The machine provides 925.1 usable TFlops and consumes 1016 kW of electrical power.

The beauty of the machine is not because of the computation power or the efficiency of power consumption, but because it is located inside a chapel! Here are some pictures from inside:


Computation Node based on Intel processors.
Computation Node based on Intel processors.

Power and cooling distribution.
Power and cooling distribution.

View from the top of the Supercomputer.
View from the top of the Supercomputer.

View from the side. Orange and yellow cables are Myrinet optical links, blue cables are management GBit ethernet.
View from the side. Orange and yellow cables are Myrinet optical links, blue cables are management GBit ethernet.

View from the top.

View of Barcelona Supercomputer from the top. Note the cooling pumps at the end.
View of Barcelona Supercomputer from the top. Note the cooling pumps at the end.