Supercomputing transfer goes off without a hitch

When the NeSI supercomputers were replaced in early 2018, REANNZ engineers were on hand to make sure the research and education advanced network smoothly handled the transfer of an estimated 900 TB (terabytes) of user data.

NeSI – the New Zealand eScience Infrastructure – makes supercomputing facilities available to scientists through REANNZ’s high-speed network, which links universities and research institutes. A collaboration between NIWA, Landcare Research and the universities of Auckland and Otago, NeSI is helping researchers solve some of today’s most pressing scientific problems.

Earlier in 2018, NeSI’s supercomputers – ‘FitzRoy’, based at NIWA’s Greta Point, Wellington site, and ‘Pan’, based at the University of Auckland – were replaced with new machines able to deliver up to 10 times the computing capability and four times the storage capacity of their predecessors.

It was no small task. Alongside logistics, physical handling, cabling, electrical work and testing, tens of millions of files and hundreds of terabytes of user data had to be migrated to the new machines – ‘Maui’ and ‘Mahuika’ in Wellington, and ‘Kupe’ in Auckland.

Using the REANNZ network was essential for supporting the massive transfer. REANNZ engineers rebalanced the traffic in the early stages of the transfer to ensure NIWA was able to make the best use of their capability.

The REANNZ network worked overtime on the transfer. Peak transfer speeds topped 19 Gbps. It amounted to the largest one-off transfer of data ever undertaken on the REANNZ network. Testament to the network’s speed, resilience and capacity, there was little to no disruption to regular user operations.

NeSI is planning for the new supercomputers to have three- to six-year lifetimes, with a major technology refresh halfway through – more than enough time to help power the next frontier of scientific research.

Data Transfer