ParTec in the SEA projects – preparing Europe for the future of supercomputing.

Munich, 18 March 2024 - A wide range of European partners, including ParTec AG worked together in the DEEP-SEA, IO-SEA and RED-SEA projects and created important results preparing Europe for the Exascale era in supercomputing.

The HPC research projects DEEP-SEA, IO-SEA and RED-SEA are wrapping up this month after a three-year project term. The three projects worked together to develop key technologies for European Exascale supercomputers, based on the dynamic Modular System Architecture (dMSA) developed and patented by ParTec in cooperation with Jülich Supercomputing Centre (JSC), a blueprint architecture for highly efficient and scalable heterogeneous Exascale HPC systems. To achieve this, the three projects collaborated on system software and programming environments, data management and storage, as well as interconnects adapted to this architecture. The results of their joint work will be presented at a co-design workshop and poster session at the EuroHPC Summit (Antwerp, 18-21 March, www.eurohpcsummit.eu).

The future of supercomputing is Exascale

DEEP-SEA, IO-SEA and RED-SEA are part of a larger family of research and innovation projects co-funded by the EuroHPC Joint Undertaking to prepare key technology and solutions for the “Exascale era” in supercomputing. This Exascale generation is capable of performing at least 10¹⁸ floating point operations per second and enable novel and more accurate scientific applications. Exascale supercomputers are a lot more complex than previous generations of slower systems and combine a wide range of different hardware components used for specific purposes. They must integrate resources such as CPUs, GPUs and other accelerators as well as communication, memory and storage technologies into one system in a manner that supports traditional HPC simulations as well as artificial intelligence and big data analysis.

A Modular Supercomputing Architecture for European Exascale supercomputers

The MSA paradigm was designed for this very purpose. It is based on a novel system design to integrate heterogeneous resources and technologies and to fulfil the requirements of a wide range of application areas, from compute-intensive, highly scalable simulation codes to data-intensive artificial intelligence workflows. Different workload components run on the module best suited to them, which improves time-to-solution and energy efficiency. The Jülich Supercomputing Centre’s (JSC) new JUPITER supercomputer will be based on this architecture. JUPITER, the installation of which starts in 2024, is co-funded by the EuroHPC JU, the German Ministry of Education and Research, and the Ministry of Culture and Research of the German state of North Rhine-Westphalia.

To implement, extend and enhance the dMSA, the three SEA projects worked on different tasks: DEEP-SEA, spearheaded by the JSC, focused on the programming environment and the HPC software stack. IO-SEA, led by CEA, prototyped a novel data management and storage platform. RED-SEA, headed by Eviden, prepared the next generation of the European BXI (Bull eXascale Interconnect) and its related software stack. While the three SEA projects are independent from each other, they are interlinked and collaborate closely to make efficient systems possible by implementing the MSA. They build upon the results of other, previous projects co-funded by the European commission and draw on a vast network of ongoing partner projects, institutions, and private companies from all over Europe.

The EuroHPC Summit 2024 – A great opportunity to showcase the project results

The integrated software stack developed by DEEP-SEA is one of the key results of the projects, alongside the establishment of the dMSA in large, operational systems like JSC’s JUWELS and the MeluXina system in Luxembourg, and soon in the JUPITER system at JSC. The stack integrates “best of breed” programming models, APIs, tools and libraries that are also relevant for non-modular HPC systems. In particular, the abstraction of optimization cycles, the support of hierarchical storage systems and the use of modern CI/CD techniques will bring direct benefits for such systems. This also applies to the pioneering work in the area of malleability. The stack is freely available as open source to all interested parties.

IO-SEA developed a storage and I/O stack suitable for the dMSA that deals with data movements across hierarchical storage systems including novel non-volatile memory, solid state devices, hard disk drives and tape storage. IO-SEA also developed the concept of short lived “ephemeral storage services” that are created on demand and enable users to flexibly operate the system. The project also introduced extensive use and exploitation of instrumentation and telemetry across the I/O stack and new I/O APIs that expose the actual semantics of data.

The RED-SEA project has laid the groundwork for the development and industrialization of the third generation of the European BXI interconnect. The project helped to bring BXI to the Exascale level, with improved scalability and resilience to reach well over 100K endpoints, while significantly improving the latency and the bandwidth available for each endpoint. RED-SEA also addressed real-world network performance with congestion management and adaptive routing mechanisms. Finally, RED-SEA opened up BXI to Ethernet compatibility, by designing a seamless interface between BXI and Ethernet via a new low-latency Gateway solution.

The ParaStation Modulo Software Suite and especially ParaStation MPI with its runtime and execution environment are key components of all three projects. They were extended and optimized to support the new concepts (for instance malleability), interconnects (BXI) and services (like the IO-SEA ephemeral storage services).

The EuroHPC Summit, which takes place from 18 to 21 March in Antwerp, Belgium, offers interested parties the opportunity to learn about the results. The projects will and present their results in the poster session and participate in the workshop on “Co-Designing the Future of European HPC: Eco-friendly Technologies, Systems and Software” which will highlight technology areas developed in Europe as a result of successful co-design, including the SEA projects outcomes. The workshop is being organised by Sai Narasimhamurthy, and Hans-Christian Hoppe will present the key SEA projects results; both are employees of ParTec AG.

The ParaStation Modulo Software Suite supports novel concepts in Exascale computing developed in these SEA projects whilst also supporting all the project partners in maintaining the dMSA prototype cluster at Juelich Supercomputing Centre.

Quotations:

“By successfully delivering on the SEA projects, ParTec continues to be at the cutting edge of innovation in Extreme Scale Computing and paving the path forward for European Exascale and Post-Exascale systems and software technologies” says Bernhard Frohwitter, CEO of ParTec AG.

Prof. Thomas Lippert, director of JSC: “Based on the groundbreaking results of the previous DEEP projects, the solid, comprehensive HPC SW stack developed by DEEP-SEA makes systems based on the modular supercomputing architecture (MSA) easily usable for HPC application developers and end users and is therefore of prime importance for the future of MSA in general and especially for the upcoming JUPITER Exascale system at JSC and other modular systems all around the world.”

About the SEA projects:

The SEA projects DEEP-SEA, RED-SEA and IO-SEA realise the joint vision of the Jülich Supercomputing Centre (JSC), the French Alternative Energies and Atomic Energy Commission (CEA), Eviden and ParTec AG for a dynamic Modular System Architecture (dMSA) for future Exascale High-Performance Computing (HPC) systems. The four core partners have brought together the best European academic and industrial expertise to develop the technologies for the next generation of supercomputers. DEEP-SEA focuses on the programming environment & software stack for Exascale systems. IO-SEA provides a data management and storage platform for these systems, and RED-SEA enhances the European interconnect technology BXI and its related low-level software.

The SEA projects have received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreements n° 955606, 95811, and 955776 and support from France, the Czech Republic, Germany, Spain, Ireland, Sweden, Switzerland, Italy and Greece.

More info at www.sea-projects.eu