DEEP - Extreme Scale Technologies

The DEEP-EST (“DEEP - Extreme Scale Technologies”) project will create a first incarnation of the Modular Supercomputer Architecture (MSA) and demonstrate its benefits. In the spirit of the DEEP and DEEP-ER projects, the MSA integrates compute modules with different performance characteristics into a single heterogeneous system. Each module is a parallel, clustered system of potentially large size. A federated network connects the module-specific interconnects. MSA brings substantial benefits for heterogeneous applications/workflows: each part can be run on an exactly matching system, improving time to solution and energy use. This is ideal for supercomputer centres running heterogeneous application mixes (higher throughput and energy efficiency). It also offers valuable flexibility to the compute providers, allowing the set of modules and their respective size to be tailored to actual usage.

The DEEP-EST prototype will include three modules: general purpose Cluster Module and Extreme Scale Booster supporting the full range of HPC applications, and Data Analytics Module specifically designed for high-performance data analytics (HPDA) workloads. Proven programming models and APIs from HPC (combining MPI and OmpSs) and HPDA will be extended and combined with a significantly enhanced resource management and scheduling system to enable straightforward use of the new architecture and achieve highest system utilisation and performance. Scalability projections will be given up to the Exascale performance class. The DEEP-EST prototype will be defined in close co-design between applications, system software and system component architects. Its implementation will employ European integration, network and software technologies. Six ambitious and highly relevant European applications from HPC and HPDA domains will drive the co-design, serving to evaluate the DEEP EST prototype and demonstrate the benefits of its innovative Modular Supercomputer Architecture.

The DEEP-EST consortium has advanced the development of key European technologies, which can now be exploited by the respective partners:
• Megware: energy efficient integration, amplified portfolio with new accelerated node designs.
• EXTOLL, by its Fabri³ integrated fabric switch, can acquire strategic customers to realize its next-gen network chip. The network attached technologies NAM and GCE will be exploited as Fabri³-attached or as standalone products.
• ParTec: brings the new features into future versions of ParaStation Modulo, for which ParTec offers commercial support.
• FHG-ITWM with its spin-off ThinkParQ offer commercial support for BeeGFS and its new features.
• BSC: improved OmpSs-2 (promoting them in the OpenMP standardization committee) plus Extrae/Paraver/Dimemas performance analysis and modelling tools, and scheduling models.
• BADW-LRZ: system monitoring tool DBDC used on ts Tier-0 and Tier-1 systems.
• JUELICH: Open Source JUBE benchmarking environment and SIONlib I/O concentrator library.
• Application developers (NMBU, KU Leuven, Astron, UoI, CERN, and NCSA) exploit their improved codes to advance science in their respective research fields.

Dissemination: Scientific results have been published in 42 open access articles and events, including several training events. The website is the main hub of project content, and social media channels (Twitter, LinkedIn, Facebook) have increased attention to the project events and materials. Recently a new Podcast channel on SoundCloud has been added. DEEP EST supports in particular initiatives that foster diversity in HPC. An Early Access Program opened the DEEP-EST prototype and its SW to the wider HPC community. The DEEP-EST dissemination and communication strategy will continue in the soon starting DEEP-SEA project.