Programmable and Adaptable Storage for AI-oriented HPC Ecosystems


High-Performance Computing (HPC) services are increasingly sought to meet the demands of Artificial Intelligence (AI) applications. While the processing capabilities offered by HPC infrastructures can scale and handle the computational requirements of AI applications, the same does not hold true for the storage counterpart. Indeed, shared storage architectures quickly become a performance bottleneck when used by multiple instances of data-centric applications.

The PAStor project aims at providing a novel Software-Defined Storage (SDS) solution for HPC that can efficiently handle I/O flows from multiple AI workloads by automatically adjusting storage configurations and resources to dynamically meet application requirements. The proposed solution will be crucial to address the aforementioned storage performance bottleneck and fairness challenges of HPC infrastructures. The research output from PAStor will be released as an open-source prototype that will provide the first building block towards a novel storage architecture suited for the exascale computing infrastructure.

By gathering the expertise of INESC TEC and Hood College researchers in the AI and distributed storage fields, and by including researchers from TACC and MACC with experience on managing HPC infrastructures, the project will produce new high quality research findings and advance the state-of-the-art for storage solutions currently deployed at HPC centers.

News and Events

PAStor project talk at Encontro Ciência'21
June 30, 2021

The research challenges and goals for the PAStor project, which will improve the convergence between AI applications and HPC supercomputers,...

PAStor researchers proposes a trusted hardware-based privacy-preserving deduplication system
May 21, 2021

INESC TEC’s team has published a paper entitled “S2Dedup: SGX-enabled Secure Deduplication” at the 14th ACM International Systems and Storage...

April 27, 2021

The 1st Workshop on High-Performance and Reliable Big Data (HPBD’21) will take place on September 20th, 2021, in Chicago, USA...