Automatic, Efficient and Scalable Provenance Registration for FAIR HPC Workflows

Abstract:

Provenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other environments with more relaxed restrictions, such as distributed ones). It also must be scalable in order to deal with large workflows, that are more typically used in HPC. We also target transparency for the user, shielding them from having to specify how provenance must be recorded. We implement our design using the COMPSs programming model as a Workflow Management System (WfMS) and use RO-Crate as a well-established specification to record and publish provenance. Experiments are provided, demonstrating the run time efficiency and scalability of our solution.

SEEK ID: https://workflowhub.eu/publications/21

DOI: 10.1109/WORKS56498.2022.00006

Teams: Workflows and Distributed Computing

Publication type: Proceedings

Journal: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)

Book Title: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)

Publisher: IEEE

Citation: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS),pp.1-9,IEEE

Date Published: 1st Nov 2022

Registered Mode: by DOI

help Submitter
Citation
Sirvent, R., Conejero, J., Lordan, F., Ejarque, J., Rodriguez-Navas, L., Fernandez, J. M., Capella-Gutierrez, S., & Badia, R. M. (2022). Automatic, Efficient and Scalable Provenance Registration for FAIR HPC Workflows. In 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS) (Vol. 533, pp. 1–9). IEEE. https://doi.org/10.1109/works56498.2022.00006
Activity

Views: 1512

Created: 2nd Aug 2023 at 15:37

Last updated: 2nd Aug 2023 at 15:41

help Tags

This item has not yet been tagged.

help Attributions

None

Powered by
(v.1.16.0-main)
Copyright © 2008 - 2024 The University of Manchester and HITS gGmbH