Automatic, Efficient and Scalable Provenance Registration for FAIR HPC workflows
Version 1

DOI
Provenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other environments with more relaxed restrictions, such as distributed ones). It also must be scalable in order to deal with large workflows, that are more typically used in HPC. We also target transparency for the user, shielding them from having to specify how provenance must be recorded. We implement our design using the COMPSs programming model as a Workflow Management System (WfMS) and use RO-Crate as a well-established specification to record and publish provenance. Experiments are provided, demonstrating the run time efficiency and scalability of our solution.

SEEK ID: https://workflowhub.eu/presentations/15?version=1

Filename: COMPSs Provenance WORKS 22.pdf  Download

Format: PDF document

Size: 2.75 MB

help Creators and Submitter
Creator
Submitter
Activity

Views: 985   Downloads: 88

Created: 2nd Aug 2023 at 15:53

Last updated: 7th Nov 2023 at 11:35

help Attributions

None

Version History

Version 1 (earliest) Created 2nd Aug 2023 at 15:53 by Raül Sirvent

No revision comments

Powered by
(v.1.16.0-main)
Copyright © 2008 - 2024 The University of Manchester and HITS gGmbH