Provenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other environments with more relaxed restrictions, such as distributed ones). It also must be scalable in order to deal with large workflows, that are more typically used in HPC. We also target transparency for the user, shielding them from having to specify how provenance must be recorded. We implement our design using the COMPSs programming model as a Workflow Management System (WfMS) and use RO-Crate as a well-established specification to record and publish provenance. Experiments are provided, demonstrating the run time efficiency and scalability of our solution.
SEEK ID: https://workflowhub.eu/publications/21
DOI: 10.1109/WORKS56498.2022.00006
Teams: Workflows and Distributed Computing
Publication type: Proceedings
Journal: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)
Book Title: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)
Publisher: IEEE
Citation: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS),pp.1-9,IEEE
Date Published: 1st Nov 2022
Registered Mode: by DOI
Views: 1512
Created: 2nd Aug 2023 at 15:37
Last updated: 2nd Aug 2023 at 15:41
This item has not yet been tagged.
None