Automatic, Efficient and Scalable Provenance Registration for FAIR HPC Workflows
View Publication
Export 
BiBTeXProvenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other environments with more relaxed restrictions, such as distributed ones). It also must be scalable in order to deal with large workflows, that are more typically used in HPC. We also target transparency for the user, shielding them from having to specify how provenance must be recorded. We implement our design using the COMPSs programming model as a Workflow Management System (WfMS) and use RO-Crate as a well-established specification to record and publish provenance. Experiments are provided, demonstrating the run time efficiency and scalability of our solution.
SEEK ID: https://workflowhub.eu/publications/21
DOI: 10.1109/WORKS56498.2022.00006
Teams: Workflows and Distributed Computing
Publication type: Proceedings
Journal: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)
Book Title: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)
Publisher: IEEE
Citation: 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS),pp.1-9,IEEE
Date Published: 1st Nov 2022
Registered Mode: by DOI
Export PNG
Download
Submitter
Views: 1639
Created: 2nd Aug 2023 at 15:37
Last updated: 2nd Aug 2023 at 15:41
TagsThis item has not yet been tagged.
AttributionsNone
Related items
Teams: GalaxyProject SARS-CoV-2, nf-core viralrecon, EOSC-Life - Demonstrator 7: Rare Diseases, iPC: individualizedPaediatricCure, EJPRD WP13 case-studies workflows, TransBioNet, OpenEBench, ELIXIR Proteomics
Organizations: Barcelona Supercomputing Center (BSC-CNS), ELIXIR
https://orcid.org/0000-0003-4929-1219
Computer Engineer in Barcelona Supercomputing Center (BSC)
Teams: Cluster Emergent del Cervell Humà, Workflows and Distributed Computing, WP6 - Tsunamis, WP7 - Earthquakes, WP8 - Anthropogenic geophysical extremes, WP5 - Volcanoes, Pillar I: Manufacturing, Pillar II: Climate, Pillar III: Urgent computing for natural hazards, eFlows4HPC general, COMPSs Tutorials
Organizations: Barcelona Supercomputing Center (BSC-CNS)
https://orcid.org/0000-0003-0606-2512
Established Researcher at Workflows and Distributed Computing Group, Computer Sciences department, Barcelona Supercomputing Center.
eFlows4HPC project aims at providing workflow software stack and an additional set of services to enable the integration of HPC simulations and modelling with big data analytics and machine learning in scientific and industrial applications. The project is also developing the HPC Workflows as a Service (HPCWaaS) methodology that aims at providing tools to simplify the development, deployment, execution and reuse of workflows. The project demonstrates its advances through three application Pillars ...
Teams: Cluster Emergent del Cervell Humà, Workflows and Distributed Computing, Pillar I: Manufacturing, Pillar II: Climate, Pillar III: Urgent computing for natural hazards, eFlows4HPC general, COMPSs Tutorials
Web page: https://eflows4hpc.eu
Distributed computing aims to offer tools and mechanisms that enable the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources in a transparent way. The research done in this team is based on the past expertise of the group, and on extending it towards the aspects of distributed computing that can benefit from this expertise. The team at BSC has a strong focus on programming models and resource management and scheduling in distributed computing ...
Provenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other
...
Creator: Raül Sirvent
Submitter: Raül Sirvent
BackTrackBB is a program for detection and space-time location of seismic sources based on multi-scale, frequency-selective statistical coherence of the wave field recorded by dense large-scale seismic networks and local antennas. The method is designed to enhance coherence of the signal statistical features across the array of sensors and consists of three steps. They are signal processing, space-time imaging and detection and location.
Source with inputs and outputs included (too big for ...
Lysozyme in Water simplest version, from COMPSs Tutorial. The original idea of this worklfow comes from http://www.mdtutorials.com/gmx/lysozyme/index.html
