

Description
Effective resource scheduling is critical in high-performance (HPC) and high-throughput computing (HTC) environments, where traditional scheduling systems struggle with resource contention, data locality, and fault tolerance. Meta-scheduling, which abstracts multiple schedulers for unified job allocation, addresses these challenges.
Galaxy, a widely used platform for data-intensive computational analysis, employs the \textit{Total Perspective Vortex (TPV)} system for resource scheduling. With over 550,000 users, Galaxy aims to optimize scheduling efficiency in large-scale environments. While TPV offers flexibility, its decision-making can be enhanced by incorporating real-time resource availability and job status.
This paper introduces the TPV Broker, a meta-scheduling framework that integrates real-time resource data to enable dynamic, data-aware scheduling. TPV Broker enhances scalability, resource utilization, and scheduling efficiency in Galaxy, offering potential for further improvements in distributed computing environments.
SEEK ID: https://workflowhub.eu/publications/55
Teams: EuroScienceGateway
Publication type: Unpublished
Publisher: Zenodo
Citation: https://doi.org/10.5281/zenodo.14936846
Date Published: 1st Feb 2025
URL: https://doi.org/10.5281/zenodo.14936846
Registered Mode: imported from a bibtex file

Views: 68
Created: 3rd Oct 2025 at 17:15

This item has not yet been tagged.

None