PoS - Proceedings of Science
Volume 378 - International Symposium on Grids & Clouds 2021 (ISGC2021) - Network, Security, Infrastructure & Operations
Application of OMAT in HTCONDOR resource management
Q. Hu*, W. Zheng, X. Jiang and J. Shi
Full text: pdf
Published on: October 22, 2021
Conventional computing resource management systems use a system model to describe resources and a scheduler to control their allocation, computing resources are divided into isolated parts to provide computing services for different experiments. To improve resource utilization and reduce the deployment complexity of differentiated operating environments, our computing resources are configured to support a running environment of all HTC (high throughput computing) experiment jobs. To prevent experiments with few computing resources from occupying a large amount of extra computing resources for a long time, we configured the running jobs’ quota of each experiment to ensure the fairness. The conventional computing resource management systems does not adapt well to the ever-expanding resources scale and complex scheduling strategies.
Faced with these problems, we developed and implemented a new framework based on device management database and Open Maintain Analysis Tools (OMAT), a flexible and general approach to manage resources in a complex environment with that significantly reduces manual intervention. Novel aspects of the framework include a flexible configuration method for configuring the relationship between device, service, and experiment; alarm policies that quickly detect unallocated computing resources, and an operationally implementable way to quickly generate a scheduling policy and make it effective. This framework is robust, flexible, and scalable that can evolve with changes in resources and experiments.
The framework was designed to solve real problems encountered in the deployment of HTCondor, a high throughput computing scheduler system at IHEP of Chinese Academy of Sciences.
DOI: https://doi.org/10.22323/1.378.0021
How to cite

Metadata are provided both in "article" format (very similar to INSPIRE) as this helps creating very compact bibliographies which can be beneficial to authors and readers, and in "proceeding" format which is more detailed and complete.

Open Access
Creative Commons LicenseCopyright owned by the author(s) under the term of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.