Distributed filesystems (GPFS, CephFS and Lustre-ZFS) deployment on Kubernetes/Docker clusters
October 22, 2021
Nowadays Kubernetes has become a powerful tool to deploy and manage containerized applications. Modern datacenters need distributed filesystems to provide user applications with access to stored data on a large number of nodes. The possibility to mount a distributed filesystem and exploit its native application programming interfaces in a Docker container, combined with the advanced orchestration features provided by Kubernetes, may enhance flexibility in data management, installation, running and monitoring of transfer services. Moreover, it would allow the execution of dedicated services on different nodes, in isolated and automatically replicable environments, so to improve deployment efficiency and fail-safeness. The goal of this work is to demonstrate the feasibility of using Kubernetes and Docker to setup clients capable to access a distributed filesystem from existing clusters and to create clusters based on containerized servers. Despite being just a proof of concept, the effort has shown the possibility of using different types of distributed filesystems (GPFS, CephFS, Lustre-ZFS) with equally positive results.
How to cite
Metadata are provided both in "article" format (very similar to INSPIRE) as this helps creating
very compact bibliographies which can be beneficial to authors and
readers, and in "proceeding" format
which is more detailed and complete.