This repository contains code designed to evaluate the weak scaling performance of the Dask distributed computing library. The primary purpose of this project is to provide a means to compare the performance of a Dask cluster’s installation and find out how different configurations affect the scalability of the system.
The scalability is expressed in terms of weak scaling, which measures the efficiency of a parallel algorithm when the problem size and the number of computational resources are increased proportionally.
In particular the code in this repository was originally developed to compare the performance of a Dask cluster installed on a HPC cluster based on the Slurm workload manager and the installation on the same hardware but using the Kubernetes container orchestration system. However the code should be easily adaptable to other distributed computing environments, for more detail see the Dask documentation related to deploying Dask on different environments.
