Optimal Scheduling of Virtualised Workloads using Learning Algorithms

The introduction of virtualization technology to grid and cloud computing infrastructures has enabled applications to be decoupled from the underlying hardware providing the benefits of portability, better control over execution environment and isolation. Virtualization layer also incurs a performance penalty, which can be significant for High Performance Computing (HPC) applications with high work volumes. Virtualization thus brings new requirements for dynamic adaptation of the scheduling to realize the potential flexibility of faster re-tasking and reconfiguration of workloads. Often scheduling approaches are based on some well-defined system-wide performance metric within the context of the given systems scope of operation. However, this is not optimized for the structure and behavior of specific applications having a mix of task types each with their own task precedences and resource requirements. This body of work is concerned with combining virtualization and adaptive scheduling techniques to achieve an optimal balance between task placement flexibility and processing performance on large scale scientific Grid infrastructure while offsetting virtualization overhead.