Fair Scheduling for Deadline Driven, Resource-Constrained Multi-Analytics Workloads

Report ID: 
2019-09
Authors: 
Stratos Dimopoulos, Chandra Krintz, Rich Wolski
Date: 
2019-09-01 00:00:00

Abstract

We present a new approach to fair-share, deadline-aware job scheduling for resource-limited cloud deployments that are managed by “big data” framework resource negotiators (e.g. YARN and Mesos), called Justice. Justice provides admission control that leverages historical traces and job deadline information to guide and adapt resource allocation decisions to changing workload conditions. We evaluate Justice using different deadline types and production analytics workloads. We find that it outperforms extant allocators in terms of fair allocation, deadline satisfaction, and useful work, among other metrics.

Document

PDF icon extended_tech_report_19_09.pdf