In-Network Scheduling for Real-Time Analytics

Abstract

This thesis presents Bolt, a novel scheduler design for large-scale real-time data analytics. Bolt achieves the scheduling accuracy of modern centralized schedulers while supporting clusters with hundreds of thousands of nodes. At Bolt’s core is a scheduler design that leverages modern programmable switches. Bolt supports a FIFO scheduling policy, as well as task priority-based and task resource constraint-based scheduling policies.

Evaluation of a Bolt prototype on our cluster with a Barefoot Tofino switch shows that the proposed approach can reduce scheduling overhead by 40x and increase the scheduling throughput by 50x compared to state-of-the-art centralized and decentralized schedulers.