Runtime Configuration of GPU Workloads for Energy-efficient Execution

Abstract

Graphics processing units (GPUs) have become prevalent accelerators for modern computational workloads. However, GPUs consume an ever-increasing amount of energy, both in absolute and relative terms, so their economic and environmental impact has made energy efficiency an important systems objective. Improving the energy efficiency of GPU workload execution can be approached at multiple layers: hardware design, software implementation, and runtime configuration. Focusing on runtime parameters, this work does not aim to redesign hardware or modify software implementations. Instead, it builds upon first principles of hardware power modeling and microbenchmark observations to establish a conceptual framework for GPU power draw. The corresponding hardware-level power draw model is transformed into an intuitive workload execution model that uses externally controllable input parameters: processor frequency and workload batching. These parameters lend themselves to straightforward configuration rules that do not require extensive energy profiling or workload-specific parameter exploration. The findings are evaluated through experiments with large language model (LLM) inference workloads and the model is shown to closely approximate the energy consumption trends of real-world workloads. This demonstrates that the given rules can serve as lightweight yet effective guidance for improving the energy efficiency of practical GPU workload executions.