Fictitious Mean-field Reinforcement Learning for Distributed Load Balancing

dc.contributor.advisorZahedi, Seyed Majid
dc.contributor.authorFardno, Fatemeh
dc.date.accessioned2022-09-29T14:04:50Z
dc.date.available2023-09-30T04:50:05Z
dc.date.issued2022-09-29
dc.date.submitted2022-09-27
dc.description.abstractIn this work, we study the application of multi-agent reinforcement learning (RL) in distributed systems. In particular, we consider a setting in which strategic clients compete over a set of heterogeneous servers. Each client receives jobs at a fixed rate. For each job, clients choose a server to run the job. The objective of each client is to minimize its average wait time. We model this setting as a Markov game and theoretically prove that the game becomes in the limit a Markov potential game (MPG). We further propose a novel mean-field reinforcement learning algorithm, combining mean-field Q-learning and fictitious play. Through rigorous experiments, we show that our algorithm outperforms naive deployment of single-agent RL, and in some cases, performs comparably to the Nash Q-learning, while being less complex in terms of memory and computation. We also empirically analyze the convergence of our proposed algorithm to a Nash equilibrium and study its performance in four benchmark examples.en
dc.identifier.urihttp://hdl.handle.net/10012/18841
dc.language.isoenen
dc.pendingfalse
dc.publisherUniversity of Waterlooen
dc.titleFictitious Mean-field Reinforcement Learning for Distributed Load Balancingen
dc.typeMaster Thesisen
uws-etd.degreeMaster of Applied Scienceen
uws-etd.degree.departmentElectrical and Computer Engineeringen
uws-etd.degree.disciplineElectrical and Computer Engineeringen
uws-etd.degree.grantorUniversity of Waterlooen
uws-etd.embargo.terms1 yearen
uws.contributor.advisorZahedi, Seyed Majid
uws.contributor.affiliation1Faculty of Engineeringen
uws.peerReviewStatusUnrevieweden
uws.published.cityWaterlooen
uws.published.countryCanadaen
uws.published.provinceOntarioen
uws.scholarLevelGraduateen
uws.typeOfResourceTexten

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Fardno_Fatemeh.pdf
Size:
2.04 MB
Format:
Adobe Portable Document Format
Description:

License bundle

Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
6.4 KB
Format:
Item-specific license agreed upon to submission
Description: