|
UWSpace >
University of Waterloo >
Electronic Theses and Dissertations (UW) >
Please use this identifier to cite or link to this item:
http://hdl.handle.net/10012/2665
|
| Title: | Automated Hierarchy Discovery for Planning in Partially Observable Domains |
| Authors: | Charlin, Laurent |
| Keywords: | Computer Science
Artificial Intelligence
Reasoning under uncertainty
Decision Making
Planning
Markov Decision Process
Partially Observable
Hierarchy Discovery
Hierarchical Policy
Optimization |
| Approved Date: | 19-Jan-2007 |
| Date Submitted: | 2006 |
| Abstract: | Planning in partially observable domains is a notoriously difficult problem. However, in many real-world scenarios, planning can be simplified by decomposing the task into a hierarchy of smaller planning problems which, can then be solved independently of one another. Several approaches, mainly dealing with fully observable domains, have been proposed to optimize a plan that decomposes according to a hierarchy specified a priori. Some researchers have also proposed to discover hierarchies in fully observable domains.
In this thesis, we investigate the problem of automatically discovering planning hierarchies in partially observable domains. The main advantage of discovering hierarchies is that, for a plan of a fixed size, hierarchical plans can be more expressive than non-hierarchical ones.
Our solution frames the discovery and optimization of a hierarchical policy as a non-convex optimization problem. By encoding the hierarchical structure as variables of the optimization problem, we can automatically discover a hierarchy. Successfully solving the optimization problem therefore yields an optimal hierarchy and an optimal policy. We describe several techniques to solve the optimization problem. Namely, we provide results using general non-linear solvers, mixed-integer linear and non-linear solvers or a form of bounded hierarchical policy iteration. Our method is flexible enough to allow any parts of the hierarchy to be specified based on prior knowledge while letting the optimization discover the unknown parts. It can also discover hierarchical policies, including recursive policies, that are more compact (potentially infinitely fewer parameters). |
| Program: | Computer Science |
| Department: | School of Computer Science |
| Degree: | Master of Mathematics |
| URI: | http://hdl.handle.net/10012/2665 |
| Appears in Collections: | Electronic Theses and Dissertations (UW)
Faculty of Mathematics Theses and Dissertations
|
This item is protected by original copyright
|
All items in UWSpace are protected by copyright, with all rights reserved.
|
