Active Sensing for Partially Observable Markov Decision Processes
Loading...
Date
2013-01-21T19:46:56Z
Authors
Koltunova, Veronika
Journal Title
Journal ISSN
Volume Title
Publisher
University of Waterloo
Abstract
Context information on a smart phone can be used to tailor applications for specific situations (e.g. provide tailored routing advice based on location, gas prices and traffic). However, typical context-aware smart phone applications use very limited context information such as user identity, location and time. In the future, smart phones will need to decide from a wide range of sensors to gather information from in order to best accommodate user needs and preferences in a given context.
In this thesis, we present a model for active sensor selection within decision-making processes, in which observational features are selected based on longer-term impact on the decisions made by the smart phone. This thesis formulates the problem as a partially observable Markov decision process (POMDP), and proposes a non-myopic solution to the problem using a state of the art approximate planning algorithm Symbolic Perseus. We have tested our method on a 3 small example domains, comparing different policy types, discount factors and cost settings. The experimental results proved that the proposed approach delivers a better policy in the situation of costly sensors, while at the same time provides the advantage of faster policy computation with less memory usage.
Description
Keywords
Active Sensing, Smart Sensor Selection, POMDP, Markov Decision Process, Probability, Utility, Sensor, Networks