Lexicographic refinements in stationary possibilistic Markov Decision Processes

Abstract

Possibilistic Markov Decision Processes offer a compact and tractable way to represent and solve problems of sequential decision under qualitative uncertainty. Even though appealing for its ability to handle qualitative problems, this model suffers from the drowning effect that is inherent to possibilistic decision theory. The present1 paper proposes to escape the drowning effect by extending to stationary possibilistic MDPs the lexicographic preference relations defined by Fargier and Sabbadin [13] for non-sequential decision problems. We propose a value iteration algorithm and a policy iteration algorithm to compute policies that are optimal for these new criteria. The practical feasibility of these algorithms is then experimented on different samples of possibilistic MDPs.