Structured markovian decision problems

Structured finite action-finite state space discounted Markovian decision problems are analyzed. Any problem of a general class is shown to be equivalent to a “separated” problem with decomposable problem structure. A modified policy iteration approach is developed for this decomposable reformulation. Both analytic and computer evaluations of the decomposition algorithm's effectiveness are presented.