Estimation of strategies in a markov game

In this paper a two-person Markov game, in discrete time, and with perfect state information, is considered from the point of view of a single player (player A) only. It is assumed that A's opponent (player B) uses the same strategy every time the game is played. It is shown that A can obtain a consistent estimate of B's strategy on the basis of his past experience of playing the game with B. Two methods of deriving such an estimate are given. Further, it is shown that using one of these estimates A can construct a strategy for himself which is asymptotically optimal. A simple example of a game in which the above method may be useful is given.