On computing the expected discounted return in a markov chain

The discounted return associated with a finite state Markov chain X₁, X₂… is given by g(X₁)+ αg(X₂) + α²g(X₃) + …, where g(x) represents the immediate return from state x. Knowing the transition matrix of the chain, it is desired to compute the expected discounted return (present worth) given the initial state. This type of problem arises in inventory theory, dynamic programming, and elsewhere. Usually the solution is approximated by solving the system of linear equations characterizing the expected return. These equations can be solved by a variety of well-known methods. This paper describes yet another method, which is a slight modification of the classical iterative scheme. The method gives sequences of upper and lower bounds which converge mono-tonely to the solution. Hence, the method is relatively free of error control problems. Computational experiments were conducted which suggest that for problems with a large number of states, the method is quite efficient. The amount of computation required to obtain the solution increases much slower with an increase in the number of states, N, than with the conventional methods. In fact, computational time is more nearly proportional to N², than to N³.     

The dynamic transportation problem: A survey

The dynamic transportation problem is a transportation problem over time. That is, a problem of selecting at each instant of time t, the optimal flow of commodities from various sources to various sinks in a given network so as to minimize the total cost of transportation subject to some supply and demand constraints. While the earliest formulation of the problem dates back to 1958 as a problem of finding the maximal flow through a dynamic network in a given time, the problem has received wider attention only in the last ten years. During these years, the problem has been tackled by network techniques, linear programming, dynamic programming, combinational methods, nonlinear programming and finally, the optimal control theory. This paper is an up-to-date survey of the various analyses of the problem along with a critical discussion, comparison, and extensions of various formulations and techniques used. The survey concludes with a number of important suggestions for future work.     

Computing equilibria via nonconvex programming

The problem of determining a vector that places a system in a state of equilibrium is studied with the aid of mathematical programming. The approach derives from the logical equivalence between the general equilibrium problem and the complementarity problem, the latter being explicitly concerned with finding a point in the set S = {x: < x, g(x)> = 0, g(x) ≦ 0, x ≧ 0}. An associated nonconvex program, min{? < x, g(x) > : g(x) ≦ 0, x ≧ 0}, is proposed whose solution set coincides with S. When the excess demand function g(x) meets certain separability conditions, equilibrium solutions are obtained by using an established branch and bound algorithm. Because the best upper bound is known at the outset, an independent check for convergence can be made at each iteration of the algorithm, thereby greatly increasing its efficiency. A number of examples drawn from economic and network theory are presented in order to demonstrate the computational aspects of the approach. The results appear promising for a wide range of problem sizes and types, with solutions occurring in a relatively small number of iterations.     

Book reviews

America's Secret Power: the CIA in a Democratic Society. By Loch K. Johnson. Oxford University Press, New York (1989), ISBN 0–19–505490–3, $24.95

The Bundeswehr and Western Security. Edited by Stephen F. Szabo. Houndmills, Basingstoke, and Macmillan, London (1990), ISBN 0–333–49880–1, £45.00

Symbolic Defense: the Cultural Significance of the Strategic Defense Initiative. By Edward Tabor Linenthal. University of Illinois Press, Chicago, IL (1989), ISBN 0–252–01619‐X, $19.95

Rethinking European Security. Edited by Furio Cerutti and Rodolfo Ragionieri. Crane Russak, New York (1990), £29.00

Alternative Conventional Defense Postures in the European Theater, Vol. 1: The Military Balance and Domestic Constraints. Edited by Hans Günter Brauch and Robert Kennedy, Crane Russak, New York (1990), £32.00

The Gulf War. Edited by Hanns Maull and Otto Pick. Pinter, London (1989), ISBN 0–86187–763–2, £36.00     

The (s,Q) inventory model with Erlang lead time and deterministic demand

We develop a simple, approximately optimal solution to a model with Erlang lead time and deterministic demand. The method is robust to misspecification of the lead time and has good accuracy. We compare our approximate solution to the optimal for the case where we have prior information on the lead‐time distribution, and another where we have no information, except for computer‐generated sample data. It turns out that our solution is as easy as the EOQ's, with an accuracy rate of 99.41% when prior information on the lead‐time distribution is available and 97.54–99.09% when only computer‐generated sample information is available. Apart from supplying the inventory practitioner with an easy heuristic, we gain insights into the efficacy of stochastic lead time models and how these could be used to find the cost and a near‐optimal policy for the general model, where both demand rate and lead time are stochastic. © 2004 Wiley Periodicals, Inc. Naval Research Logistics, 2004     

Even if it were but betwixt memory and reputation,in retrospect serves neither

Robert S. McNamara [and Brian Van De Mark,] In Retrospect: The Tragedy and Lessons of Vietnam, NewYork:NewYork Times Books, Random House, 1995. Pp.xviii + 414. $27.50. ISBN 08129–2523–8.     

Using a birth‐and‐death process to estimate the steady‐state distribution of a periodic queue

If the number of customers in a queueing system as a function of time has a proper limiting steady‐state distribution, then that steady‐state distribution can be estimated from system data by fitting a general stationary birth‐and‐death (BD) process model to the data and solving for its steady‐state distribution using the familiar local‐balance steady‐state equation for BD processes, even if the actual process is not a BD process. We show that this indirect way to estimate the steady‐state distribution can be effective for periodic queues, because the fitted birth and death rates often have special structure allowing them to be estimated efficiently by fitting parametric functions with only a few parameters, for example, 2. We focus on the multiserver M_t/GI/s queue with a nonhomogeneous Poisson arrival process having a periodic time‐varying rate function. We establish properties of its steady‐state distribution and fitted BD rates. We also show that the fitted BD rates can be a useful diagnostic tool to see if an M_t/GI/s model is appropriate for a complex queueing system. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 664–685, 2015