An iterative procedure for nondiscounted discrete-time markov decisions

This paper considers the problem of computing, by iterative methods, optimal policies for Markov decision processes. The policies computed are optimal for all sufficiently small interest rates.     

Assembly of systems having maximum reliability

The first problem considered in this paper is concerned with the assembly of independent components into parallel systems so as to maximize the expected number of systems that perform satisfactorily. Associated with each component is a probability of it performing successfully. It is shown that an optimal assembly is obtained if the reliability of each assembled system can be made equal. If such equality is not attainable, then bounds are given so that the maximum expected number of systems that perform satisfactorily will lie within these stated bounds; the bounds being a function of an arbitrarily chosen assembly. An improvement algorithm is also presented. A second problem treated is concerned with the optimal design of a system. Instead of assembling given units, there is an opportunity to “control” their quality, i.e., the manufacturer is able to fix the probability, p, of a unit performing successfully. However, his resources, are limited so that a constraint is imposed on these probabilities. For (1) series systems, (2) parallel systems, and (3) k out of n systems, results are obtained for finding the optimal p's which maximize the reliability of a single system, and which maximize the expected number of systems that perform satisfactorily out of a total assembly of J systems.     

A proportional defense model

This paper considers the problem of defending a set of point targets of differing values. The defense is proportional in that it forces the offense to pay a price, in terms of reentry vehicles expended, that is proportional to the value of the target. The objective of the defense is to balance its resources so that no matter what attack is launched, the offense will have to pay a price greater than or equal to some fixed value for every unit of damage inflicted. The analysis determines which targets should be defended and determines the optimal firing doctrine for interceptors at defended targets. A numerical example is included showing the relationship between the total target damage and the size of the interceptor force for different values of p, the interceptor single shot kill probability. Some generalizations are discussed.     

Optimal design of a multi-item,multi-location,multi-repair type repair and supply system

The design of a system with many locations, each with many items which may fail while in use, is considered. When items fail, they require repair; the particular type of repair being governed by a probability distribution. As repairs may be lengthy, spares are kept on hand to replace failed items. System ineffectiveness is measured by expected weighted shortages over all items and locations, in steady state. This can be reduced by either having more spares or shorter expected repair times. Design consists of a provisioning of the number of spares for each item, by location; and specifying the expected repair times for each type of repair, by item and location. The optimal design minimizes expected shortages within a budget constraint, which covers both (i) procurement of spares and (ii) procurement of equipment and manning levels for the repair facilities. All costs are assumed to be separable so that a Lagrangian approach is fruitful, yielding an implementable algorithm with outputs useful for sensitivity analysis. A numerical example is presented.     

Applications of Mills' Differential

Using Mills Differential a game value function ?, used in the Kemeny-Morgenstern-Thompson growth model to show the existence of solutions, is discussed. In doing so, several proofs become shorter, in addition known results are seen in a new context. Finally connections between the existence of a so-called economic solution and the differentiability or non-differentiability of function ? are analyzed.     

Scheduling jobs,with exponentially distributed processing times,on two machines of a flow shop

This paper treats the problem of sequencing n jobs on two machines in a “flow shop.” (That is, each job in the shop is required to flow through the same sequence of the machines.) The processing time of a given job on a given machine is assumed to be distributed exponentially, with a known mean. The objective is to minimize the expected job completion time. This paper proves an optimal ordering rule, previously conjectured by Talwar [10]. A formula is also derived through Markov Chain analysis, which evaluates the expected job completion time for any given sequence of the jobs. In addition, the performance of a heuristic rule is discussed in the light of the optimal solution.     

Structured markovian decision problems

Structured finite action-finite state space discounted Markovian decision problems are analyzed. Any problem of a general class is shown to be equivalent to a “separated” problem with decomposable problem structure. A modified policy iteration approach is developed for this decomposable reformulation. Both analytic and computer evaluations of the decomposition algorithm's effectiveness are presented.     

The lower bounded and partial upper bounded distribution model

This paper pruvides a smaller equivalelnt bounded variable transportation problem than that in Charnes, Glover, and Klingman [1] for the lower bounded and partial upper bounded distribution model.     

Bounds on the availability function

This paper gives bounds on the availability function for an alternating renewal process with exponential failure and general repair times. A bound on the error is also given. Several of the bounds with greatest practical consequence are worked out and illustrated. Repair distributions for which a lower bound on availability is easily computed are gamma (integer shape parameter), log normal, and Weibull. Finally, some simulation results for log normal repair versus gamma repair are given.