Optimal betting strategies for favorable games

We examine the problem of a gambler interested in maximizing the expected value of a convex utility function of his fortune after n plays of a game. We allow any probability distribution to rule the outcome of each play, and this distribution may change from play to play according to a Markov process. We present results regarding the existence of an optimal policy and its structural dependence on the gambler's fortune. The well-known results of Bellman and Kalaba for exponential and logarithmic utility functions and coin-tossing games are generalized. We also examine the situation of general stale spaces and show that the same structural results hold.     

On a search for a moving target

We consider the problem of searching for a target that moves in discrete time and space according to some Markovian process. At each time, a searcher attempts to detect the target. If the searcher's action at each time is such as to maximize his chances of immediate detection, we call his strategy “myopic.” We provide a computationally useful necessary condition for optimality, and use it to provide an example wherein the myopic strategy is not optimal.     

Acceptance sampling,imperfect production,and the optimality of zero defects

Acceptance sampling is often used to monitor the quality of raw materials and components when product testing is destructive, time-consuming, or expensive. In this paper we consider the effect of a buyer-imposed acceptance sampling policy on the optimal batch size and optimal quality level delivered by an expected cost minimizing supplier. We define quality as the supplier's process capability, i.e., the probability that a unit conforms to all product specifications, and we assume that unit cost is an increasing function of the quality level. We also assume that the supplier faces a known and constant “pass-through” cost, i.e., a fixed cost per defective unit passed on to the buyer. We show that the acceptance sampling plan has a significant impact on the supplier's optimal quality level, and we derive the conditions under which zero defects (100% conformance) is the policy that minimizes the supplier's expected annual cost. © 1997 John Wiley & Sons, Inc. Naval Research Logistics 44: 515–530, 1997     

An approximate dynamic programing approach to the development of heuristics for the scheduling of impatient jobs in a clearing system

A single server is faced with a collection of jobs of varying duration and urgency. Each job has a random lifetime during which it is available for nonpreemptive service. Should a job's lifetime expire before its service begins then it is lost from the system unserved. The goal is to schedule the jobs for service to maximize the expected number served to completion. Two heuristics have been proposed in the literature. One (labeled π^S) operates a static priority among the job classes and works well in a “no premature job loss” limit, whereas the second (π^M) is a myopic heuristic which works well when lifetimes are short. Both can exhibit poor performance for problems at some distance from the regimes for which they were designed. We develop a robustly good heuristic by an approximative approach to the application of a policy improvement step to the asymptotically optimal heuristic π^S, in which we use a fluid model to obtain an approximation for the value function of π^S. The performance of the proposed heuristic is investigated in an extensive numerical study. © 2010 Wiley Periodicals, Inc. Naval Research Logistics 2010     

Production to order and off‐line inspection when the production process is partially observable

This study combines inspection and lot‐sizing decisions. The issue is whether to INSPECT another unit or PRODUCE a new lot. A unit produced is either conforming or defective. Demand need to be satisfied in full, by conforming units only. The production process may switch from a “good” state to a “bad” state, at constant rate. The proportion of conforming units in the good state is higher than in the bad state. The true state is unobservable and can only be inferred from the quality of units inspected. We thus update, after each inspection, the probability that the unit, next candidate for inspection, was produced while the production process was in the good state. That “good‐state‐probability” is the basis for our decision to INSPECT or PRODUCE. We prove that the optimal policy has a simple form: INSPECT only if the good‐state‐probability exceeds a control limit. We provide a methodology to calculate the optimal lot size and the expected costs associated with INSPECT and PRODUCE. Surprisingly, we find that the control limit, as a function of the demand (and other problem parameters) is not necessarily monotone. Also, counter to intuition, it is possible that the optimal action is PRODUCE, after revealing a conforming unit. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

Strategic capacity decision‐making in a stochastic manufacturing environment using real‐time approximate dynamic programming

In this study, we illustrate a real‐time approximate dynamic programming (RTADP) method for solving multistage capacity decision problems in a stochastic manufacturing environment, by using an exemplary three‐stage manufacturing system with recycle. The system is a moderate size queuing network, which experiences stochastic variations in demand and product yield. The dynamic capacity decision problem is formulated as a Markov decision process (MDP). The proposed RTADP method starts with a set of heuristics and learns a superior quality solution by interacting with the stochastic system via simulation. The curse‐of‐dimensionality associated with DP methods is alleviated by the adoption of several notions including “evolving set of relevant states,” for which the value function table is built and updated, “adaptive action set” for keeping track of attractive action candidates, and “nonparametric k nearest neighbor averager” for value function approximation. The performance of the learned solution is evaluated against (1) an “ideal” solution derived using a mixed integer programming (MIP) formulation, which assumes full knowledge of future realized values of the stochastic variables (2) a myopic heuristic solution, and (3) a sample path based rolling horizon MIP solution. The policy learned through the RTADP method turned out to be superior to polices of 2 and 3. © 2010 Wiley Periodicals, Inc. Naval Research Logistics 2010     

Optimal reject allowance with constant marginal production efficiency

A job shop must fulfill an order for N good items. Production is conducted in “lots,” and the number of good items in a lot can be accurately determined only after production of that lot is completed. If the number of good items falls short of the outstanding order, the shop must produce further lots, as necessary. Processes with “constant marginal production efficiency” are investigated. The revealed structure allows efficient exact computation of optimal policy. The resulting minimal cost exhibits a consistent (but not universal) pattern whereby higher quality of production is advantageous even at proportionately higher marginal cost.     

Optimal replacement policies with two or more loaded sliding standbys

This article defines optimal replacement policies for identical components performing different functions in a given system, when more than one spare part is available. The problem is first formulated for two components and any number of spare parts and the optimal replacement time y(x) at time x is found to have a certain form. Sufficient conditions are then provided for y(x) to be a constant y* for x > y*, and y(x) = x for x > y* (single-critical-number policy). Under the assumption that the optimal policies are of the single-critical-number type, the results are extended to the n-component case, and a theorem is provided that reduces the required number of critical numbers. Finally, the theory is applied to the case of the exponential and uniform failure laws, in which single-critical-number policies are optimal, and to another failure law in which they are not.     

Periodic review inventory management with contingent use of two freight modes with fixed costs

We study a stochastic inventory model of a firm that periodically orders a product from a make‐to‐order manufacturer. Orders can be shipped by a combination of two freight modes that differ in lead‐times and costs, although orders are not allowed to cross. Placing an order as well as each use of each freight mode has a fixed and a quantity proportional cost. The decision of how to allocate units between the two freight modes utilizes information about demand during the completion of manufacturing. We derive the optimal freight mode allocation policy, and show that the optimal policy for placing orders is not an (s,S) policy in general. We provide tight bounds for the optimal policy that can be calculated by solving single period problems. Our analysis enables insights into the structure of the optimal policy specifying the conditions under which it simplifies to an (s,S) policy. We characterize the best (s,S) policy for our model, and through extensive numerical investigation show that its performance is comparable with the optimal policy in most cases. Our numerical study also sheds light on the benefits of the dual freight model over the single freight models. © 2011 Wiley Periodicals, Inc. Naval Research Logistics, 2011     

An examination of the effects of the criterion functional on optimal fire-support policies

This paper examines the dependence of the structure of optimal time-sequential fire-support policies on the quantification of military objectives by considering four specific problems, each corresponding to a different quantification of objectives (i.e. criterion functional). We consider the optimal time-sequential allocation of supporting fires during the “approach to contact” of friendly infantry against enemy defensive positions. The combat dynamics are modelled by deterministic Lanchester-type equations of warfare, and the optimal fire-support policy for each one-sided combat optimization problem is developed via optimal control theory. The problems are all nonconvex, and local optima are a particular difficulty in one of them. For the same combat dynamics, the splitting of supporting fires between two enemy forces in any optimal policy (i.e. the optimality of singular subarcs) is shown to depend only on whether the terminal payoff reflects the objective of attaining an “overall” military advantage or a “local” one. Additionally, switching times for changes in the ranking of target priorities are shown to be different (sometimes significantly) when the decision criterion is the difference and the ratio of the military worths (computed according to linear utilities) of total infantry survivors and also the difference and the ratio of the military worths (computed according to linear utilities) of total infantry survivors and also the difference and the ratio of the military worths of the combatants' total infantry losses. Thus, the optimal fire-support policy for this attack scenario is shown to be significantly influenced by the quantification of military objectives.     

Scheduling stochastic jobs with asymmetric earliness and tardiness penalties

We consider a stochastic counterpart of the well-known earliness-tardiness scheduling problem with a common due date, in which n stochastic jobs are to be processed on a single machine. The processing times of the jobs are independent and normally distributed random variables with known means and known variances that are proportional to the means. The due dates of the jobs are random variables following a common probability distribution. The objective is to minimize the expectation of a weighted combination of the earliness penalty, the tardiness penalty, and the flow-time penalty. One of our main results is that an optimal sequence for the problem must be V-shaped with respect to the mean processing times. Other characterizations of the optimal solution are also established. Two algorithms are proposed, which can generate optimal or near-optimal solutions in pseudopolynomial time. The proposed algorithms are also extended to problems where processing times do not satisfy the assumption in the model above, and are evaluated when processing times follow different probability distributions, including general normal (without the proportional relation between variances and means), uniform, Laplace, and exponential. © 1997 John Wiley & Sons, Inc. Naval Research Logistics 44, 531–557, 1997.     

On optimal aiming to hit a linear target

Typically weapon systems have an inherent systematic error and a random error for each round, centered around its mean point of impact. The systematic error is common to all aimings. Assume such a system for which there is a preassigned amount of ammunition of n rounds to engage a given target simultaneously, and which is capable of administering their fire with individual aiming points (allowing “offsets”). The objective is to determine the best aiming points for the system so as to maximize the probability of hitting the target by at least one of the n rounds. In this paper we focus on the special case where the target is linear (one‐dimensional) and there are no random errors. We prove that as long as the aiming error is symmetrically distributed and possesses one mode at zero, the optimal aiming is independent of the particular error distribution, and we specify the optimal aiming points. Possible extensions are further discussed, as well as civilian applications in manufacturing, radio‐electronics, and detection. © 1999 John Wiley & Sons, Inc. Naval Research Logistics 46: 323–333, 1999     

Asymptotic distribution of some multivariate “success run” renewal processes,applied to a 2-i.i.d. unit repairable system

If the probability of “failure” in a multivariate renewal process of the “success run” type is very small, then if certain conditions are imposed on the components of the renewals, the joint distribution of their total durations is approximately exponential with all mass along one line. This result is applied to a 2-i.i.d. unit repairable system of the “1 out of 2:G, Cold Standby” type.     

Monotone optimal preventive maintenance policies for stochastically failing equipment

This paper examines various models for maintenance of a machine operating subject to stochastic deterioration. Three alternative models are presented for the deterioration process. For each model, in addition to the replacement decision, the option exists of performing preventive maintenance. The effect of this maintenance is to “slow” the deterioration process. With an appropriate reward structure imposed on the processes, the models are formulated as continuous time Markov decision processes. the optimality criterion being the maximization of expected discounted reward earned over an infinite time horizon. For each model conditions are presented under which the optimal maintenance policy exhibits the following monotonic structure. First, there exists a control limit rule for replacement. That is, there exists a number i* such that if the state of machine deterioration exceeds i* the optimal policy replaces the machine by a new machine. Secondly, prior to replacement the optimal level of preventive maintenance is a nonincreasing function of the state of machine deterioration. The conditions which guarantee this result have a cost/benefit interpretation.     

Quasi-separable utility functions

This paper is concerned with a method for the assessment of utility functions of multi-numeraire consequences. It is proven that given von Neumann and Morgenstern's axioms of “rational behavior” and two additional assumptions, the utility function for (x, y) consequences can be written as U(x, y) = U_x(x) + U_y(y) + KU_x(x) U_y(y). K is a constant that must be evaluated empirically. This form shall be designated as a quasi-separable utility function. It is more general than the separable utility function and is shown to be nearly as easy to use. Implications and ramifications of such a utility function and its requisite assumptions are discussed. A technique for practical application of this work is presented.     

Search and the introduction of improved technologies

Modeling R&D as standard sequential search, we consider a monopolist who can implement a sequence of technological discoveries during the technology search process: he earns revenue on his installed technology while he engages in R&D to find improved technology. What is not standard is that he has a finite number of opportunities to introduce improved technology. We show that his optimal policy is characterized by thresholds ξ_i(x): introduce the newly found technology if and only if it exceeds ξ_i(x) when x is the state of the currently installed technology and i is the number of remaining introductions allowed. We also analyze a nonstationary learning‐by‐doing model in which the monopolist's experience in implementing new technologies imparts increased capability in generating new technologies. Because this nonstationary model is not in the class of monotone stopping problems, a number of surprising results hold and several seemingly obvious properties of the stationary model no longer hold. © 2011 Wiley Periodicals, Inc. Naval Research Logistics, 2011     

Assembly of systems having maximum reliability

The first problem considered in this paper is concerned with the assembly of independent components into parallel systems so as to maximize the expected number of systems that perform satisfactorily. Associated with each component is a probability of it performing successfully. It is shown that an optimal assembly is obtained if the reliability of each assembled system can be made equal. If such equality is not attainable, then bounds are given so that the maximum expected number of systems that perform satisfactorily will lie within these stated bounds; the bounds being a function of an arbitrarily chosen assembly. An improvement algorithm is also presented. A second problem treated is concerned with the optimal design of a system. Instead of assembling given units, there is an opportunity to “control” their quality, i.e., the manufacturer is able to fix the probability, p, of a unit performing successfully. However, his resources, are limited so that a constraint is imposed on these probabilities. For (1) series systems, (2) parallel systems, and (3) k out of n systems, results are obtained for finding the optimal p's which maximize the reliability of a single system, and which maximize the expected number of systems that perform satisfactorily out of a total assembly of J systems.     

Flowshop/no-idle or no-wait scheduling to minimize the sum of completion times

This paper deals with flowshop/sum of completion times scheduling problems, working under a “no-idle” or a “no-wait” constraint, the former prescribes for the machines to work continuously without idle intervals and the latter for the jobs to be processed continuously without waiting times between consecutive machines. Under either of the constraints the problem is unary NP-Complete for two machines. We prove some properties of the optimal schedule for n/2/F, no-idle/σC_i. For n/m/P, no-idle/σC_i, and n/m/P, no-wait/σC_i, with an increasing or decreasing series of dominating machines, we prove theorems that are the basis for polynomial bounded algorithms. All theorems are demonstrated numerically.     

Nonrobustness of the normal approximation of lead‐time demand in a (Q,R) system

For computing an optimal (Q, R) or kindred inventory policy, the current literature provides mixed signals on whether or when it is safe to approximate a nonnormal lead‐time‐demand (“LTD”) distribution by a normal distribution. The first part of this paper examines this literature critically to justify why the issue warrants further investigations, while the second part presents reliable evidence showing that the system‐cost penalty for using the normal approximation can be quite serious even when the LTD‐distribution's coefficient of variation is quite low—contrary to the prevalent view of the literature. We also identify situations that will most likely lead to large system‐cost penalty. Our results indicate that, given today's technology, it is worthwhile to estimate an LTD‐distribution's shape more accurately and to compute optimal inventory policies using statistical distributions that more accurately reflect the LTD‐distributions' actual shapes. © 2003 Wiley Periodicals, Inc. Naval Research Logistics, 2003     

Dynamic inventory management with cash flow constraints

In this article, we consider a classic dynamic inventory control problem of a self‐financing retailer who periodically replenishes its stock from a supplier and sells it to the market. The replenishment decisions of the retailer are constrained by cash flow, which is updated periodically following purchasing and sales in each period. Excess demand in each period is lost when insufficient inventory is in stock. The retailer's objective is to maximize its expected terminal wealth at the end of the planning horizon. We characterize the optimal inventory control policy and present a simple algorithm for computing the optimal policies for each period. Conditions are identified under which the optimal control policies are identical across periods. We also present comparative statics results on the optimal control policy. © 2008 Wiley Periodicals, Inc. Naval Research Logistics 2008