Optimal control of a production‐inventory system with both backorders and lost sales

We consider the optimal control of a production inventory‐system with a single product and two customer classes where items are produced one unit at a time. Upon arrival, customer orders can be fulfilled from existing inventory, if there is any, backordered, or rejected. The two classes are differentiated by their backorder and lost sales costs. At each decision epoch, we must determine whether or not to produce an item and if so, whether to use this item to increase inventory or to reduce backlog. At each decision epoch, we must also determine whether or not to satisfy demand from a particular class (should one arise), backorder it, or reject it. In doing so, we must balance inventory holding costs against the costs of backordering and lost sales. We formulate the problem as a Markov decision process and use it to characterize the structure of the optimal policy. We show that the optimal policy can be described by three state‐dependent thresholds: a production base‐stock level and two order‐admission levels, one for each class. The production base‐stock level determines when production takes place and how to allocate items that are produced. This base‐stock level also determines when orders from the class with the lower shortage costs (Class 2) are backordered and not fulfilled from inventory. The order‐admission levels determine when orders should be rejected. We show that the threshold levels are monotonic (either nonincreasing or nondecreasing) in the backorder level of Class 2. We also characterize analytically the sensitivity of these thresholds to the various cost parameters. Using numerical results, we compare the performance of the optimal policy against several heuristics and show that those that do not allow for the possibility of both backordering and rejecting orders can perform poorly.© 2010 Wiley Periodicals, Inc. Naval Research Logistics 2010     

Analysis of artillery shoot‐and‐scoot tactics

Firing multiple artillery rounds from the same location has two main benefits: a high rate of fire at the enemy and improved accuracy as the shooter's aim adjusts to previous rounds. However, firing repeatedly from the same location carries significant risk that the enemy will detect the artillery's location. Therefore, the shooter may periodically move locations to avoid counter‐battery fire. This maneuver is known as the shoot‐and‐scoot tactic. This article analyzes the shoot‐and‐scoot tactic for a time‐critical mission using Markov models. We compute optimal move policies and develop heuristics for more complex and realistic settings. Spending a reasonable amount of time firing multiple shots from the same location is often preferable to moving immediately after firing an initial salvo. Moving frequently reduces risk to the artillery, but also limits the artillery's ability to inflict damage on the enemy.     

Analysis of a finite‐buffer bulk‐service queue with discrete‐Markovian arrival process: D‐MAP/Ga,b/1/N

Discrete‐time queues with D‐MAP arrival process are more useful in modeling and performance analysis of telecommunication networks based on the ATM environment. This paper analyzes a finite‐buffer discrete‐time queue with general bulk‐service rule, wherein the arrival process is D‐MAP and service times are arbitrarily and independently distributed. The distributions of buffer contents at various epochs (departure, random, and prearrival) have been obtained using imbedded Markov chain and supplementary variable methods. Finally, some performance measures such as loss probability and average delay are discussed. Numerical results are also presented in some cases. © 2003 Wiley Periodicals, Inc. Naval Research Logistics 50: 345–363, 2003.     

Dynamic server assignment policies for assembly‐type queues with flexible servers

We seek dynamic server assignment policies in finite‐capacity queueing systems with flexible and collaborative servers, which involve an assembly and/or a disassembly operation. The objective is to maximize the steady‐state throughput. We completely characterize the optimal policy for a Markovian system with two servers, two feeder stations, and instantaneous assembly and disassembly operations. This optimal policy allocates one server per station unless one of the stations is blocked, in which case both servers work at the unblocked station. For Markovian systems with three stations and instantaneous assembly and/or disassembly operations, we consider similar policies that move a server away from his/her “primary” station only when that station is blocked or starving. We determine the optimal assignment of each server whose primary station is blocked or starving in systems with three stations and zero buffers, by formulating the problem as a Markov decision process. Using this optimal assignment, we develop heuristic policies for systems with three or more stations and positive buffers, and show by means of a numerical study that these policies provide near‐optimal throughput. Furthermore, our numerical study shows that these policies developed for assembly‐type systems also work well in tandem systems. © 2008 Wiley Periodicals, Inc. Naval Research Logistics, 2008     

Dynamic coordination of production planning and sales admission control in the presence of a spot market

We consider the decision‐making problem of dynamically scheduling the production of a single make‐to stock (MTS) product in connection with the product's concurrent sales in a spot market and a long‐term supply channel. The spot market is run by a business to business (B2B) online exchange, whereas the long‐term channel is established by a structured contract. The product's price in the spot market is exogenous, evolves as a continuous time Markov chain, and affects demand, which arrives sequentially as a Markov‐modulated Poisson process (MMPP). The manufacturer is obliged to fulfill demand in the long‐term channel, but is able to rein in sales in the spot market. This is a significant strategic decision for a manufacturer in entering a favorable contract. The profitability of the contract must be evaluated by optimal performance. The current problem, therefore, arises as a prerequisite to exploring contracting strategies. We reveal that the optimal strategy of coordinating production and sales is structured by the spot price dependent on the base stock and sell‐down thresholds. Moreover, we can exploit the structural properties of the optimal strategy to conceive an efficient algorithm. © 2010 Wiley Periodicals, Inc. Naval Research Logistics, 2010     

Optimal maintenance policies for a safety‐critical system and its deteriorating sensor

We consider the integrated problem of optimally maintaining an imperfect, deteriorating sensor and the safety‐critical system it monitors. The sensor's costless observations of the binary state of the system become less informative over time. A costly full inspection may be conducted to perfectly discern the state of the system, after which the system is replaced if it is in the out‐of‐control state. In addition, a full inspection provides the opportunity to replace the sensor. We formulate the problem of adaptively scheduling full inspections and sensor replacements using a partially observable Markov decision process (POMDP) model. The objective is to minimize the total expected discounted costs associated with system operation, full inspection, system replacement, and sensor replacement. We show that the optimal policy has a threshold structure and demonstrate the value of coordinating system and sensor maintenance via numerical examples. © 2017 Wiley Periodicals, Inc. Naval Research Logistics 64: 399–417, 2017     

Optimal admission pricing policies for M/Ek/1 queues

This paper extends the Low-Lippman M/M/1 model to the case of Gamma service times. Specifically, we have a queue in which arrivals are Poisson, service time is Gamma-distributed, and the arrival rate to the system is subject to setting an admission fee p. The arrival rate λ(p) is non-increasing in p. We prove that the optimal admission fee p^* is a non-decreasing function of the customer work load on the server. The proof is for an infinite capacity queue and holds for the infinite horizon continuous time Markov decision process. In the special case of exponential service time, we extend the Low-Lippman model to include a state-dependent service rate and service cost structure (for finite or infinite time horizon and queue capacity). Relatively recent dynamic programming techniques are employed throughout the paper. Due to the large class of functions represented by the Gamma family, the extension is of interest and utility.     

Dual‐ascent and primal heuristics for production‐assembly‐distribution system design

This article proposes two dual‐ascent algorithms and uses each in combination with a primal drop heuristic embedded within a branch and bound framework to solve the uncapacitated production assembly distribution system (i.e., supply chain) design problem, which is formulated as a mixed integer program. Computational results indicate that one approach, which combines primal drop and dual‐ascent heuristics, can solve instances within reasonable time and prescribes solutions with gaps between the primal and dual solution values that are less than 0.15%, an efficacy suiting it for actual large‐scale applications. © 2012 Wiley Periodicals, Inc. Naval Research Logistics, 2013     

Simultaneous determination of production and maintenance schedules using in‐line equipment condition and yield information

In many manufacturing environments, equipment condition has a significant impact on product quality, or yield. This paper presents a semi‐Markov decision process model of a single‐stage production system with multiple products and multiple maintenance actions. The model simultaneously determines maintenance and production schedules, accounting for the fact that equipment condition affects the yield of each product differently. It extends earlier work by allowing the expected time between decision epochs to vary by both action and machine state, by allowing multiple maintenance actions, and by treating the outcome of maintenance as less than certain. Sufficient conditions are developed that ensure the monotonicity of both the optimal production and maintenance actions. While the maintenance conditions closely resemble previously studied conditions for this type of problem, the production conditions represent a significant departure from earlier results. The simultaneous solution method is compared to an approach commonly used in industry, where the maintenance and production problems are treated independently. Solving more than one thousand test problems confirms that the combination of both features of the model—accounting for product differences and solving the problems simultaneously—has a significant impact on performance. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2008     

On a Markov‐modulated shock and wear process

We present transient and asymptotic reliability indices for a single‐unit system that is subject to Markov‐modulated shocks and wear. The transient results are derived from the (transform) solution of an integro‐differential equation describing the joint distribution of the cumulative degradation process and the state of the modulating process. Additionally, we prove the asymptotic normality of a properly centered and time‐scaled version of the cumulative degradation at time t. This asymptotic result leads to a simple normal approximation for a properly centered and space‐scaled version of the systes lifetime distribution. Two numerical examples illustrate the quality of the normal approximation. © 2009 Wiley Periodicals, Inc. Naval Research Logistics 2009     

Heuristics for the multi‐resource generalized assignment problem

The well‐known generalized assignment problem (GAP) involves the identification of a minimum‐cost assignment of tasks to agents when each agent is constrained by a resource in limited supply. The multi‐resource generalized assignment problem (MRGAP) is the generalization of the GAP in which there are a number of different potentially constraining resources associated with each agent. This paper explores heuristic procedures for the MRGAP. We first define a three‐phase heuristic which seeks to construct a feasible solution to MRGAP and then systematically attempts to improve the solution. We then propose a modification of the heuristic for the MRGAP defined previously by Gavish and Pirkul. The third procedure is a hybrid heuristic that combines the first two heuristics, thus capturing their relative strengths. We discuss extensive computational experience with the heuristics. The hybrid procedure is seen to be extremely effective in solving MRGAPs, generating feasible solutions to more than 99% of the test problems and consistently producing near‐optimal solutions. © 2001 John Wiley & Sons, Inc. Naval Research Logistics 48: 468–483, 2001     

On the first come–first served rule in multi‐echelon inventory control

A two‐echelon distribution inventory system with a central warehouse and a number of retailers is considered. The retailers face stochastic demand and replenish from the warehouse, which, in turn, replenishes from an outside supplier. The system is reviewed continuously and demands that cannot be met directly are backordered. Standard holding and backorder costs are considered. In the literature on multi‐echelon inventory control it is standard to assume that backorders at the warehouse are served according to a first come–first served policy (FCFS). This allocation rule simplifies the analysis but is normally not optimal. It is shown that the FCFS rule can, in the worst case, lead to an asymptotically unbounded relative cost increase as the number of retailers approaches infinity. We also provide a new heuristic that will always give a reduction of the expected costs. A numerical study indicates that the average cost reduction when using the heuristic is about two percent. The suggested heuristic is also compared with two existing heuristics. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

Optimal control of an M/G/1 queue with impatient priority customers

An optimal operating policy is characterized for the infinite‐horizon average‐cost case of a single server queueing control problem. The server may be turned on at arrival epochs or off at departure epochs. Two classes of customers, each of them arriving according to an independent Poisson processes, are considered. An arriving 1‐customer enters the system if the server is turned on upon his arrival, or if the server is on and idle. In the former case, the 1‐customer is selected for service ahead of those customers waiting in the system; otherwise he leaves the system immediately. 2‐Customers remain in the system until they complete their service requirements. Under a linear cost structure, this paper shows that a stationary optimal policy exists such that either (1) leaves the server on at all times, or (2) turns the server off when the system is empty. In the latter case, we show that the stationary optimal policy is a threshold strategy, this feature being commonplace in most of priority queueing systems and inventory models. However, the optimal policy in our model is determined by two thresholds instead of one. © 2001 John Wiley & Sons, Inc. Naval Research Logistics 48: 201–209, 2001     

Maintaining systems with heterogeneous spare parts

We consider the problem of optimally maintaining a stochastically degrading, single‐unit system using heterogeneous spares of varying quality. The system's failures are unannounced; therefore, it is inspected periodically to determine its status (functioning or failed). The system continues in operation until it is either preventively or correctively maintained. The available maintenance options include perfect repair, which restores the system to an as‐good‐as‐new condition, and replacement with a randomly selected unit from the supply of heterogeneous spares. The objective is to minimize the total expected discounted maintenance costs over an infinite time horizon. We formulate the problem using a mixed observability Markov decision process (MOMDP) model in which the system's age is observable but its quality must be inferred. We show, under suitable conditions, the monotonicity of the optimal value function in the belief about the system quality and establish conditions under which finite preventive maintenance thresholds exist. A detailed computational study reveals that the optimal policy encourages exploration when the system's quality is uncertain; the policy is more exploitive when the quality is highly certain. The study also demonstrates that substantial cost savings are achieved by utilizing our MOMDP‐based method as compared to more naïve methods of accounting for heterogeneous spares.     

Equilibrium customer strategies and social–profit maximization in the single‐server constant retrial queue

We consider the single‐server constant retrial queue with a Poisson arrival process and exponential service and retrial times. This system has not waiting space, so the customers that find the server busy are forced to abandon the system, but they can leave their contact details. Hence, after a service completion, the server seeks for a customer among those that have unsuccessfully applied for service but left their contact details, at a constant retrial rate. We assume that the arriving customers that find the server busy decide whether to leave their contact details or to balk based on a natural reward‐cost structure, which incorporates their desire for service as well as their unwillingness to wait. We examine the customers' behavior, and we identify the Nash equilibrium joining strategies. We also study the corresponding social and profit maximization problems. We consider separately the observable case where the customers get informed about the number of customers waiting for service and the unobservable case where they do not receive this information. Several extensions of the model are also discussed. © 2011 Wiley Periodicals, Inc. Naval Research Logistics, 2011     

Optimal service rates of a service facility with perishable inventory items

In this paper we optimally control service rates for an inventory system of service facilities with perishable products. We consider a finite capacity system where arrivals are Poisson‐distributed, lifetime of items have exponential distribution, and replenishment is instantaneous. We determine the service rates to be employed at each instant of time so that the long‐run expected cost rate is minimized for fixed maximum inventory level and capacity. The problem is modelled as a semi‐Markov decision problem. We establish the existence of a stationary optimal policy and we solve it by employing linear programming. Several numerical examples which provide insight to the behavior of the system are presented. © 2002 Wiley Periodicals, Inc. Naval Research Logistics 49: 464–482, 2002; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/nav.10021     

Using a birth‐and‐death process to estimate the steady‐state distribution of a periodic queue

If the number of customers in a queueing system as a function of time has a proper limiting steady‐state distribution, then that steady‐state distribution can be estimated from system data by fitting a general stationary birth‐and‐death (BD) process model to the data and solving for its steady‐state distribution using the familiar local‐balance steady‐state equation for BD processes, even if the actual process is not a BD process. We show that this indirect way to estimate the steady‐state distribution can be effective for periodic queues, because the fitted birth and death rates often have special structure allowing them to be estimated efficiently by fitting parametric functions with only a few parameters, for example, 2. We focus on the multiserver M_t/GI/s queue with a nonhomogeneous Poisson arrival process having a periodic time‐varying rate function. We establish properties of its steady‐state distribution and fitted BD rates. We also show that the fitted BD rates can be a useful diagnostic tool to see if an M_t/GI/s model is appropriate for a complex queueing system. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 664–685, 2015     

Decentralized admission control of a queueing system: A game‐theoretic model

Consider a distributed system where many gatekeepers share a single server. Customers arrive at each gatekeeper according to independent Poisson processes with different rates. Upon arrival of a new customer, the gatekeeper has to decide whether to admit the customer by sending it to the server, or to block it. Blocking costs nothing. The gatekeeper receives a reward after a customer completes the service, and incurs a cost if an admitted customer finds a busy server and therefore has to leave the system. Assuming an exponential service distribution, we formulate the problem as an n‐person non‐zero‐sum game in which each gatekeeper is interested in maximizing its own long‐run average reward. The key result is that each gatekeeper's optimal policy is that of a threshold type regardless what other gatekeepers do. We then derive Nash equilibria and discuss interesting insights. © 2003 Wiley Periodicals, Inc. Naval Research Logistics 50: 702–718, 2003.     

Many‐server loss models with non‐poisson time‐varying arrivals

This article proposes an approximation for the blocking probability in a many‐server loss model with a non‐Poisson time‐varying arrival process and flexible staffing (number of servers) and shows that it can be used to set staffing levels to stabilize the time‐varying blocking probability at a target level. Because the blocking probabilities necessarily change dramatically after each staffing change, we randomize the time of each staffing change about the planned time. We apply simulation to show that (i) the blocking probabilities cannot be stabilized without some form of randomization, (ii) the new staffing algorithm with randomiation can stabilize blocking probabilities at target levels and (iii) the required staffing can be quite different when the Poisson assumption is dropped. © 2017 Wiley Periodicals, Inc. Naval Research Logistics 64: 177–202, 2017     

Limiting behavior of the stochastic sequential assignment problem

The stochastic sequential assignment problem (SSAP) considers how to allocate available distinct workers to sequentially arriving tasks with stochastic parameters such that the expected total reward obtained from the sequential assignments is maximized. Implementing the optimal assignment policy for the SSAP involves calculating a new set of breakpoints upon the arrival of each task (i.e., for every time period), which is impractical for large‐scale problems. This article studies two problems that are concerned with obtaining stationary policies, which achieve the optimal expected reward per task as the number of tasks approaches infinity. The first problem considers independent and identically distributed (IID) tasks with a known distribution function, whereas in the second problem tasks are derived from r different unobservable distributions governed by an ergodic Markov chain. The convergence rate of the expected reward per task to the optimal value is also obtained for both problems. © 2013 Wiley Periodicals, Inc. Naval Research Logistics, 2013