Optimal maintenance policies for a safety‐critical system and its deteriorating sensor

We consider the integrated problem of optimally maintaining an imperfect, deteriorating sensor and the safety‐critical system it monitors. The sensor's costless observations of the binary state of the system become less informative over time. A costly full inspection may be conducted to perfectly discern the state of the system, after which the system is replaced if it is in the out‐of‐control state. In addition, a full inspection provides the opportunity to replace the sensor. We formulate the problem of adaptively scheduling full inspections and sensor replacements using a partially observable Markov decision process (POMDP) model. The objective is to minimize the total expected discounted costs associated with system operation, full inspection, system replacement, and sensor replacement. We show that the optimal policy has a threshold structure and demonstrate the value of coordinating system and sensor maintenance via numerical examples. © 2017 Wiley Periodicals, Inc. Naval Research Logistics 64: 399–417, 2017     

Model parameter estimation and residual life prediction for a partially observable failing system

We consider a partially observable degrading system subject to condition monitoring and random failure. The system's condition is categorized into one of three states: a healthy state, a warning state, and a failure state. Only the failure state is observable. While the system is operational, vector data that is stochastically related to the system state is obtained through condition monitoring at regular sampling epochs. The state process evolution follows a hidden semi‐Markov model (HSMM) and Erlang distribution is used for modeling the system's sojourn time in each of its operational states. The Expectation‐maximization (EM) algorithm is applied to estimate the state and observation parameters of the HSMM. Explicit formulas for several important quantities for the system residual life estimation such as the conditional reliability function and the mean residual life are derived in terms of the posterior probability that the system is in the warning state. Numerical examples are presented to demonstrate the applicability of the estimation procedure and failure prediction method. A comparison results with hidden Markov modeling are provided to illustrate the effectiveness of the proposed model. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 190–205, 2015     

Optimal policies for stochastic clearing systems with time-dependent delay penalties

We study stochastic clearing systems with a discrete-time Markovian input process, and an output mechanism that intermittently and instantaneously clears the system partially or completely. The decision to clear the system depends on both quantities and delays of outstanding inputs. Clearing the system incurs a fixed cost, and outstanding inputs are charged a delay penalty, which is a general increasing function of the quantities and delays of individual inputs. By recording the quantities and delays of outstanding inputs in a sequence, we model the clearing system as a tree-structured Markov decision process over both a finite and infinite horizon. We show that the optimal clearing policies, under realistic conditions, are of the on-off type or the threshold type. Based on the characterization of the optimal policies, we develop efficient algorithms to compute parameters of the optimal policies for such complex clearing systems for the first time. We conduct a numerical analysis on the impact of the nonlinear delay penalty cost function, the comparison of the optimal policy and the classical hybrid policy (ie, quantity and age thresholds), and the impact of the state of the input process. Our experiments demonstrate that (a) the classical linear approximation of the cost function can lead to significant performance differences; (b) the classical hybrid policy may perform poorly (as compared to the optimal policies); and (c) the consideration of the state of the input process makes significant improvement in system performance.     

Minimizing the false alarm rate in systems with transient abnormality

We consider a stochastic partially observable system that can switch between a normal state and a transient abnormal state before entering a persistent abnormal state. Only the persistent abnormal state requires alarms. The transient and persistent abnormal states may be similar in appearance, which can result in excess false alarms. We propose a partially observable Markov decision process model to minimize the false alarm rate, subject to a given upper bound on the expected alarm delay time. The cost parameter is treated as the Lagrange multiplier, which can be estimated from the bound of the alarm delay. We show that the optimal policy has a control‐limit structure on the probability of persistent abnormality, and derive closed‐form bounds for the control limit and present an algorithm to specify the Lagrange multiplier. We also study a specialized model where the transient and persistent abnormal states have the same observation distribution, in which case an intuitive “watchful‐waiting” policy is optimal. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 320–334, 2016     

The LP/POMDP marriage: Optimization with imperfect information

A new technique for solving large‐scale allocation problems with partially observable states and constrained action and observation resources is introduced. The technique uses a master linear program (LP) to determine allocations among a set of control policies, and uses partially observable Markov decision processes (POMDPs) to determine improving policies using dual prices from the master LP. An application is made to a military problem where aircraft attack targets in a sequence of stages, with information acquired in one stage being used to plan attacks in the next. © 2000 John Wiley & Sons, Inc., Naval Research Logistics 47: 607–619, 2000     

Structured maintenance policies on interior sample paths

We examine the problem of adaptively scheduling perfect observations and preventive replacements for a multi‐state, Markovian deterioration system with silent failures such that total expected discounted cost is minimized. We model this problem as a partially observed Markov decision process and show that the structural properties of the optimal policy hold for certain non‐extreme sample paths. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

Technical note: Reexamination of all‐or‐none inspection policies in a supply chain with endogenous product quality

This note studies the optimal inspection policies in a supply chain in which a manufacturer purchases components from a supplier but has no direct control of component quality. The manufacturer uses an inspection policy and a damage cost sharing contract to encourage the supplier to improve the component quality. We find that all‐or‐none inspection policies are optimal for the manufacturer if the supplier's share of the damage cost is larger than a threshold; otherwise, the manufacturer should inspect a fraction of a batch. © 2008 Wiley Periodicals, Inc. Naval Research Logistics, 2008     

On‐line parameter estimation for a partially observable system subject to random failure

In this paper, we study the on‐line parameter estimation problem for a partially observable system subject to deterioration and random failure. The state of the system evolves according to a continuous time homogeneous Markov process with a finite state space. The system state is not observable, except for the failure state. The information related to the system state is available at discrete times through inspections. A recursive maximum likelihood (RML) algorithm is proposed for the on‐line parameter estimation of the model. The RML algorithm proposed in the paper is considerably faster and easier to apply than other RML algorithms in the literature, because it does not require projection into the constraint domain and calculation of the gradient on the surface of the constraint manifolds. The algorithm is illustrated by an example using real vibration data. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006     

Maintenance and flight scheduling of low observable aircraft

In this article, we focus on relatively new maintenance and operational scheduling challenges that are faced by the United States Air Force concerning low‐observable (LO) or stealth aircraft. The LO capabilities of an aircraft degrade stochastically as it flies, making it difficult to make maintenance scheduling decisions. Maintainers can address these damages, but must decide, which aircraft should be put into maintenance, and for how long. Using data obtained from an active duty Air Force F‐22 wing and interviews with Air Force maintainers and program specialists, we model this problem as a generalization of the well‐known restless multiarmed bandit superprocess. Specifically, we use an extension of the traditional model to allow for actions that require varying lengths of time, and generate two separate index policies from a single model; one for maintenance actions and one for the flying action. These index policies allow maintenance schedulers to intuitively, quickly, and effectively rank a fleet of aircraft based on each aircraft's LO status and decide, which aircraft should enter into LO maintenance and for how long, and which aircraft should be used to satisfy daily sortie requirements. Finally, we present extensive data‐driven, detailed simulation results, where we compare the performance of the index policies against policies currently used by the Air Force, as well as some other possible more naive heuristics. The results indicate that the index policies significantly outperform existing policies in terms of fully mission capable (FMC) rates. In particular, the experiments highlight the importance of coordinated maintenance and flying decisions. © 2015 Wiley Periodicals, Inc. 62:60–80, 2015     

Some estimates of reliability using interference theory

According to “interference theory” of reliability, a component fails if the maximum stress exceeds the component's strength. Assuming that both these quantities are random and their distributions are normal, we obtain in this paper some point and interval estimates of reliability when the stress distribution is known and a few observations exist on component strengths.     

Optimal FCFS allocation rules for periodic‐review assemble‐to‐order systems

In Assemble‐To‐Order (ATO) systems, situations may arise in which customer demand must be backlogged due to a shortage of some components, leaving available stock of other components unused. Such unused component stock is called remnant stock. Remnant stock is a consequence of both component ordering decisions and decisions regarding allocation of components to end‐product demand. In this article, we examine periodic‐review ATO systems under linear holding and backlogging costs with a component installation stock policy and a First‐Come‐First‐Served (FCFS) allocation policy. We show that the FCFS allocation policy decouples the problem of optimal component allocation over time into deterministic period‐by‐period optimal component allocation problems. We denote the optimal allocation of components to end‐product demand as multimatching. We solve the multi‐matching problem by an iterative algorithm. In addition, an approximation scheme for the joint replenishment and allocation optimization problem with both upper and lower bounds is proposed. Numerical experiments for base‐stock component replenishment policies show that under optimal base‐stock policies and optimal allocation, remnant stock holding costs must be taken into account. Finally, joint optimization incorporating optimal FCFS component allocation is valuable because it provides a benchmark against which heuristic methods can be compared. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 158–169, 2015     

Solving partially observable agent‐intruder games with an application to border security problems

In this paper, we introduce partially observable agent‐intruder games (POAIGs). These games model dynamic search games on graphs between security forces (an agent) and an intruder given possible (border) entry points and high value assets that require protection. The agent faces situations with dynamically changing, partially observable information about the state of the intruder and vice versa. The agent may place sensors at selected locations, while the intruder may recruit partners to observe the agent's movement. We formulate the problem as a two‐person zero‐sum game, and develop efficient algorithms to compute each player's optimal strategy. The solution to the game will help the agent choose sensor locations and design patrol routes that can handle imperfect information. First, we prove the existence of ?‐optimal strategies for POAIGs with an infinite time horizon. Second, we introduce a Bayesian approximation algorithm to identify these ?‐optimal strategies using belief functions that incorporate the imperfect information that becomes available during the game. For the solutions of large POAIGs with a finite time horizon, we use a solution method common to extensive form games, namely, the sequence form representation. To illustrate the POAIGs, we present several examples and numerical results.     

Revenue management under randomly evolving economic conditions

We consider a dynamic pricing model in which the instantaneous rate of the demand arrival process is dependent on not only the current price charged by the concerned firm, but also the present state of the world. While reflecting the current economic condition, the state evolves in a Markovian fashion. This model represents the real‐life situation in which the sales season is relatively long compared to the fast pace at which the outside environment changes. We establish the value of being better informed on the state of the world. When reasonable monotonicity conditions are met, we show that better present economic conditions will lead to higher prices. Our computational study is partially calibrated with real data. It demonstrates that the benefit of heeding varying economic conditions is on par with the value of embracing randomness in the demand process. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 66:73–89,2019     

Life distribution analysis based on Lévy subordinators for degradation with random jumps

For a component or a system subject to stochastic degradation with sporadic jumps that occur at random times and have random sizes, we propose to model the cumulative degradation with random jumps using a single stochastic process based on the characteristics of Lévy subordinators, the class of nondecreasing Lévy processes. Based on the inverse Fourier transform, we derive a new closed‐form reliability function and probability density function for lifetime, represented by Lévy measures. The reliability function derived using the traditional convolution approach for common stochastic models such as gamma degradation process with random jumps, is revealed to be a special case of our general model. Numerical experiments are used to demonstrate that our model performs well for different applications, when compared with the traditional convolution method. More importantly, it is a general and useful tool for life distribution analysis of stochastic degradation with random jumps in multidimensional cases. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 483–492, 2015     

An optimal structured policy for maintenance of partially observable aircraft engine components

This paper considers the maintenance of aircraft engine components that are subject to stress. We model the deterioration process by means of the cumulative jump process representation of crack growth. However, because in many cases cracks are not easily observable, maintenance decisions must be made on the basis of other information. We incorporate stress information collected via sensors into the scheduling decision process by means of a partially observable Markov decision process model. Using this model, we demonstrate the optimality of structured maintenance policies, which support practical maintenance schedules. © 1998 John Wiley & Sons, Inc. Naval Research Logistics 45: 335–352, 1998     

Optimal and asymptotically optimal policies for assemble‐to‐order n‐ and W‐systems

We consider two specially structured assemble‐to‐order (ATO) systems—the N‐ and W‐systems—under continuous review, stochastic demand, and nonidentical component replenishment leadtimes. Using a hybrid approach that combines sample‐path analysis, linear programming, and the tower property of conditional expectation, we characterize the optimal component replenishment policy and common‐component allocation rule, present comparative statics of the optimal policy parameters, and show that some commonly used heuristic policies can lead to significant optimality loss. The optimality results require certain symmetry in the cost parameters. In the absence of this symmetry, we show that, for systems with high demand volume, the asymptotically optimal policy has essentially the same structure; otherwise, the optimal policies have no clear structure. For these latter systems, we develop heuristic policies and show their effectiveness. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 62: 617–645, 2015     

A produce‐to‐stock system with advance demand information and secondary customers

We consider a manufacturer (i.e., a capacitated supplier) that produces to stock and has two classes of customers. The primary customer places orders at regular intervals of time for a random quantity, while the secondary customers request a single item at random times. At a predetermined time the manufacturer receives advance demand information regarding the order size of the primary customer. If the manufacturer is not able to fill the primary customer's demand, there is a penalty. On the other hand, serving the secondary customers results in additional profit; however, the manufacturer can refuse to serve the secondary customers in order to reserve inventory for the primary customer. We characterize the manufacturer's optimal production and stock reservation policies that maximize the manufacturer's discounted profit and the average profit per unit time. We show that these policies are threshold‐type policies, and these thresholds are monotone with respect to the primary customer's order size. Using a numerical study we provide insights into how the value of information is affected by the relative demand size of the primary and secondary customers. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

Requirements planning for modular products

The focus of this paper is on determining the requirements of different component options of a modular end‐product in an uncertain environment. We explicitly model two distinct sources of uncertainty: stochastic end‐product demand and unknown market proportions for the different product options available. Our cost minimizing model focuses on determining the optimal requirements policies for component options that meet a pre‐set service level. We show that simple common‐sense requirements policies are not generally optimal; there is a non‐linear connection between service level and component requirements that is hard to characterize without a detailed analysis. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006     

Markov decision processes with restricted observations: Finite horizon case

In this article we consider a Markov decision process subject to the constraints that result from some observability restrictions. We assume that the state of the Markov process under consideration is unobservable. The states are grouped so that the group that a state belongs to is observable. So, we want to find an optimal decision rule depending on the observable groups instead of the states. This means that the same decision applies to all the states in the same group. We prove that a deterministic optimal policy exists for the finite horizon. An algorithm is developed to compute policies minimizing the total expected discounted cost over a finite horizon. © 1997 John Wiley & Sons, Inc. Naval Research Logistics 44 : 439–456, 1997     

Layers of Experiments with Adaptive Combined Design

In the field of nanofabrication, engineers often face unique challenges in resource‐limited experimental budgets, the sensitive nature of process behavior with respect to controllable variables, and highly demanding tolerance requirements. To effectively overcome these challenges, this article proposes a methodology for a sequential design of experiments through batches of experimental runs, aptly named Layers of Experiments with Adaptive Combined Design (LoE/ACD). In higher layers, where process behavior is less understood, experimental regions cover more design space and data points are more spread out. In lower layers, experimental regions are more focused to improve understanding of process sensitivities in a local, data‐rich environment. The experimental design is a combination of a space‐filling and an optimal design with a tuning parameter that is dependent on the amount of information accumulated over the various layers. The proposed LoE/ACD method is applied to optimize a carbon dioxide (epet‐CO2) assisted deposition process for fabricating silver nanoparticles with pressure and temperature variables. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 127–142, 2015