Optimal replacement of continuously degrading systems in partially observed environments

Motivated by wind energy applications, we consider the problem of optimally replacing a stochastically degrading component that resides and operates in a partially observable environment. The component's rate of degradation is modulated by the stochastic environment process, and the component fails when it is accumulated degradation first reaches a fixed threshold. Assuming periodic inspection of the component, the objective is to minimize the long‐run average cost per unit time of performing preventive and reactive replacements for two distinct cases. The first case examines instantaneous replacements and fixed costs, while the second considers time‐consuming replacements and revenue losses accrued during periods of unavailability. Formulated and solved are mixed state space, partially observable Markov decision process models, both of which reveal the optimality of environment‐dependent threshold policies with respect to the component's cumulative degradation level. Additionally, it is shown that for each degradation value, a threshold policy with respect to the environment belief state is optimal if the environment alternates between two states. The threshold policies are illustrated by way of numerical examples using both synthetic and real wind turbine data. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 395–415, 2015     

Monitoring poisson count data with probability control limits when sample sizes are time varying

This article considers the problem of monitoring Poisson count data when sample sizes are time varying without assuming a priori knowledge of sample sizes. Traditional control charts, whose control limits are often determined before the control charts are activated, are constructed based on perfect knowledge of sample sizes. In practice, however, future sample sizes are often unknown. Making an inappropriate assumption of the distribution function could lead to unexpected performance of the control charts, for example, excessive false alarms in the early runs of the control charts, which would in turn hurt an operator's confidence in valid alarms. To overcome this problem, we propose the use of probability control limits, which are determined based on the realization of sample sizes online. The conditional probability that the charting statistic exceeds the control limit at present given that there has not been a single alarm before can be guaranteed to meet a specified false alarm rate. Simulation studies show that our proposed control chart is able to deliver satisfactory run length performance for any time‐varying sample sizes. The idea presented in this article can be applied to any effective control charts such as the exponentially weighted moving average or cumulative sum chart. © 2013 Wiley Periodicals, Inc. Naval Research Logistics 60: 625–636, 2013     

Markov decision processes with restricted observations: Finite horizon case

In this article we consider a Markov decision process subject to the constraints that result from some observability restrictions. We assume that the state of the Markov process under consideration is unobservable. The states are grouped so that the group that a state belongs to is observable. So, we want to find an optimal decision rule depending on the observable groups instead of the states. This means that the same decision applies to all the states in the same group. We prove that a deterministic optimal policy exists for the finite horizon. An algorithm is developed to compute policies minimizing the total expected discounted cost over a finite horizon. © 1997 John Wiley & Sons, Inc. Naval Research Logistics 44 : 439–456, 1997     

Optimal maintenance policies for a safety‐critical system and its deteriorating sensor

We consider the integrated problem of optimally maintaining an imperfect, deteriorating sensor and the safety‐critical system it monitors. The sensor's costless observations of the binary state of the system become less informative over time. A costly full inspection may be conducted to perfectly discern the state of the system, after which the system is replaced if it is in the out‐of‐control state. In addition, a full inspection provides the opportunity to replace the sensor. We formulate the problem of adaptively scheduling full inspections and sensor replacements using a partially observable Markov decision process (POMDP) model. The objective is to minimize the total expected discounted costs associated with system operation, full inspection, system replacement, and sensor replacement. We show that the optimal policy has a threshold structure and demonstrate the value of coordinating system and sensor maintenance via numerical examples. © 2017 Wiley Periodicals, Inc. Naval Research Logistics 64: 399–417, 2017     

基于TSMD-SVM-HMM的机内测试智能降虚警方法

机内测试虚警问题是影响系统完好性和使用保障费用的重要因素。针对环境因素导致的虚警问题,设计了时间环境应力测量装置,应用支持向量机的小样本学习优点,建立虚警与环境因素的关联关系,应用隐马尔可夫模型的连续动态信号处理能力,描述系统长期工作历程中虚警的发生规律,提出了基于时间环境应力测量装置-支持向量机-隐马尔可夫模型的机内测试智能降虚警方法。最后,在某型直升机航向姿态系统上进行了应用与验证,试验结果表明:该方法有效识别出了机内测试的虚警。     

Dynamic pricing and replenishment: Optimality,bounds, and asymptotics

In many applications, managers face the problem of replenishing and selling products during a finite time horizon. We investigate the problem of making dynamic and joint decisions on product replenishment and selling in order to improve profit. We consider a backlog scenario in which penalty cost (resulting from fulfillment delay) and accommodation cost (resulting from shortage at the end of the selling horizon) are incurred. Based on continuous‐time and discrete‐state dynamic programming, we study the optimal joint decisions and characterize their structural properties. We establish an upper bound for the optimal expected profit and develop a fluid policy by resorting to the deterministic version of the problem (ie, the fluid problem). The fluid policy is shown to be asymptotically optimal for the original stochastic problem when the problem size is sufficiently large. The static nature of the fluid policy and its lack of flexibility in matching supply with demand motivate us to develop a “target‐inventory” heuristic, which is shown, numerically, to be a significant improvement over the fluid policy. Scenarios with discrete feasible sets and lost‐sales are also discussed in this article.     

累积和检验算法应用于主动声纳检测时的性能

研究了一种基于瞬态信号检测的累积和序贯检测器,其检验统计量由归一化匹配滤波器输出决定,受到声纳方程估算的信噪比影响.从理论上分析了该算法的虚警和检测性能,并和归一化匹配滤波器作了比较.海试数据的验证结果表明,该算法检测性能优于归一化匹配滤波器,并且对混响有较好的抑制作用.     

A partial characterization of the optimal ordering/rationing policy for a periodic review system with two demand classes and backordering

We consider a finite horizon periodic review, single product inventory system with a fixed setup cost and two stochastic demand classes that differ in their backordering costs. In each period, one must decide whether and how much to order, and how much demand of the lower class should be satisfied. We show that the optimal ordering policy can be characterized as a state dependent (s,S) policy, and the rationing structure is partially obtained based on the subconvexity of the cost function. We then propose a simple heuristic rationing policy, which is easy to implement and close to optimal for intensive numerical examples. We further study the case when the first demand class is deterministic and must be satisfied immediately. We show the optimality of the state dependent (s,S) ordering policy, and obtain additional rationing structural properties. Based on these properties, the optimal ordering and rationing policy for any state can be generated by finding the optimal policy of only a finite set of states, and for each state in this set, the optimal policy is obtained simply by choosing a policy from at most two alternatives. An efficient algorithm is then proposed. © 2010 Wiley Periodicals, Inc. Naval Research Logistics, 2010     

低空复杂场景红外弱小目标快速精准检测

针对低空复杂场景下红外弱小动目标检测难度大、虚警率高等问题,面向探测系统中高帧频图像实时处理应用需求,提出基于全卷积网络的弱小目标精准检测方法和基于现场可编程逻辑门阵列(field programmable gate array, FPGA)的低时延并行处理方法。采用轻量化全卷积网络对红外图像中弱小目标进行空域检测,对相邻图像帧疑似目标进行时域轨迹关联以进一步降低虚警率。实验结果表明:上述方法相比于五种传统方法在检测率和虚警率性能方面均有显著提升,并在单片FPGA上完成100 Hz图像实时处理,处理时延低于1.8 ms,实现低空复杂场景弱小目标高精度高鲁棒快速实时检测。     

Solving partially observable agent‐intruder games with an application to border security problems

In this paper, we introduce partially observable agent‐intruder games (POAIGs). These games model dynamic search games on graphs between security forces (an agent) and an intruder given possible (border) entry points and high value assets that require protection. The agent faces situations with dynamically changing, partially observable information about the state of the intruder and vice versa. The agent may place sensors at selected locations, while the intruder may recruit partners to observe the agent's movement. We formulate the problem as a two‐person zero‐sum game, and develop efficient algorithms to compute each player's optimal strategy. The solution to the game will help the agent choose sensor locations and design patrol routes that can handle imperfect information. First, we prove the existence of ?‐optimal strategies for POAIGs with an infinite time horizon. Second, we introduce a Bayesian approximation algorithm to identify these ?‐optimal strategies using belief functions that incorporate the imperfect information that becomes available during the game. For the solutions of large POAIGs with a finite time horizon, we use a solution method common to extensive form games, namely, the sequence form representation. To illustrate the POAIGs, we present several examples and numerical results.     

Model parameter estimation and residual life prediction for a partially observable failing system

We consider a partially observable degrading system subject to condition monitoring and random failure. The system's condition is categorized into one of three states: a healthy state, a warning state, and a failure state. Only the failure state is observable. While the system is operational, vector data that is stochastically related to the system state is obtained through condition monitoring at regular sampling epochs. The state process evolution follows a hidden semi‐Markov model (HSMM) and Erlang distribution is used for modeling the system's sojourn time in each of its operational states. The Expectation‐maximization (EM) algorithm is applied to estimate the state and observation parameters of the HSMM. Explicit formulas for several important quantities for the system residual life estimation such as the conditional reliability function and the mean residual life are derived in terms of the posterior probability that the system is in the warning state. Numerical examples are presented to demonstrate the applicability of the estimation procedure and failure prediction method. A comparison results with hidden Markov modeling are provided to illustrate the effectiveness of the proposed model. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 190–205, 2015     

基于灰色关联度和模糊贴近度的复杂设备瞬态虚警辨识

分析了电子设备和武器装备的ＢＩＴ系统虚警形式之一——瞬态虚警的产生原因和一般的预防措施；提出运用灰色关联度和模糊贴近度对瞬态虚警进行辨识并给出了这两种方法的基本原理；分别采用数值模拟和物理模拟的手段，获得了具有温度、湿度、振动（电源）三环境参数系统的状态变化样本群，以及因振动（电源）参数突变而产生的瞬态虚警样本；运用灰色关联度和模糊贴近度方法识别虚警及产生虚警的环境因素，结果表明两种方法可以胜任要求。     

Interdiction models for delaying adversarial attacks against critical information technology infrastructure

Information technology (IT) infrastructure relies on a globalized supply chain that is vulnerable to numerous risks from adversarial attacks. It is important to protect IT infrastructure from these dynamic, persistent risks by delaying adversarial exploits. In this paper, we propose max‐min interdiction models for critical infrastructure protection that prioritizes cost‐effective security mitigations to maximally delay adversarial attacks. We consider attacks originating from multiple adversaries, each of which aims to find a “critical path” through the attack surface to complete the corresponding attack as soon as possible. Decision‐makers can deploy mitigations to delay attack exploits, however, mitigation effectiveness is sometimes uncertain. We propose a stochastic model variant to address this uncertainty by incorporating random delay times. The proposed models can be reformulated as a nested max‐max problem using dualization. We propose a Lagrangian heuristic approach that decomposes the max‐max problem into a number of smaller subproblems, and updates upper and lower bounds to the original problem via subgradient optimization. We evaluate the perfect information solution value as an alternative method for updating the upper bound. Computational results demonstrate that the Lagrangian heuristic identifies near‐optimal solutions efficiently, which outperforms a general purpose mixed‐integer programming solver on medium and large instances.     

Optimal policies for stochastic clearing systems with time-dependent delay penalties

We study stochastic clearing systems with a discrete-time Markovian input process, and an output mechanism that intermittently and instantaneously clears the system partially or completely. The decision to clear the system depends on both quantities and delays of outstanding inputs. Clearing the system incurs a fixed cost, and outstanding inputs are charged a delay penalty, which is a general increasing function of the quantities and delays of individual inputs. By recording the quantities and delays of outstanding inputs in a sequence, we model the clearing system as a tree-structured Markov decision process over both a finite and infinite horizon. We show that the optimal clearing policies, under realistic conditions, are of the on-off type or the threshold type. Based on the characterization of the optimal policies, we develop efficient algorithms to compute parameters of the optimal policies for such complex clearing systems for the first time. We conduct a numerical analysis on the impact of the nonlinear delay penalty cost function, the comparison of the optimal policy and the classical hybrid policy (ie, quantity and age thresholds), and the impact of the state of the input process. Our experiments demonstrate that (a) the classical linear approximation of the cost function can lead to significant performance differences; (b) the classical hybrid policy may perform poorly (as compared to the optimal policies); and (c) the consideration of the state of the input process makes significant improvement in system performance.     

Production to order and off‐line inspection when the production process is partially observable

This study combines inspection and lot‐sizing decisions. The issue is whether to INSPECT another unit or PRODUCE a new lot. A unit produced is either conforming or defective. Demand need to be satisfied in full, by conforming units only. The production process may switch from a “good” state to a “bad” state, at constant rate. The proportion of conforming units in the good state is higher than in the bad state. The true state is unobservable and can only be inferred from the quality of units inspected. We thus update, after each inspection, the probability that the unit, next candidate for inspection, was produced while the production process was in the good state. That “good‐state‐probability” is the basis for our decision to INSPECT or PRODUCE. We prove that the optimal policy has a simple form: INSPECT only if the good‐state‐probability exceeds a control limit. We provide a methodology to calculate the optimal lot size and the expected costs associated with INSPECT and PRODUCE. Surprisingly, we find that the control limit, as a function of the demand (and other problem parameters) is not necessarily monotone. Also, counter to intuition, it is possible that the optimal action is PRODUCE, after revealing a conforming unit. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

Value management of diagnostic equipment with cancelation,no‐show,and emergency patients

Magnetic resonance imaging and other multifunctional diagnostic facilities, which are considered as scarce resources of hospitals, typically provide services to patients with different medical needs. This article examines the admission policies during the appointment management of such facilities. We consider two categories of patients: regular patients who are scheduled in advance through an appointment system and emergency patients with randomly generated demands during the workday that must be served as soon as possible. According to the actual medical needs of patients, regular patients are segmented into multiple classes with different cancelation rates, no‐show probabilities, unit value contributions, and average service times. Management makes admission decisions on whether or not to accept a service request from a regular patient during the booking horizon to improve the overall value that could be generated during the workday. The decisions should be made by considering the cancelation and no‐show behavior of booked patients as well as the emergency patients that would have to be served because any overtime service would lead to higher costs. We studied the optimal admission decision using a continuous‐time discrete‐state dynamic programming model. Identifying an optimal policy for this discrete model is analytically intractable and numerically inefficient because the state is multidimensional and infinite. We propose to study a deterministic counterpart of the problem (i.e., the fluid control problem) and to develop a time‐based fluid policy that is shown to be asymptotically optimal for large‐scale problems. Furthermore, we propose to adopt a mixed fluid policy that is developed based on the information obtained from the fluid control problem. Numerical experiments demonstrate that this improved policy works effectively for small‐scale problems. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 287–304, 2016     

Structured maintenance policies on interior sample paths

We examine the problem of adaptively scheduling perfect observations and preventive replacements for a multi‐state, Markovian deterioration system with silent failures such that total expected discounted cost is minimized. We model this problem as a partially observed Markov decision process and show that the structural properties of the optimal policy hold for certain non‐extreme sample paths. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

BIT系统的三态马尔可夫模型分析

BIT技术是改善系统测试性和诊断能力的重要途径,但是较高的虚警率一直是阻碍BIT广泛应用的一个重要原因。研究过程中将系统状态划分为正常、间歇、故障三种状态,建立了马尔可夫模型,分析、对比了三态模型与两态模型的关系。理论分析结果表明:基于三态模型的BIT,在提高故障检测率的同时,还可以较大程度地降低虚警率。     

一种基于帧间相关的红外弱小目标运动轨迹检测方法

针对传统帧间相关法对部分虚警轨迹难以排除的问题,研究了一种简单的帧间相关去噪算法,并提出在此基础上再次运用数学形态学方法来进一步排除虚假轨迹。算法采用形态学top-hat变换和二值化方法预处理原始图像,然后根据目标帧间相关而噪声点不相关的特点去噪,之后相加得到初步运动轨迹图像;最后采用形态学开运算,排除虚假轨迹,得到最终的检测结果。结果中检测出的目标轨迹清晰,虚警少。实验表明,该方法能有效地检测出弱小目标的运动轨迹。     

Stochastic regret minimization for revenue management problems with nonstationary demands

We study an admission control model in revenue management with nonstationary and correlated demands over a finite discrete time horizon. The arrival probabilities are updated by current available information, that is, past customer arrivals and some other exogenous information. We develop a regret‐based framework, which measures the difference in revenue between a clairvoyant optimal policy that has access to all realizations of randomness a priori and a given feasible policy which does not have access to this future information. This regret minimization framework better spells out the trade‐offs of each accept/reject decision. We proceed using the lens of approximation algorithms to devise a conceptually simple regret‐parity policy. We show the proposed policy achieves 2‐approximation of the optimal policy in terms of total regret for a two‐class problem, and then extend our results to a multiclass problem with a fairness constraint. Our goal in this article is to make progress toward understanding the marriage between stochastic regret minimization and approximation algorithms in the realm of revenue management and dynamic resource allocation. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 433–448, 2016