On‐line parameter estimation for a partially observable system subject to random failure

In this paper, we study the on‐line parameter estimation problem for a partially observable system subject to deterioration and random failure. The state of the system evolves according to a continuous time homogeneous Markov process with a finite state space. The system state is not observable, except for the failure state. The information related to the system state is available at discrete times through inspections. A recursive maximum likelihood (RML) algorithm is proposed for the on‐line parameter estimation of the model. The RML algorithm proposed in the paper is considerably faster and easier to apply than other RML algorithms in the literature, because it does not require projection into the constraint domain and calculation of the gradient on the surface of the constraint manifolds. The algorithm is illustrated by an example using real vibration data. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006     

Application of Markov renewal theory and semi-Markov decision processes in maintenance modeling and optimization of multi-unit systems

In this paper, a condition-based maintenance model for a multi-unit production system is proposed and analyzed using Markov renewal theory. The units of the system are subject to gradual deterioration, and the gradual deterioration process of each unit is described by a three-state continuous time homogeneous Markov chain with two working states and a failure state. The production rate of the system is influenced by the deterioration process and the demand is constant. The states of the units are observable through regular inspections and the decision to perform maintenance depends on the number of units in each state. The objective is to obtain the steady-state characteristics and the formula for the long-run average cost for the controlled system. The optimal policy is obtained using a dynamic programming algorithm. The result is validated using a semi-Markov decision process formulation and the policy iteration algorithm. Moreover, an analytical expression is obtained for the calculation of the mean time to initiate maintenance using the first passage time theory.     

基于马尔科夫决策的目标选择策略

目标选择是军事计划的关键要素之一。基于马尔科夫决策方法,解决具有复杂目标间关联的多阶段目标选择问题。使用与或树描述目标体系各层状态间的影响关联,并以目标体系整体失效为求解目的,建立了基于离散时间MDP的多阶段打击目标选择模型。在LRTDP算法基础上提出一种启发式方法,通过判断从当前目标体系状态到达体系失效状态的演化过程中的可能资源消耗和失败概率,来提供对当前状态的评估值,该方法能有效排除问题搜索空间中不能到达体系失效目的的中间状态,压缩了由于目标间复杂关联而增长的巨大状态空间。用实验验证了该方法有效性,实验结果表明,该方法直观实用,对目标间具有复杂关联关系的目标打击决策有一定参考价值。     

基于隐马尔可夫模型的IDS程序行为异常检测

提出一种新的基于隐马尔可夫模型的程序行为异常检测方法,此方法利用系统调用序列,并基于隐马尔可夫模型来描述程序行为,根据程序行为模式的出现频率对其进行分类,并将行为模式类型同隐马尔可夫模型的状态联系在一起。由于各状态对应的观测值集合互不相交,模型训练中采用了运算量较小的序列匹配方法,与传统的Baum Welch算法相比,训练时间有较大幅度的降低。考虑到模型中状态的特殊含义以及程序行为的特点,将加窗平滑后的状态序列出现概率作为判决依据。实验表明,此方法具有很高的检测准确性,其检测效率也优于同类方法。     

Maintaining systems with heterogeneous spare parts

We consider the problem of optimally maintaining a stochastically degrading, single‐unit system using heterogeneous spares of varying quality. The system's failures are unannounced; therefore, it is inspected periodically to determine its status (functioning or failed). The system continues in operation until it is either preventively or correctively maintained. The available maintenance options include perfect repair, which restores the system to an as‐good‐as‐new condition, and replacement with a randomly selected unit from the supply of heterogeneous spares. The objective is to minimize the total expected discounted maintenance costs over an infinite time horizon. We formulate the problem using a mixed observability Markov decision process (MOMDP) model in which the system's age is observable but its quality must be inferred. We show, under suitable conditions, the monotonicity of the optimal value function in the belief about the system quality and establish conditions under which finite preventive maintenance thresholds exist. A detailed computational study reveals that the optimal policy encourages exploration when the system's quality is uncertain; the policy is more exploitive when the quality is highly certain. The study also demonstrates that substantial cost savings are achieved by utilizing our MOMDP‐based method as compared to more naïve methods of accounting for heterogeneous spares.     

Markov decision processes with restricted observations: Finite horizon case

In this article we consider a Markov decision process subject to the constraints that result from some observability restrictions. We assume that the state of the Markov process under consideration is unobservable. The states are grouped so that the group that a state belongs to is observable. So, we want to find an optimal decision rule depending on the observable groups instead of the states. This means that the same decision applies to all the states in the same group. We prove that a deterministic optimal policy exists for the finite horizon. An algorithm is developed to compute policies minimizing the total expected discounted cost over a finite horizon. © 1997 John Wiley & Sons, Inc. Naval Research Logistics 44 : 439–456, 1997     

Optimal replacement of continuously degrading systems in partially observed environments

Motivated by wind energy applications, we consider the problem of optimally replacing a stochastically degrading component that resides and operates in a partially observable environment. The component's rate of degradation is modulated by the stochastic environment process, and the component fails when it is accumulated degradation first reaches a fixed threshold. Assuming periodic inspection of the component, the objective is to minimize the long‐run average cost per unit time of performing preventive and reactive replacements for two distinct cases. The first case examines instantaneous replacements and fixed costs, while the second considers time‐consuming replacements and revenue losses accrued during periods of unavailability. Formulated and solved are mixed state space, partially observable Markov decision process models, both of which reveal the optimality of environment‐dependent threshold policies with respect to the component's cumulative degradation level. Additionally, it is shown that for each degradation value, a threshold policy with respect to the environment belief state is optimal if the environment alternates between two states. The threshold policies are illustrated by way of numerical examples using both synthetic and real wind turbine data. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 395–415, 2015     

Minimizing the false alarm rate in systems with transient abnormality

We consider a stochastic partially observable system that can switch between a normal state and a transient abnormal state before entering a persistent abnormal state. Only the persistent abnormal state requires alarms. The transient and persistent abnormal states may be similar in appearance, which can result in excess false alarms. We propose a partially observable Markov decision process model to minimize the false alarm rate, subject to a given upper bound on the expected alarm delay time. The cost parameter is treated as the Lagrange multiplier, which can be estimated from the bound of the alarm delay. We show that the optimal policy has a control‐limit structure on the probability of persistent abnormality, and derive closed‐form bounds for the control limit and present an algorithm to specify the Lagrange multiplier. We also study a specialized model where the transient and persistent abnormal states have the same observation distribution, in which case an intuitive “watchful‐waiting” policy is optimal. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 320–334, 2016     

A model to predict the residual life of aircraft engines based upon oil analysis data

This paper reports on a study using the available oil monitoring information, such as the data obtained using the Spectrometric Oil Analysis Programme (SOAP), to predict the residual life of a set of aircraft engines. The relationship between oil monitoring information and the residual life is established using the concept of the proportional residual, which states that the predicted residual life may be proportional to the wear increment measured by the oil analysis programmes. Assuming such a relationship between wear and the residual life exists, we formulated a recursive prediction model for the item's residual life given measured oil monitoring information to date. A set of censored life data of 30 aircraft engines (right censored due to preventive overhaul) along with the history of their monitored metal concentration information are available to us. The metal concentration information includes many variables, such as Fe, Cu, Al, etc.; not all of them are useful, and some of them may be correlated. The principal component analysis (PCA) has been adopted to reduce the dimension of the original data set and to produce a new set of uncorrelated variables, which we shall use in the prediction model. The procedure associated with estimating model parameters is discussed. The model is fitted to the actual SOAP data from the aircraft engines, and the goodness‐of‐fit test has been carried out. © 2005 Wiley Periodicals, Inc. Naval Research Logistics, 2005.     

Repairable consecutive‐k‐out‐of‐n: F system with Markov dependence

In this article, a model for a repairable consecutive‐k‐out‐of‐n: F system with Markov dependence is studied. A binary vector is used to represent the system state. The failure rate of a component in the system depends on the state of the preceding component. The failure risk of a system state is then introduced. On the basis of the failure risk, a priority repair rule is adopted. Then the transition density matrix can be determined, and the analysis of the system reliability can be conducted accordingly. One example each of a linear and a circular system is then studied in detail to explain the model and methodology developed in this paper. © 2000 John Wiley & Sons, Inc. Naval Research Logistics 47: 18–39, 2000     

Uniformly minimum variance unbiased estimates of operational readiness and reliability in a two-state system

This paper obtains the uniformly minimum variance unbiased estimates of two indices of performance of a system which alternates between two states “up” or “down” in accordance with a Markov process. The two indices are (1) operational readiness, which measures the probability that the system will be up when needed; and (2) operational reliability, which measures the probability that the system will be up during the entire time of need. For the purpose of obtaining these estimates, two types of observations are considered: (a) those which reveal only the state of system at isolated time-points, and (b) those which continuously record the duration of the “up” and “down” times of the system.     

基于复合失效模式的寿命分布检验方法

产品往往受随机失效和耗损失效两种模式的双重影响,因此,构造指数威布尔分布表征该类产品的失效规律。首先,用图检验法对该类产品的寿命数据进行初步检验;其次,研究了该类产品寿命数据的极大似然估计的求解方法,进一步用伽方检验方法对寿命分布类进行了检验;最后,通过工程仿真例子说明了该方法的具体操作流程,最终验证了方法的正确性。     

惯性平台自标定陀螺仪误差系数可观测度分析

针对惯性平台自标定陀螺仪误差系数的可观测度问题,从可观性定义角度出发,提出一种可观测度分析方法。利用状态量解析解表达式中观测量导数的最高阶数定义该状态量的可观测度。在此基础上,研究惯性平台自标定系统可观性与陀螺仪误差系数可观测度,分析系统可观测的状态量及其可观测度,得出陀螺仪本轴一次项误差系数可观测度最差的结论。仿真结果验证了该方法的正确性和有效性,为惯性平台自标定中施矩方案的设计提供了理论依据。     

Complexity and workload considerations in product mix decisions under the theory of constraints

The literature on the product mix decision (or master production scheduling) under the Theory of Constraints (TOC), which was developed in the past two decades, has addressed this problem as a static operational decision. Consequently, the developed solution techniques do not consider the system's dynamism and the associated challenges arising from the complexity of operations during the implementation of master production schedules. This paper aims to address this gap by developing a new heuristic approach for master production scheduling under the TOC philosophy that considers the main operational factors that influence actual throughput after implementation of the detailed schedule. We examine the validity of the proposed heuristic by comparison to Integer Linear Programming and two heuristics in a wide range of scenarios using simulation modelling. Statistical analyses indicate that the new algorithm leads to significantly enhanced performance during implementation for problems with setup times. The findings show that the bottleneck identification approach in current methods in the TOC literature is not effective and accurate for complex operations in real‐world job shop systems. This study contributes to the literature on master production scheduling and product mix decisions by enhancing the likelihood of achieving anticipated throughput during the implementation of the detailed schedule. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 357–369, 2015     

经马氏修正的Poisson过程的极大似然估计

近年来 ,隐马氏模型成为研究相依随机变量的一个十分有用的工具。实际应用过程中的一个很重要的问题是如何对隐马氏模型的参数进行估计。将一类连续时间隐马氏模型的问题转化为离散时间隐马氏模型的问题 ,给出了具体的隐马氏模型———经马氏修正的Poisson过程的极大似然估计及其算法。此类过程被广泛用来对复杂电信网络的交通流进行建模     

指标融合和隐马尔可夫模型的舰船装备技术状态评估

科学合理的技术状态评估技术是保障舰船安全性和任务完好性的有效途径之一,针对目前舰船装备技术状态评估缺乏动态性以及评估指标过多的问题,构建舰船装备技术状态多指标融合模型,结合技术状态评估指标融合模型和隐马尔可夫模型所具有双随机性和严谨数学推理能力的特点,建立基于指标融合模型和隐马尔可夫模型的舰船装备技术状态动态评估模型。最后通过案例分析验证了所提出的方法能够有效体现技术状态评估中的动态性,为舰船装备技术状态评估提供新的途径。     

A stochastic air combat logistics decision model for Blue versus Red opposition

Technologically advanced aircraft rely on robust and responsive logistics systems to ensure a high state of operational readiness. This paper fills a critical gap in the literature for combat models by closely relating effectiveness of the logistics system to determinants of success in combat. We present a stochastic diffusion model of an aerial battle between Blue and Red forces. The number of aircraft of Blue forces aloft and ready to be aloft on combat missions is limited by the maximum number of assigned aircraft, the reliability of aircraft subsystems, and the logistic system's ability to repair and replenish those subsystems. Our parsimonious model can illustrate important trade‐offs between logistics decision variables and operational success.     

二维隐马氏模型的一种简化算法及其参数估计

针对现有的二维隐马氏模型算法给出了一种简化算法及参数估计方法。该算法与现有的算法相比非常简单。基于此方法给出了相应的识别方法和参数估计,并且该估计与极大似然估计是等同的。     

Transient analysis of Markov models of fault‐tolerant systems with deferred repair using split regenerative randomization

The (standard) randomization method is an attractive alternative for the transient analysis of continuous time Markov models. The main advantages of the method are numerical stability, well‐controlled computation error, and ability to specify the computation error in advance. However, the fact that the method can be computationally very expensive limits its applicability. In this paper, we develop a new method called split regenerative randomization, which, having the same good properties as standard randomization, can be significantly more efficient. The method covers reliability‐like models with a particular but quite general structure and requires the selection of a subset of states and a regenerative state satisfying some conditions. For a class of continuous time Markov models, model class C₂, including typical failure/repair reliability‐like models with exponential failure and repair time distributions and deferred repair, natural selections are available for both the subset of states and the regenerative state and, for those natural selections, theoretical results are available assessing the efficiency of the method in terms of “visible” model characteristics. Those results can be used to anticipate when the method can be expected to be competitive. We illustrate the application of the method using a large class C₂ model and show that for models in that class the method can indeed be significantly more efficient than previously available randomization‐based methods. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006     

静止目标纯距离测量下的定位原理与方法研究

针对水下网络战条件下目标定位的特点,分析了纯距离系统可观测的条件,即对静止目标、观测站作直线运动,系统是不可观测的;并通过与同条件下的纯方位系统的对比,得出了在一般情况下,纯距离系统比纯方位系统具有更弱的观测性的结论;给出了估计目标位置参数的一种递推算法,该算法具有形式简单,不需要初值的特点;仿真试验表明,算法具有收敛速度快,精度好,稳定的优点.应用前景良好.