Application of Markov renewal theory and semi-Markov decision processes in maintenance modeling and optimization of multi-unit systems

In this paper, a condition-based maintenance model for a multi-unit production system is proposed and analyzed using Markov renewal theory. The units of the system are subject to gradual deterioration, and the gradual deterioration process of each unit is described by a three-state continuous time homogeneous Markov chain with two working states and a failure state. The production rate of the system is influenced by the deterioration process and the demand is constant. The states of the units are observable through regular inspections and the decision to perform maintenance depends on the number of units in each state. The objective is to obtain the steady-state characteristics and the formula for the long-run average cost for the controlled system. The optimal policy is obtained using a dynamic programming algorithm. The result is validated using a semi-Markov decision process formulation and the policy iteration algorithm. Moreover, an analytical expression is obtained for the calculation of the mean time to initiate maintenance using the first passage time theory.     

运动目标轨迹分类与识别

运动目标轨迹识别是运动分析中的基本问题,其目的是解释所监视场景中发生的事件,对所监视场景中运动目标轨迹的行为模式进行分析与识别,智能地做出自动分类.对轨迹有效性判断后采用K均值聚类,引入改进的隐马尔可夫模型算法,针对轨迹的复杂程度对各个轨迹模式类建立相应的隐马尔可夫模型,利用训练样本训练模型得到可靠的模型参数,计算测试样本对于各个模型的最大似然概率,选取最大概率值对应的轨迹模式类作为轨迹识别的结果,对两种场景中聚类后的轨迹进行训练与识别,平均识别率较高,实验结果表明该方法是有效的.     

Dynamic server assignment policies for assembly‐type queues with flexible servers

We seek dynamic server assignment policies in finite‐capacity queueing systems with flexible and collaborative servers, which involve an assembly and/or a disassembly operation. The objective is to maximize the steady‐state throughput. We completely characterize the optimal policy for a Markovian system with two servers, two feeder stations, and instantaneous assembly and disassembly operations. This optimal policy allocates one server per station unless one of the stations is blocked, in which case both servers work at the unblocked station. For Markovian systems with three stations and instantaneous assembly and/or disassembly operations, we consider similar policies that move a server away from his/her “primary” station only when that station is blocked or starving. We determine the optimal assignment of each server whose primary station is blocked or starving in systems with three stations and zero buffers, by formulating the problem as a Markov decision process. Using this optimal assignment, we develop heuristic policies for systems with three or more stations and positive buffers, and show by means of a numerical study that these policies provide near‐optimal throughput. Furthermore, our numerical study shows that these policies developed for assembly‐type systems also work well in tandem systems. © 2008 Wiley Periodicals, Inc. Naval Research Logistics, 2008     

Optimal maintenance policies for a safety‐critical system and its deteriorating sensor

We consider the integrated problem of optimally maintaining an imperfect, deteriorating sensor and the safety‐critical system it monitors. The sensor's costless observations of the binary state of the system become less informative over time. A costly full inspection may be conducted to perfectly discern the state of the system, after which the system is replaced if it is in the out‐of‐control state. In addition, a full inspection provides the opportunity to replace the sensor. We formulate the problem of adaptively scheduling full inspections and sensor replacements using a partially observable Markov decision process (POMDP) model. The objective is to minimize the total expected discounted costs associated with system operation, full inspection, system replacement, and sensor replacement. We show that the optimal policy has a threshold structure and demonstrate the value of coordinating system and sensor maintenance via numerical examples. © 2017 Wiley Periodicals, Inc. Naval Research Logistics 64: 399–417, 2017     

Transient analysis of Markov models of fault‐tolerant systems with deferred repair using split regenerative randomization

The (standard) randomization method is an attractive alternative for the transient analysis of continuous time Markov models. The main advantages of the method are numerical stability, well‐controlled computation error, and ability to specify the computation error in advance. However, the fact that the method can be computationally very expensive limits its applicability. In this paper, we develop a new method called split regenerative randomization, which, having the same good properties as standard randomization, can be significantly more efficient. The method covers reliability‐like models with a particular but quite general structure and requires the selection of a subset of states and a regenerative state satisfying some conditions. For a class of continuous time Markov models, model class C₂, including typical failure/repair reliability‐like models with exponential failure and repair time distributions and deferred repair, natural selections are available for both the subset of states and the regenerative state and, for those natural selections, theoretical results are available assessing the efficiency of the method in terms of “visible” model characteristics. Those results can be used to anticipate when the method can be expected to be competitive. We illustrate the application of the method using a large class C₂ model and show that for models in that class the method can indeed be significantly more efficient than previously available randomization‐based methods. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006     

差分跳频码性能检验研究

差分跳频是一种新的扩展频谱通信技术.在介绍差分跳频基本原理的基础上,将差分跳频的频率跳变过程建模成齐次马尔可夫链.分析了G函数的功能,重点讨论了差分跳频码性能的检验方法,包括不可约性、频隙滞留、均匀性和随机性检验,其中频隙滞留是首次提出应用于差分跳频码性能的检验.这些检验方法对于差分跳频G函数的设计具有一定的指导意义.     

Model parameter estimation and residual life prediction for a partially observable failing system

We consider a partially observable degrading system subject to condition monitoring and random failure. The system's condition is categorized into one of three states: a healthy state, a warning state, and a failure state. Only the failure state is observable. While the system is operational, vector data that is stochastically related to the system state is obtained through condition monitoring at regular sampling epochs. The state process evolution follows a hidden semi‐Markov model (HSMM) and Erlang distribution is used for modeling the system's sojourn time in each of its operational states. The Expectation‐maximization (EM) algorithm is applied to estimate the state and observation parameters of the HSMM. Explicit formulas for several important quantities for the system residual life estimation such as the conditional reliability function and the mean residual life are derived in terms of the posterior probability that the system is in the warning state. Numerical examples are presented to demonstrate the applicability of the estimation procedure and failure prediction method. A comparison results with hidden Markov modeling are provided to illustrate the effectiveness of the proposed model. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 190–205, 2015     

航天测控通信系统可靠性分析的Krylov子空间投影算法

基于Markov模型对航天测控通信系统进行可靠性分析的过程中,若系统中测控通信设备数量较多,模型中的状态空间随设备数量呈指数增长,将会导致数值计算困难.提出了一种基于Krylov子空间技术的可靠性分析方法,将大规模问题投影至小规模子空间中,求得问题的近似解.实验结果证明,Krylov子空间方法的计算速度及精度优于Ross方法和前向Euler法(forward Euler method,FEM).     

Higher‐order Markov chain models for categorical data sequences*

In this paper we study higher‐order Markov chain models for analyzing categorical data sequences. We propose an efficient estimation method for the model parameters. Data sequences such as DNA and sales demand are used to illustrate the predicting power of our proposed models. In particular, we apply the developed higher‐order Markov chain model to the server logs data. The objective here is to model the users' behavior in accessing information and to predict their behavior in the future. Our tests are based on a realistic web log and our model shows an improvement in prediction. © 2004 Wiley Periodicals, Inc. Naval Research Logistics, 2004     

基于马尔可夫过程的活动目标侦察雷达搜索效率模型研究

根据活动目标侦察雷达的特点和发现目标的环节,用马尔可夫过程对侦察系统的雷达搜索效率进行分析,得出计算模型,给出应用实例.对提高雷达的作战效能有一定意义.