Application of Markov renewal theory and semi-Markov decision processes in maintenance modeling and optimization of multi-unit systems

In this paper, a condition-based maintenance model for a multi-unit production system is proposed and analyzed using Markov renewal theory. The units of the system are subject to gradual deterioration, and the gradual deterioration process of each unit is described by a three-state continuous time homogeneous Markov chain with two working states and a failure state. The production rate of the system is influenced by the deterioration process and the demand is constant. The states of the units are observable through regular inspections and the decision to perform maintenance depends on the number of units in each state. The objective is to obtain the steady-state characteristics and the formula for the long-run average cost for the controlled system. The optimal policy is obtained using a dynamic programming algorithm. The result is validated using a semi-Markov decision process formulation and the policy iteration algorithm. Moreover, an analytical expression is obtained for the calculation of the mean time to initiate maintenance using the first passage time theory.     

Model parameter estimation and residual life prediction for a partially observable failing system

We consider a partially observable degrading system subject to condition monitoring and random failure. The system's condition is categorized into one of three states: a healthy state, a warning state, and a failure state. Only the failure state is observable. While the system is operational, vector data that is stochastically related to the system state is obtained through condition monitoring at regular sampling epochs. The state process evolution follows a hidden semi‐Markov model (HSMM) and Erlang distribution is used for modeling the system's sojourn time in each of its operational states. The Expectation‐maximization (EM) algorithm is applied to estimate the state and observation parameters of the HSMM. Explicit formulas for several important quantities for the system residual life estimation such as the conditional reliability function and the mean residual life are derived in terms of the posterior probability that the system is in the warning state. Numerical examples are presented to demonstrate the applicability of the estimation procedure and failure prediction method. A comparison results with hidden Markov modeling are provided to illustrate the effectiveness of the proposed model. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 190–205, 2015     

On‐line parameter estimation for a partially observable system subject to random failure

In this paper, we study the on‐line parameter estimation problem for a partially observable system subject to deterioration and random failure. The state of the system evolves according to a continuous time homogeneous Markov process with a finite state space. The system state is not observable, except for the failure state. The information related to the system state is available at discrete times through inspections. A recursive maximum likelihood (RML) algorithm is proposed for the on‐line parameter estimation of the model. The RML algorithm proposed in the paper is considerably faster and easier to apply than other RML algorithms in the literature, because it does not require projection into the constraint domain and calculation of the gradient on the surface of the constraint manifolds. The algorithm is illustrated by an example using real vibration data. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006