Markovian approximation for manufacturing systems of unreliable machines in tandem

This paper studies production planning of manufacturing systems of unreliable machines in tandem. The manufacturing system considered here produces one type of product. The demand is assumed to be a Poisson process and the processing time for one unit of product in each machine is exponentially distributed. A broken machine is subject to a sequence of repairing processes. The up time and the repairing time in each phase are assumed to be exponentially distributed. We study the manufacturing system by considering each machine as an individual system with stochastic supply and demand. The Markov Modulated Poisson Process (MMPP) is applied to model the process of supply. Numerical examples are given to demonstrate the accuracy of the proposed method. We employ (s, S) policy as production control. Fast algorithms are presented to solve the average running costs of the machine system for a given (s, S) policy and hence the approximated optimal (s, S) policy. © 2001 John Wiley & Sons, Inc. Naval Research Logistics 48: 65–78, 2001     

马尔可夫随机场中应用蚁群系统的红外图像分割

利用蚁群系统(ACS)的信息正反馈和启发式的搜索特点,针对马尔可夫随机场(MRF)图像模型的局部相关特性和最大后验概率(MAP)估计,可以在较短时间内得到图像分割目标函数的全局最优解,从而可以避免传统模拟退火算法的庞大时间复杂度.通过对军用红外图像的分割实验,可以看出这种算法能够在抗噪声、保留目标边缘和降低时间复杂度方面得到较满意的结果.     

Modeling and analysis of vessel casualties resulting from tanker traffic through narrow waterways

In this paper, we present a physics-based stochastic model to investigate vessel casualties resulting from tanker traffic through a narrow waterway. A state-space model is developed to represent the waterway and the location of vessels at a given time. We first determine the distribution of surface current at a given location of the waterway depending on channel geometry, bottom topography, boundary conditions, and the distribution of wind. Then we determine the distribution of the angular drift for a given vessel travelling at a given location of a waterway. Finally, we incorporate the drift probabilities and random arrival of vessels into a Markov chain model. By analyzing the time-dependent and the steady-state probabilities of the Markov chain, we obtain risk measures such as the probability of casualty at a given location and also the expected number of casualties for a given number of vessels arriving per unit time. Analysis of the Markovian model also yields an analytical result that shows that the expected number of casualties is proportional to square of the tanker arrival rate. We present our methodology on an experimental model of a hypothetical narrow waterway. © 1999 John Wiley & Sons, Inc. Naval Reseach Logistics 46: 871–892, 1999     

The LP/POMDP marriage: Optimization with imperfect information

A new technique for solving large‐scale allocation problems with partially observable states and constrained action and observation resources is introduced. The technique uses a master linear program (LP) to determine allocations among a set of control policies, and uses partially observable Markov decision processes (POMDPs) to determine improving policies using dual prices from the master LP. An application is made to a military problem where aircraft attack targets in a sequence of stages, with information acquired in one stage being used to plan attacks in the next. © 2000 John Wiley & Sons, Inc., Naval Research Logistics 47: 607–619, 2000     

对多次命中时舰艇主动力装置生命力问题的研究

针对舰艇主动力装置生命力 ,以及多次命中的生命力难以进行定量分析 ,定义了主动力装置多次命中后所处的各种可能状态 ,并运用马尔科夫过程的有关理论 ,推导出了多次命中条件下舰艇处于各种状态的概率 .     

Reliability of a two-dimensional demand-based networked system with multistate components

Existing research on multistate system reliability has mainly focused on one-dimensional systems such as parallel systems, linear sliding window systems, and linearly consecutively connected systems. However, two-dimensional networked systems widely exist in real-world applications such as lighting systems, monitoring systems, and computer network systems. This research considers a two-dimensional networked system consisting of multistate components. The system fails if the cumulative performance of any row or any column cannot meet a predetermined demand. A novel reliability evaluation algorithm is proposed for the considered two-dimensional networked system by extending the universal generating function technique. Furthermore, the proposed model and reliability evaluation algorithm are extended to a two-dimensional networked system with phased missions. The proposed models and algorithms are illustrated by a matrix heating system in a thermoforming machine.     

Optimal production ramp-up in the smartphone manufacturing industry

Motivated by challenges in the smartphone manufacturing industry, we develop a dynamic production ramp-up model that can be applied to economically satisfy nonstationary demand for short-life-cycle products by high-tech companies. Due to shorter life cycles and more rapid evolution of smartphones, production ramp-up has been increasingly critical to the success of a new smartphone. In the production ramp-up, the key challenge is to match the increasing capacity to nonstationary demand. The high-tech smartphone manufacturers are urged to jointly consider the effect of increasing capacity and decreasing demand. We study the production planning problem using a high-dimensional Markov decision process (MDP) model to characterize the production ramp-up. To address the curse of dimensionality, we refine Monte Carlo tree search (MCTS) algorithm and theoretically analyze its convergence and computational complexity. In a real case study, we find that the MDP model achieves revenue improvement by stopping producing the existing product earlier than the benchmark policy. In synthetic instances, we validate that the proposed MCTS algorithm saves computation time without loss of solution quality compared with traditional value iteration algorithm. As part of the Lenovo production solution, our MDP model enables high-tech smartphone manufacturers to better plan the production ramp-up.     

Revenue maximization in two-station tandem queueing systems

We study optimal pricing for tandem queueing systems with finite buffers. The service provider dynamically quotes prices to incoming price sensitive customers to maximize the long-run average revenue. We present a Markov decision process model for the optimization problem. For systems with two stations, general-sized buffers, and two or more prices, we describe the structure of the optimal dynamic pricing policy and develop tailored policy iteration algorithms to find an optimal pricing policy. For systems with two stations but no intermediate buffer, we characterize conditions under which quoting either a high or a low price to all customers is optimal and provide an easy-to-implement algorithm to solve the problem. Numerical experiments are conducted to compare the developed algorithms with the regular policy iteration algorithm. The work also discusses possible extensions of the obtained results to both three-station systems and two-station systems with price and congestion sensitive customers using numerical analysis.     

On the reduction of total‐cost and average‐cost MDPs to discounted MDPs

This article provides conditions under which total‐cost and average‐cost Markov decision processes (MDPs) can be reduced to discounted ones. Results are given for transient total‐cost MDPs with transition rates whose values may be greater than one, as well as for average‐cost MDPs with transition probabilities satisfying the condition that there is a state such that the expected time to reach it is uniformly bounded for all initial states and stationary policies. In particular, these reductions imply sufficient conditions for the validity of optimality equations and the existence of stationary optimal policies for MDPs with undiscounted total cost and average‐cost criteria. When the state and action sets are finite, these reductions lead to linear programming formulations and complexity estimates for MDPs under the aforementioned criteria.© 2017 Wiley Periodicals, Inc. Naval Research Logistics 66:38–56, 2019     

Exact analysis of (R,s, S) inventory control systems with lost sales and zero lead time

We study an (R, s, S) inventory control policy with stochastic demand, lost sales, zero lead‐time and a target service level to be satisfied. The system is modeled as a discrete time Markov chain for which we present a novel approach to derive exact closed‐form solutions for the limiting distribution of the on‐hand inventory level at the end of a review period, given the reorder level (s) and order‐up‐to level (S). We then establish a relationship between the limiting distributions for adjacent values of the reorder point that is used in an efficient recursive algorithm to determine the optimal parameter values of the (R, s, S) replenishment policy. The algorithm is easy to implement and entails less effort than solving the steady‐state equations for the corresponding Markov model. Point‐of‐use hospital inventory systems share the essential characteristics of the inventory system we model, and a case study using real data from such a system shows that with our approach, optimal policies with significant savings in inventory management effort are easily obtained for a large family of items.