Stochastic control of queueing systems

Suppose that the state of a queueing system is described by a Markov process { Y_t, t ≥ 0}, and the profit from operating it up to a time t is given by the function f(Y_t). We operate the system up to a time T, where the random variable T is a stopping time for the process Yt. Optimal stochastic control is achieved by choosing the stopping time T that maximizes Ef(Y_T) over a given class of stopping times. In this paper a theory of stochastic control is developed for a single server queue with Poisson arrivals and general service times.     

On the first time a separately maintained parallel system has been down for a fixed time

Consider a system consisting of n separately maintained independent components where the components alternate between intervals in which they are “up” and in which they are “down”. When the i^th component goes up [down] then, independent of the past, it remains up [down] for a random length of time, having distribution F_i[G_i], and then goes down [up]. We say that component i is failed at time t if it has been “down” at all time points s ?[t-A.t]: otherwise it is said to be working. Thus, a component is failed if it is down and has been down for the previous A time units. Assuming that all components initially start “up,” let T denote the first time they are all failed, at which point we say the system is failed. We obtain the moment-generating function of T when n = l, for general F and G, thus generalizing previous results which assumed that at least one of these distributions be exponential. In addition, we present a condition under which T is an NBU (new better than used) random variable. Finally we assume that all the up and down distributions F_i and G_i i = l,….n, are exponential, and we obtain an exact expression for E(T) for general n; in addition we obtain bounds for all higher moments of T by showing that T is NBU.     

Using a birth‐and‐death process to estimate the steady‐state distribution of a periodic queue

If the number of customers in a queueing system as a function of time has a proper limiting steady‐state distribution, then that steady‐state distribution can be estimated from system data by fitting a general stationary birth‐and‐death (BD) process model to the data and solving for its steady‐state distribution using the familiar local‐balance steady‐state equation for BD processes, even if the actual process is not a BD process. We show that this indirect way to estimate the steady‐state distribution can be effective for periodic queues, because the fitted birth and death rates often have special structure allowing them to be estimated efficiently by fitting parametric functions with only a few parameters, for example, 2. We focus on the multiserver M_t/GI/s queue with a nonhomogeneous Poisson arrival process having a periodic time‐varying rate function. We establish properties of its steady‐state distribution and fitted BD rates. We also show that the fitted BD rates can be a useful diagnostic tool to see if an M_t/GI/s model is appropriate for a complex queueing system. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 664–685, 2015     

A duel complicated by false targets and uncertainty as to opponent type

An attacker, being one of two types, initiates an attack at some time in the interval [-T, 0]. The a priori probabilities of each type are known. As time elapses the defender encounters false targets which occur according to a known Poisson process and which can be properly classified with known probability. The detection and classification probabilities for each type attacker are given. If the defender responds with a weapon at the time of attack, he survives with a probability which depends on the number of weapons in his possession and on attacker type. If he does not respond, his survival probability is smaller. These probabilities are known, as well as the current number of weapons in the defender's possession. They decrease as the number of weapons decreases. The payoff is the defender's survival probability. An iterative system of first-order differential equations is derived whose unique solution V₁(t),V₂(t),…,V_k(t) is shown to be the value of the game at time t, when the defender has 1, 2,…, k,… weapons, respectively. The optimal strategies are determined. Limiting results are obtained as t→-∞, while the ratio of the number of weapons to the expected number of false targets remaining is held constant.     

Preservation of unimodality under shock models

A number of results pertaining to preservation of aging properties (IFR, IFRA etc.) under various shock models are available in the literature. Our aim in this paper is to examine in the same spirit, the preservation of unimodality under various shock models. For example, it is proved that in a non-homogeneous Poisson shock model if {p_k}_K≥0, the sequence of probabilities with which the device fails on the kth shock, is unimodal then under some suitable conditions on the mean value function Λ (t), the corresponding survival function is also unimodal. The other shock models under which the preservation of unimodality is considered in this paper are pure birth shock model and a more general shock model in which shocks occur according to a general counting process. © 1999 John Wiley & Sons, Inc. Naval Research Logistics 46: 952–957, 1999     

A note on a prediction interval for a first-order Gauss Markov process

Let X_t, t = 1,2, ?, be a stationary Gaussian Markov process with E(X_t) = μ and Cov(X_t, X_t+k) = σ²ρ^k. We derive a prediction interval for X_2n+1 based on the preceding 2n observations X₁,X₂, ?,X_2n.     

Estimation of ordered parameters from k stochastically increasing distributions

There are given k (? 2) univariate cumulative distribution functions (c.d.f.'s) G(x; θ_i) indexed by a real-valued parameter θ_i, i=1,…, k. Assume that G(x; θ_i) is stochastically increasing in θ_i. In this paper interval estimation on the i^th smallest of the θ's and related topics are studied. Applications are considered for location parameter, normal variance, binomial parameter, and Poisson parameter.     

An alternative proof of the IFRA property of some shock models

Let , where A (t)/t is nondecreasing in t, {P(k)^1/k} is nonincreasing. It is known that H(t) = 1 — H (t) is an increasing failure rate on the average (IFRA) distribution. A proof based on the IFRA closure theorem is given. H(t) is the distribution of life for systems undergoing shocks occurring according to a Poisson process where P (k) is the probability that the system survives k shocks. The proof given herein shows there is an underlying connection between such models and monotone systems of independent components that explains the IFRA life distribution occurring in both models.     

A decomposable transshipment algorithm for a multiperiod transportation problem

A dynamic version of the transportation (Hitchcock) problem occurs when there are demands at each of n sinks for T periods which can be fulfilled by shipments from m sources. A requirement in period t₂ can be satisfied by a shipment in the same period (a linear shipping cost is incurred) or by a shipment in period t₁ < t₂ (in addition to the linear shipping cost a linear inventory cost is incurred for every period in which the commodity is stored). A well known method for solving this problem is to transform it into an equivalent single period transportation problem with mT sources and nT sinks. Our approach treats the model as a transshipment problem consisting of T, m source — n sink transportation problems linked together by inventory variables. Storage requirements are proportional to T² for the single period equivalent transportation algorithm, proportional to T, for our algorithm without decomposition, and independent of T for our algorithm with decomposition. This storage saving feature enables much larger problems to be solved than were previously possible. Futhermore, we can easily incorporate upper bounds on inventories. This is not possible in the single period transportation equivalent.     

Warranty analysis and renewal function estimation

Estimation of the expected cost of a warranty for a stochastically failing unit is closely tied to estimation of the renewal function. The renewal function is a basic tool also used in probabilistic models arising in other areas such as reliability theory, inventory theory, and continuous sampling plans. In these other areas, estimation of a straight line approximation of the renewal function instead of direct estimation of the renewal function has proved successful. This approximation is based on a limit expression for large values of the argument, say t, of the renewal function. However, in warranty analusis, typically t is small compared to the mean failure time of the unit. Hence, alternative methods for renewal function estimation, both parametric and nonparametric, are presented and discussed. An important aspect of this paper is to discuss the performance of the renewal function estimators when only a small number of failed units is available. A Monte Carlo study is given which suggests guidelines for choosing an estimator under various circumstances.     

Markovian approximation for manufacturing systems of unreliable machines in tandem

This paper studies production planning of manufacturing systems of unreliable machines in tandem. The manufacturing system considered here produces one type of product. The demand is assumed to be a Poisson process and the processing time for one unit of product in each machine is exponentially distributed. A broken machine is subject to a sequence of repairing processes. The up time and the repairing time in each phase are assumed to be exponentially distributed. We study the manufacturing system by considering each machine as an individual system with stochastic supply and demand. The Markov Modulated Poisson Process (MMPP) is applied to model the process of supply. Numerical examples are given to demonstrate the accuracy of the proposed method. We employ (s, S) policy as production control. Fast algorithms are presented to solve the average running costs of the machine system for a given (s, S) policy and hence the approximated optimal (s, S) policy. © 2001 John Wiley & Sons, Inc. Naval Research Logistics 48: 65–78, 2001     

Deterministic and stochastic scheduling with teamwork tasks

We study a class of new scheduling problems which involve types of teamwork tasks. Each teamwork task consists of several components, and requires a team of processors to complete, with each team member to process a particular component of the task. Once the processor completes its work on the task, it will be available immediately to work on the next task regardless of whether the other components of the last task have been completed or not. Thus, the processors in a team neither have to start, nor have to finish, at the same time as they process a task. A task is completed only when all of its components have been processed. The problem is to find an optimal schedule to process all tasks, under a given objective measure. We consider both deterministic and stochastic models. For the deterministic model, we find that the optimal schedule exhibits the pattern that all processors must adopt the same sequence to process the tasks, even under a general objective function GC = F(f₁(C₁), f₂(C₂), … , f_n(C_n)), where f_i(C_i) is a general, nondecreasing function of the completion time C_i of task i. We show that the optimal sequence to minimize the maximum cost MC = max f_i(C_i) can be derived by a simple rule if there exists an order f₁(t) ≤ … ≤ f_n(t) for all t between the functions {f_i(t)}. We further show that the optimal sequence to minimize the total cost TC = ∑ f_i(C_i) can be constructed by a dynamic programming algorithm. For the stochastic model, we study three optimization criteria: (A) almost sure minimization; (B) stochastic ordering; and (C) expected cost minimization. For criterion (A), we show that the results for the corresponding deterministic model can be easily generalized. However, stochastic problems with criteria (B) and (C) become quite difficult. Conditions under which the optimal solutions can be found for these two criteria are derived. © 2004 Wiley Periodicals, Inc. Naval Research Logistics, 2004     

An integral equation for the second moment function of a geometric process and its numerical solution

In this article, an integral equation satisfied by the second moment function M₂(t) of a geometric process is obtained. The numerical method based on the trapezoidal integration rule proposed by Tang and Lam for the geometric function M(t) is adapted to solve this integral equation. To illustrate the numerical method, the first interarrival time is assumed to be one of four common lifetime distributions, namely, exponential, gamma, Weibull, and lognormal. In addition to this method, a power series expansion is derived using the integral equation for the second moment function M₂(t), when the first interarrival time has an exponential distribution.     

Nonlinear one-parametric linear programming and t-norm transportation problems

Previous methods for solving the nonlinear one-parametric linear programming problem min {c(t)^Tx |Ax = b, x ≥ 0} for t ? [α,β] were based on the simplex method using a considerably extended tableau. The proposed method avoids such an extension. A finite sequence of feasible bases (B^k | k = 1, 2, …, r) optimal in [t^k, t^k+1] for k = 1, 2, …,^r with α = t¹ < t² < … < t^r+1 = β is determined using the zeroes of a set of nonlinear functions. Computational experience is discussed in the special case of t-norm transportation problems.     

A note on randomly evolving hazard rate functions

Let τ be a finite stopping time with random hazard rate function (λ_t:t ≥ 0). We prove that στ λ_t dt is exponentially distributed with mean 1.     

Multiple dispatches in a poisson process

An optimal policy is characterized for operating the following system. Customers arrive in [O, T] according to a homogeneous Poisson process. Instantaneous services are provided at times O and T. Additional instantaneous services can be provided at N intermediate stop ping times. These times must be chosen to minimize the total expected customer-hours in [O, T] spent waiting for service.     

On the conditional residual lifetime of coherent systems under double regularly checking

In this paper, we consider a coherent system with n independent and identically distributed components under the condition that the system is monitored at time instances t₁ and t₂ (t₁ < t₂). First, various mixture representations for reliability function of the conditional residual lifetime of the coherent system are derived under different scenarios at times t₁ and t₂ (t₁ < t₂). Several stochastic comparisons between two systems are also made based on the proposed conditional random variables. Then, we consider the conditional residual lifetime of the functioning components of the system given that j components have failed at time t₁ and the system has failed at time t₂. Some stochastic comparisons on the proposed conditional residual lifetimes are investigated. Several illustrative graphs and examples are also provided.     

Estimation of the kinetic parameters for thermal decomposition of HNIW and its adiabatic time-to-explosion by Kooij formula

A differential/integral method to estimate the kinetic parameters (apparent activation energy E_a and pre-exponential factor A) for thermal decomposition reaction of energetic materials based on Kooij formula are applied to study the nonisothermal decomposition reaction kinetics of hexanitrohexaazaisowurtzitane (HNIW) by analyzing nonisothermal DSC curve data. The apparent activation energy (E_a) obtained by the integral isoconversional non-isothermal method based on Kooij formula is used to check the constancy and validity of apparent activation energy by the differential/integral method based on Kooij formula. The most probable mechanism function of thermal decomposition reaction of HNIW is determined by a logical choice method. The equations for calculating the critical temperatures of thermal explosion (T_b) and adiabatic time-to-explosion (t_TIad) based on Kooij formula are used to calculate the values of T_b and t_TIad to evaluate the thermal safety and heat-resistant ability of HNIW. All the original data needed for analyzing the kinetic parameters are from nonisothermal DSC curves. The results show that the kinetic model function in differential form and the values of E_a and A of decomposition reaction of HNIW are 3(1 − α)[−ln(1 − α)]^2/3, 152.73 kJ mol⁻¹ and 10^11.97 s⁻¹, respectively, and the values of self-accelerating decomposition temperature (T_SADT), T_b and t_TIad are 486.55 K, 493.11 K and 52.01 s, respectively.     

An M/M/1 queue with delayed feedback

We present some results for M/M/1 queues with finite capacities with delayed feedback. The delay in the feedback to an M/M/1 queue is modelled as another M-server queue with a finite capacity. The steady state probabilities for the two dimensional Markov process {N(t), M(t)} are solved when N(t) = queue length at server 1 at t and M(t) = queue length at server 2 at t. It is shown that a matrix operation can be performed to obtain the steady state probabilities. The eigenvalues of the operator and its eigenvectors are found. The problem is solved by fitting boundary conditions to the general solution and by normalizing. A sample problem is run to show that the solution methods can be programmed and meaningful results obtained numerically.