Modeling the operation of multireservoir systems using decomposition and stochastic dynamic programming

Stochastic dynamic programming models are attractive for multireservoir control problems because they allow non‐linear features to be incorporated and changes in hydrological conditions to be modeled as Markov processes. However, with the exception of the simplest cases, these models are computationally intractable because of the high dimension of the state and action spaces involved. This paper proposes a new method of determining an operating policy for a multireservoir control problem that uses stochastic dynamic programming, but is practical for systems with many reservoirs. Decomposition is first used to reduce the problem to a number of independent subproblems. Each subproblem is formulated as a low‐dimensional stochastic dynamic program and solved to determine the operating policy for one of the reservoirs in the system. © 2006 Wiley Periodicals, Inc. Naval Research Logistics, 2006     

What you should know about approximate dynamic programming

Approximate dynamic programming (ADP) is a broad umbrella for a modeling and algorithmic strategy for solving problems that are sometimes large and complex, and are usually (but not always) stochastic. It is most often presented as a method for overcoming the classic curse of dimensionality that is well‐known to plague the use of Bellman's equation. For many problems, there are actually up to three curses of dimensionality. But the richer message of approximate dynamic programming is learning what to learn, and how to learn it, to make better decisions over time. This article provides a brief review of approximate dynamic programming, without intending to be a complete tutorial. Instead, our goal is to provide a broader perspective of ADP and how it should be approached from the perspective of different problem classes. © 2009 Wiley Periodicals, Inc. Naval Research Logistics 2009     

Strategic capacity decision‐making in a stochastic manufacturing environment using real‐time approximate dynamic programming

In this study, we illustrate a real‐time approximate dynamic programming (RTADP) method for solving multistage capacity decision problems in a stochastic manufacturing environment, by using an exemplary three‐stage manufacturing system with recycle. The system is a moderate size queuing network, which experiences stochastic variations in demand and product yield. The dynamic capacity decision problem is formulated as a Markov decision process (MDP). The proposed RTADP method starts with a set of heuristics and learns a superior quality solution by interacting with the stochastic system via simulation. The curse‐of‐dimensionality associated with DP methods is alleviated by the adoption of several notions including “evolving set of relevant states,” for which the value function table is built and updated, “adaptive action set” for keeping track of attractive action candidates, and “nonparametric k nearest neighbor averager” for value function approximation. The performance of the learned solution is evaluated against (1) an “ideal” solution derived using a mixed integer programming (MIP) formulation, which assumes full knowledge of future realized values of the stochastic variables (2) a myopic heuristic solution, and (3) a sample path based rolling horizon MIP solution. The policy learned through the RTADP method turned out to be superior to polices of 2 and 3. © 2010 Wiley Periodicals, Inc. Naval Research Logistics 2010     

Generation and storage dispatch in electricity networks with generator disruptions

We present methods for optimizing generation and storage decisions in an electricity network with multiple unreliable generators, each colocated with one energy storage unit (e.g., battery), and multiple loads under power flow constraints. Our model chooses the amount of energy produced by each generator and the amount of energy stored in each battery in every time period in order to minimize power generation and storage costs when each generator faces stochastic Markovian supply disruptions. This problem cannot be optimized easily using stochastic programming and/or dynamic programming approaches. Therefore, in this study, we present several heuristic methods to find an approximate optimal solution for this system. Each heuristic involves decomposing the network into several single‐generator, single‐battery, multiload systems and solving them optimally using dynamic programming, then obtaining a solution for the original problem by recombining. We discuss the computational performance of the proposed heuristics as well as insights gained from the models. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 493–511, 2015     

运用动态规划的智能卫星集群星间通信路由算法

通过建立智能卫星集群网络模型,把智能卫星集群星间通信路由问题转换为时延最短路径问题,进而提出一种求解此问题的智能卫星集群星间通信路由算法.该路由算法采用动态规划策略分阶段规划智能卫星集群两个成员之间的星间通信路由,在每个规划阶段,负责发送数据的智能卫星自主调用一种星间通信路由静态规划算法,以求出其在当前时刻的后继卫星来...     

New heuristic methods for the capacitated multi‐facility Weber problem

In this paper we consider the capacitated multi‐facility Weber problem with the Euclidean, squared Euclidean, and ?_p‐distances. This problem is concerned with locating m capacitated facilities in the Euclidean plane to satisfy the demand of n customers with the minimum total transportation cost. The demand and location of each customer are known a priori and the transportation cost between customers and facilities is proportional to the distance between them. We first present a mixed integer linear programming approximation of the problem. We then propose new heuristic solution methods based on this approximation. Computational results on benchmark instances indicate that the new methods are both accurate and efficient. © 2006 Wiley Periodicals, Inc. Naval Research Logistics 2006     

Scheduling to minimize the total compression and late costs

We consider a single-machine scheduling model in which the job processing times are controllable variables with linear costs. The objective is to minimize the sum of the cost incurred in compressing job processing times and the cost associated with the number of late jobs. The problem is shown to be NP-hard even when the due dates of all jobs are identical. We present a dynamic programming solution algorithm and a fully polynomial approximation scheme for the problem. Several efficient heuristics are proposed for solving the problem. Computational experiments demonstrate that the heuristics are capable of producing near-optimal solutions quickly. © 1998 John Wiley & Sons, Inc. Naval Research Logistics 45: 67–82, 1998     

Robust capacity planning in semiconductor manufacturing

We present a stochastic programming approach to capacity planning under demand uncertainty in semiconductor manufacturing. Given multiple demand scenarios together with associated probabilities, our aim is to identify a set of tools that is a good compromise for all these scenarios. More precisely, we formulate a mixed‐integer program in which expected value of the unmet demand is minimized subject to capacity and budget constraints. This is a difficult two‐stage stochastic mixed‐integer program which cannot be solved to optimality in a reasonable amount of time. We instead propose a heuristic that can produce near‐optimal solutions. Our heuristic strengthens the linear programming relaxation of the formulation with cutting planes and performs limited enumeration. Analyses of the results in some real‐life situations are also presented. © 2005 Wiley Periodicals, Inc. Naval Research Logistics, 2005.     

Quadratic as parametric linear programming

This paper describes an approximate solution procedure for quadratic programming problems using parametric linear programming. Limited computational experience suggests that the approximation can be expected to be “good”.     

Robust shortest path planning and semicontractive dynamic programming

In this article, we consider shortest path problems in a directed graph where the transitions between nodes are subject to uncertainty. We use a minimax formulation, where the objective is to guarantee that a special destination state is reached with a minimum cost path under the worst possible instance of the uncertainty. Problems of this type arise, among others, in planning and pursuit‐evasion contexts, and in model predictive control. Our analysis makes use of the recently developed theory of abstract semicontractive dynamic programming models. We investigate questions of existence and uniqueness of solution of the optimality equation, existence of optimal paths, and the validity of various algorithms patterned after the classical methods of value and policy iteration, as well as a Dijkstra‐like algorithm for problems with nonnegative arc lengths.© 2016 Wiley Periodicals, Inc. Naval Research Logistics 66:15–37, 2019     

Parallel machine scheduling with batch deliveries to minimize total flow time and delivery cost

Motivated by the flow of products in the iron and steel industry, we study an identical and parallel machine scheduling problem with batch deliveries, where jobs finished on the parallel machines are delivered to customers in batches. Each delivery batch has a capacity and incurs a cost. The objective is to find a coordinated production and delivery schedule that minimizes the total flow time of jobs plus the total delivery cost. This problem is an extension of the problem considered by Hall and Potts, Ann Oper Res 135 (2005) 41–64, who studied a two‐machine problem with an unbounded number of transporters and unbounded delivery capacity. We first provide a dynamic programming algorithm to solve a special case with a given job assignment to the machines. A heuristic algorithm is then presented for the general problem, and its worst‐case performance ratio is analyzed. The computational results show that the heuristic algorithm can generate near‐optimal solutions. Finally, we offer a fully polynomial‐time approximation scheme for a fixed number of machines. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 492–502, 2016     

Integrated capacity,demand, and production planning with subcontracting and overtime options

Models for integrated production and demand planning decisions can serve to improve a producer's ability to effectively match demand requirements with production capabilities. In contexts with price‐sensitive demands, economies of scale in production, and multiple capacity options, such integrated planning problems can quickly become complex. To address these complexities, this paper provides profit‐maximizing production planning models for determining optimal demand and internal production capacity levels under price‐sensitive deterministic demands, with subcontracting and overtime options. The models determine a producer's optimal price, production, inventory, subcontracting, overtime, and internal capacity levels, while accounting for production economies of scale and capacity costs through concave cost functions. We use polyhedral properties and dynamic programming techniques to provide polynomial‐time solution approaches for obtaining an optimal solution for this class of problems when the internal capacity level is time‐invariant. © 2007 Wiley Periodicals, Inc. Naval Research Logistics, 2007     

用线性规划法设计复系数FIR滤波器

讨论了在复Chebyshev逼近意义下设计复系数FIR滤波器问题。直接把复Chebyshev逼近问题离散化成有限维线性规划问题 ,再用单纯形法求解这种方法一直被认为只能设计实系数滤波器 ,而且计算量大、收敛速度慢。本文从直接离散化出发 ,推导出一种求解此问题的改进的单纯形算法 ,适用于设计复系数滤波器 ,极大地减小了计算量 ,提高了收敛速度。并证明了它与通过求解半无限线性规划的对偶问题而得到的改进的单纯形法是等价的。最后给出了算法的仿真结果     

Economic lot sizing with constant capacities and concave inventory costs

This article studies the classical single‐item economic lot‐sizing problem with constant capacities, fixed‐plus‐linear order costs, and concave inventory costs, where backlogging is allowed. We propose an O(T³) optimal algorithm for the problem, which improves upon the O(T⁴) running time of the famous algorithm developed by Florian and Klein (Manage Sci18 (1971) 12–20). Instead of using the standard dynamic programming approach by predetermining the minimal cost for every possible subplan, we develop a backward dynamic programming algorithm to obtain a more efficient implementation. © 2012 Wiley Periodicals, Inc. Naval Research Logistics, 2012     

非高斯噪声下的滤波方法

干扰条件下的机动目标跟踪在一些文献[1][2]中已有讨论,但利用多传感器,尤其是被动传感器进行非高斯观测噪声条件下的目标跟踪仍需要研究。本文讨论了被动传感器在随机干扰条件下进行机动目标跟踪的方法,其观测量包含非高斯噪声,也可能包含影响观测值的随机干扰。与基于卡尔曼滤波的常见方法不同,采用动态规划算法进行多假设检验,从而估计目标的状态。仿真试验表明,本文方法能有效地处理非高斯噪声情况下的目标跟踪问题,而基于卡尔曼滤波的跟踪方法,比如EKF,则效果较差。     

An adaptive dynamic programming algorithm for a stochastic multiproduct batch dispatch problem

We address the problem of dispatching a vehicle with different product classes. There is a common dispatch cost, but holding costs that vary by product class. The problem exhibits multidimensional state, outcome and action spaces, and as a result is computationally intractable using either discrete dynamic programming methods, or even as a deterministic integer program. We prove a key structural property for the decision function, and exploit this property in the development of continuous value function approximations that form the basis of an approximate dispatch rule. Comparisons on single product‐class problems, where optimal solutions are available, demonstrate solutions that are within a few percent of optimal. The algorithm is then applied to a problem with 100 product classes, and comparisons against a carefully tuned myopic heuristic demonstrate significant improvements. © 2003 Wiley Periodicals, Inc. Naval Research Logistics 50: 742–769, 2003.     

A sequential dynamic pricing model and its applications

Consider a sequential dynamic pricing model where a seller sells a given stock to a random number of customers. Arriving one at a time, each customer will purchase one item if the product price is lower than her personal reservation price. The seller's objective is to post a potentially different price for each customer in order to maximize the expected total revenue. We formulate the seller's problem as a stochastic dynamic programming model, and develop an algorithm to compute the optimal policy. We then apply the results from this sequential dynamic pricing model to the case where customers arrive according to a continuous‐time point process. In particular, we derive tight bounds for the optimal expected revenue, and develop an asymptotically optimal heuristic policy. © 2004 Wiley Periodicals, Inc. Naval Research Logistics, 2004.     

Optimal capacity in a coordinated supply chain

We consider a supply chain in which a retailer faces a stochastic demand, incurs backorder and inventory holding costs and uses a periodic review system to place orders from a manufacturer. The manufacturer must fill the entire order. The manufacturer incurs costs of overtime and undertime if the order deviates from the planned production capacity. We determine the optimal capacity for the manufacturer in case there is no coordination with the retailer as well as in case there is full coordination with the retailer. When there is no coordination the optimal capacity for the manufacturer is found by solving a newsvendor problem. When there is coordination, we present a dynamic programming formulation and establish that the optimal ordering policy for the retailer is characterized by two parameters. The optimal coordinated capacity for the manufacturer can then be obtained by solving a nonlinear programming problem. We present an efficient exact algorithm and a heuristic algorithm for computing the manufacturer's capacity. We discuss the impact of coordination on the supply chain cost as well as on the manufacturer's capacity. We also identify the situations in which coordination is most beneficial. © 2008 Wiley Periodicals, Inc. Naval Research Logistics, 2008     

Shortest path interdiction problem with arc improvement recourse: A multiobjective approach

We consider the shortest path interdiction problem involving two agents, a leader and a follower, playing a Stackelberg game. The leader seeks to maximize the follower's minimum costs by interdicting certain arcs, thus increasing the travel time of those arcs. The follower may improve the network after the interdiction by lowering the costs of some arcs, subject to a cardinality budget restriction on arc improvements. The leader and the follower are both aware of all problem data, with the exception that the leader is unaware of the follower's improvement budget. The effectiveness of an interdiction action is given by the length of a shortest path after arc costs are adjusted by both the interdiction and improvement. We propose a multiobjective optimization model for this problem, with each objective corresponding to a different possible improvement budget value. We provide mathematical optimization techniques to generate a complete set of strategies that are Pareto‐optimal. Additionally, for the special case of series‐parallel graphs, we provide a dynamic‐programming algorithm for generating all Pareto‐optimal solutions.     

Optimal patrol to uncover threats in time when detection is imperfect

Consider a patrol problem, where a patroller traverses a graph through edges to detect potential attacks at nodes. An attack takes a random amount of time to complete. The patroller takes one time unit to move to and inspect an adjacent node, and will detect an ongoing attack with some probability. If an attack completes before it is detected, a cost is incurred. The attack time distribution, the cost due to a successful attack, and the detection probability all depend on the attack node. The patroller seeks a patrol policy that minimizes the expected cost incurred when, and if, an attack eventually happens. We consider two cases. A random attacker chooses where to attack according to predetermined probabilities, while a strategic attacker chooses where to attack to incur the maximal expected cost. In each case, computing the optimal solution, although possible, quickly becomes intractable for problems of practical sizes. Our main contribution is to develop efficient index policies—based on Lagrangian relaxation methodology, and also on approximate dynamic programming—which typically achieve within 1% of optimality with computation time orders of magnitude less than what is required to compute the optimal policy for problems of practical sizes. © 2014 Wiley Periodicals, Inc. Naval Research Logistics, 61: 557–576, 2014