基于多智能体强化学习的无人机集群对抗方法研究

针对复杂动态不确定环境下的无人机集群对抗问题,基于多智能体强化学习开展了对抗决策方法的研究。首先,基于MaCA环境构建了无人机集群对抗模型;其次,引入集中训练网络的混合架构模式,改进了传统DDPG算法,设计了面向无人机集群对抗的MADDPG算法,分别采用基于规则的对抗策略和基于DQN的对抗策略对算法进行了训练,提升了对抗算法的鲁棒性、适应性和泛化性;最后,通过搭建对抗仿真环境,验证了所设计方法的有效性和可靠性。     

微分博弈在无人机集群攻防中的应用研究

为了解决传统微分博弈理论的局中人规模受限问题,针对固定翼无人机目标-攻击-防御三方集群对抗场景,提出了一种基于微分博弈的集群攻防决策算法。该集群攻防决策算法利用基于Dubins路径价值函数的拍卖算法将集群攻防对抗问题解耦为目标-攻击者-防御者三方微分博弈问题,并且融入载体坐标系转换以及自动驾驶仪控制模型等约束,实现三维空间中以加速度为控制输入的集群决策,为等规模三方集群对抗问题提供协同决策控制。仿真实验结果证明,提出的集群攻防决策算法对于多种规模的无人机集群均能生成攻防角色对应的决策控制,实现包括目标-攻击-防御三种角色的三方集群攻防空战决策,在优化个体决策的同时,兼顾集群内无人机间的协同性。     

融合先验知识的异构多智能体强化学习算法研究

近年来,基于深度强化学习的机器学习技术突破性进展为智能博弈对抗提供了新的技术发展方向。针对智能对抗中异构多智能体强化学习算法训练收敛速度慢,训练效果差异大等问题,提出了一种先验知识驱动的多智能体强化学习博弈对抗算法PK-MADDPG,构建了双重Critic框架下的MADDPG模型。该模型使用了经验优先回放技术来优化先验知识提取,在博弈对抗训练中取得显著的效果。论文成果应用于MaCA异构多智能体博弈对抗全国竞赛,将PK-MADDPG算法与经典规则算法的博弈对抗结果进行比较,验证了所提算法的有效性。     

P3C-MADDPG算法的多无人机协同追捕对抗策略研究

针对策略未知逃逸无人机环境中多无人机协同追捕对抗任务,提出P3C-MADDPG算法的多无人机协同追捕对抗策略。首先,为解决多智能体深度确定性策略梯度(Multi-Agent Deep Deterministic Policy Gradient, MADDPG)算法训练速度慢和Q值高估问题,在MADDPG算法中分别采用基于树形结构储存的优先经验回放机制(Prioritized Experience Replay, PER)和设计的3线程并行Critic网络模型,提出P3C-MADDPG算法。然后基于构建的无人机运动学模型,设计追逃无人机的状态空间、稀疏奖励与引导式奖励相结合的奖励函数、加速度不同的追逃动作空间等训练要素。最后基于上述训练要素,通过P3C-MADDPG算法生成策略未知逃逸无人机环境中多无人机协同追捕对抗策略。仿真实验表明,P3C-MADDPG算法在训练速度上平均提升了11.7%,Q值平均降低6.06%,生成的多无人机协同追捕对抗策略能有效避开障碍物,能实现对策略未知逃逸无人机的智能追捕。     

基于改进强化学习的多无人机协同对抗算法研究

多无人机的作战协同研究内容主要包含飞行协同、侦察协同及干扰协同,随着无人机数量及协同决策内容的增加,多智能体强化学习模型的状态空间及动作空间维度呈指数增长,多智能体强化学习算法在训练中不易收敛,协同决策水平难以得到显著提升。采用并对多智能体深度确定性策略梯度(MADDPG)算法原理进行模型构建,在此基础上提出了一种选择性经验存储策略的多智能体深度确定性策略梯度(SES-MADDPG)算法。该算法通过设置回收存储标准以及选择性因子,对进入经验池的经验进行选择性存储,以缓解奖励稀疏的问题。仿真实验表明,在保证算法时间复杂度的前提下,SES-MADDPG算法比其他强化学习算法有了更好的收敛效果,相较于MADDPG算法,任务完成率提高了25.427%。     

基于DRL与微分对策的无人机空战决策研究

随着无人机战场环境越来越复杂,空战对抗将逐渐成为主要的一种无人机作战方式.为了能够确保我方无人机在快速演变的战场态势下抓住先机、精确决策、快速致胜,需要根据实际作战环境、作战样式,建立无人机和环境进行交互的规则、无人机空战对抗中采用的战术使用规则,并结合规则,通过智能决策算法,达到提升无人机空战对抗胜率的目的.提出一种结合微分对策(Differential Games,DG)的深度强化学习方法(Deep Reinforcement Learning,DRL)解决此问题,利用深度强化学习的智能决策性以及微分对策的准确机动性,实现战术决策到机动决策.最后以空战对抗1V1为例,对提出的方法进行验证,结果证明方法可行有效.     

基于深度强化学习潜艇攻防对抗训练指挥决策研究

潜艇和水面舰艇编队间的攻防对抗是潜艇作战研究的重点内容,如何确保潜艇在舰艇编队、反潜直升机等兵力的联合封锁下存活和突围,是对潜艇指挥决策的考验.为此,针对潜舰机博弈对抗场景,从深度强化学习和规则推理两个方面构建潜艇智能体,提出两种近端策略优化(Proximal Policy Optimization,PPO)算法改进机...     

通信对抗无人机集群侦察区域规划规则研究

侦察区域规划是通信对抗无人机集群智能侦察的首要任务.能否合理规划侦察区域,对通信对抗侦察任务的完成至关重要.通过研究侦察区域规划分配和动态调整流程,结合通信对抗战斗原则和相关军事经验,整理归纳出通信对抗无人机集群进行侦察区域规划的行为规则、判定规则和策略规则,并提出了规则的使用建议,构想了具体应用场景.     

面向体系仿真的智能无人机集群作战建模总体框架研究

提出了智能无人机集群作战建模应当遵循的基本理念.在总体上将"面向体系仿真的智能无人机集群作战建模"视为一个"概念设计—模型建立—分析应用"的完整过程,而非一个孤立的建模仿真活动.为此,提出了涵盖"体系设计—模型训练—想定生成—探索仿真—结果分析"5个基础阶段的建模总体框架,以及包含支撑层、运行层、分析层和应用层在内的能够支撑该框架使用的系统体系架构.该建模总体框架,有助于解决智能无人机集群作战建模中作战概念不清晰、应用场景不明确的问题,同时可以将基于机器学习的智能实体建模技术集成到建模框架当中,以充分反映智能无人机实体及其集群的自学习、自成长特性.     

基于深度强化学习的智能对手自主空战决策技术

为提升空战训练对手的智能性与自主性,提升空战训练效果。针对空战战术训练智能对手自主空战决策问题,提出一种基于深度强化学习的智能体训练方法,采用最大熵强化学习（SAC）算法平衡策略探索与利用的优势,引入自博弈和多智能体联盟训练方法提升空战智能体策略的多样性和鲁棒性。针对一对一近距格斗空战场景建立智能博弈框架及奖励函数,仿真结果表明,基于零经验训练得到的智能体能够有效自主机动决策并实施近距导弹攻击,产生较好的战术效果,证明该方法在一对一近距格斗空战智能体训练中的有效性。     

Putin – the masked nemesis of the strategy of ambiguity

Recent events demonstrate the complex and adaptive approach employed by Russia to reassert influence in Europe. The changing face of Russia’s strategy commenced in 2007 when it launched a crippling cyber-attack against Estonia. This was followed by a large Russian conventional attack against Georgia in 2008, occupying two large areas of the nation. 2014 witnessed the Russian annexation of Crimea where in just a week, Russia seized control of Crimea “without firing a shot.” The annexation of Crimea was rapidly followed by a Russian inspired and led subversive war in eastern Ukraine. The common thread among these diverse Russian operations is its use of ambiguity to confound and confuse decision makers in the West.     

India's counter-insurgency experience: the ‘trust and nurture’ strategy

The Indian Army, a force trained primarily for conventional warfare, has been engaged in internal counter-insurgency operations since the 1950s. Despite such a long innings on a counter-insurgency mode, little attention has been accorded within military circles to doctrinal innovation for waging sub-conventional warfare in India's democratic political context. At best, the Army continues to view counter-insurgency duty as secondary to its primary duty of defending India from external conventional threats. By conceptualizing a counter-insurgency strategy of ‘trust and nurture’, this article aims to fill this critical doctrinal gap in India's military policy. The author argues that a counter-insurgency strategy of ‘trust and nurture’ based on democratic political culture, measured military methods, special counter-insurgency forces, local social and cultural awareness and an integrative nation-building approach will result in positive handling of India's internal security problems. The author utilizes India's counter-insurgency experiences in Assam, Mizoram, Nagaland, Punjab, and Operation ‘Sadhbhavana’ in Jammu and Kashmir as illustrative empirical indicants in order to validate the ‘trust and nurture’ strategy.     

Helping hands: civil–military cooperation and Italy's military operation abroad

In the Post-Bipolar Era the growing complexity of the military operations requires a new approach for the resolution of international crises. Since the end of the Cold War, peace support operations (PSO) have become the mainstay and principal occupation of most Western armies. At the same time, Italy has been one of the most important actors in such an area. The article focuses on the cooperation between military and civil components (a process called CIMIC) as a key variable in the Italian PSOs. We will analyse in detail the main lessons learned from past military interventions as well as the general context in which new tendencies are taking place. The maintaining of a minimum security frame becomes essential to fulfil activities ‘collateral’ to the mission: reconstructing services and infrastructure, food distribution, water and medication, law and order, de-mining, training of local forces, and supporting local institutions. These are the main tasks to obtain thrust and support from the population.     

Terrorism,insurgency, kidnapping,and security in Africa's energy sector

Energy continues to serve as the bedrock of modern economies and the main driver of modern society. For Africa, the production and supply of energy resources such as crude oil, natural gas, uranium, coal, biomass, biofuels and other renewables are an important source of employment, rents, taxes, royalties and profits. This sector brings in several tens of billions of dollars of revenue annually. The production and delivery of such resources, however, depend on critical infrastructures such as pipelines, refineries, processing plants, terminals, rigs, electrical energy pylons, substations, pump stations, vessels, and tankers. These infrastructures have been attacked by terrorists, insurgents, vandals and saboteurs, all of whom see them as targets against which to register their grievances and extract concessions from the state. This paper is a chronological account of some of the documented incidents of terrorism, insurgency, kidnapping, destruction, sabotage, and human casualties suffered in the oil and gas sectors in Africa between 1999 and 2012. It is based on data extracted from the databases of the RAND Database of World Terrorism Incidents and the University of Maryland's Global Terrorism Database (GTD).     

Tacit support for terrorism: The rapprochement between the USSR and Palestinian guerrilla organizations following the 1967 war

This article demonstrates the inconsistent and wavering Soviet attitude towards national liberation movements in general and the Palestinian organizations in particular. Until the late 1960s, the Soviets viewed these organizations with suspicion, hesitating to engage in political dialogue with them. However, in the 1970s, political and military events in the region, as well as modifications in the Kremlin's Cold War strategies, led to a general shift towards the Middle East in Soviet foreign policy. Soviet leaders showed increased willingness to provide certain Palestinian organizations with arms with which to conduct terrorist activities against Israeli, pro-Israeli, Jewish and Western targets. The article explores the complex relations between Palestinian organizations and the USSR in the field of international terror. The study also exposes and analyzes the nature and content of Soviet–Palestinian arms dialogues and transactions. It provides clear evidence that Soviet policymakers and other luminaries were fully informed of, and sometimes directly involved in, these transactions and dialogues at the highest levels.     

A1 Qaeda and the radical Islamic challenge to western strategy

The September 11 global crisis prompted by the attacks on the World Trade Center and the Pentagon raises major questions concerning the nature and trajectory of terrorism in the post-Cold War global order. Hitherto, terrorism has been largely debated by analysts at the level of nation states. Terrorist and insurgent movements have also been largely anchored in nationalist and ethnic power bases even when they have sought to mobilise a transnational ideological appeal on religious or class grounds. There have been a few exceptions to this pattern such as the alliance between the German Baader-Meinhof group and the Japanese Red Army Faction, but even such international alliances as this did not, until at least the 1980s, presage anything like a global terrorist network necessitating a global strategic response. This study examines terrorism and global strategic responses.     

Helping hands: external support for the KNU insurgency in Burma

This article analyses to what extent the Burmese KNU insurgency made use of external support from states, refugees, and diasporas. Based on extensive fieldwork it is concluded that support from neighbouring states and refugees has for years kept the Karen rebellion alive. Western countries perceived forms of resistance to the illegitimate Burmese regime as just and have therefore played a crucial role in the continuation of conflict in Karen State. It is important that policymakers and donors as well as executing organizations continue to reflect critically on the way they exercise their work.     

Reviews

Douglas S. Derrer, We Are All the Target: A Handbook of Terrorism Avoidance and Hostage Survival. Annapolis, MD: US Naval Institute Press, 1992. Pp. x+135, notes, index. $14.95. ISBN 01–55750–150–5

Ian Knight, Zulu: Isandlwana and Rorke's Drift, 22–23 January 1879. London: Windrow & Greene, 1992. Pp.136, 150 illus., incl 8 colour plates, maps, biblio. £35. ISBN 1–872004–23–7

Ian Knight, By the Orders of the Great White Queen: Campaigning in Zululand through the Eyes of the British Soldiers, 1879. London: Greenhill Books and Novato: Presidio Press, 1992. Pp. 272, 17 illus., 1 map. £18.95. ISBN 1–85367–122–3

Manfried Rauchensteiner and Erwin A. Schmidl (eds.), Formen des Krieges: vom Mittelalter zum ‘Low‐intensity’ Conflict’. Graz : Verlag Styria, 1991. Pp.208. DM35. ISBN 3–22–12139–7

Harold J. Kearsley, Maritime Power and the Twenty‐First Century. Dartmouth: Dartmouth Publishing Company, Limited, 1992. Pp.xv + 203, 13 diagrams, index. £32.50. ISBN 1–85521–288–9     

基于GTST-DMLD-ESD的装备RMS建模仿真

装备的RMS是装备的重要设计参数,是影响战备完好和保障能力的关键因素.引入GTST-DMLD作为主要建模工具,建立起装备系统的RMS描述模型,以ESD为补充和扩展,对维修过程进行建模.并以GTST-DMLD-ESD模型作为装备RMS仿真的核心,对装备RMS仿真模型的总体框架进行了研究,并对各模块进行了较为详细的探讨.