Imagine being transported into a world where battle plans unfold in vibrant holograms, and strategy feels like second nature. Back in 2013, Ender’s Game dazzled audiences with its futuristic take on soldier training – using breathtaking holographic simulations to prepare for a fight against an unstoppable enemy. Fast forward to today, and we’re already stepping into a similar reality. Games like Tom Clancy’s Ghost Recon let players dive into virtual battlefields, plotting moves and outmaneuvering enemies with augmented precision. What once seemed like pure science fiction is now shaping the tools of modern conflict and decision-making. This phenomenon of blending the physical and virtual world through an interactive interface is part of ‘spatial computing’ that allows machines to comprehend our physical environment – a combination of Virtual Reality (VR) and Augmented Reality (AR) technologies.
VR and AR are no longer just buzzwords; they’ve become essential tools in modern warfare. VR immerses soldiers entirely into alternative virtual environments, creating hyper-realistic scenarios for simulated training or mission planning. AR, on the other hand, enhances the real world by overlaying additional virtual layers – be it visual, acoustic, or haptic – on the physical environment, sharpening the sensory perception of soldiers, sailors, and pilots. In 2019, Israeli Defence Forces (IDF) trained for tunnel warfare against Hezbollah without physically being underground. The trainings were conducted through VR headsets that transported the soldiers to a simulated tunnel environment, and may have been instrumental vis-Ă¡-vis IDF’s mission planning for subsequent military operations against Hezbollah.
A prominent AR model is the Integrated Visual Augmentation System (IVAS) with the Unoted States military. Launched in 2018 in collaboration with Microsoft, it is a USD 22 billion project that constructs an AR-driven synthetic display via high-tech goggles that upgrade a soldier’s sensory perception, target acquisition and target engagement. In 2024, Microsoft integrated AI into this system enabling it to autonomously detect threats and reinforce soldiers’ tactical superiority on the battlefield. This upgrade would also enable the goggles to receive data from other platforms such as drones and aircraft in real time, potentially reflecting the nexus of spatial computing, AI and the Internet of Things (IoT) on the battlefield.
The Chinese People’s Liberation Army (PLA) is also testing the Military Augmented Reality System (MARS) headset which is reportedly not as advanced as IVAS but it could become an integral feature in China’s high-tech military transformation. As showcased in trials, it elevates a soldier’s visibility, effectively allowing them to identify and engage targets from behind a cover. Thus, these AR-powered headsets could break the opacity of cover and substantially increase the lethality of warfare.
While the potential of VR and AR in modern warfare is undeniable, it’s important to acknowledge that systems like IVAS, MARS, or similar technologies have yet to see real-time deployment on an actual battlefield. This gap underscores the challenges that still need to be addressed. One major hurdle lies in the military applications of spatial computing, which are constrained by inherent limitations. For instance, the sheer volume of data streams these systems generate can overwhelm soldiers, leading to the risk of information and cognitive overload. This is likely the reason why IVAS users complained of side effects such as nausea and dizziness. Cognitive overload could eventually retard rather than enhance situational awareness in combat. Moreover, the data-intensive and AI-augmented nature of this technology raises legitimate apprehensions regarding cyber threats such as data breaches or jamming. For example, researchers at the University of Chicago were able to exploit security vulnerabilities in Meta’s Quest VR system. Similar threats prompted the US military to work with high-tech contractors to develop preventive measures for its devices. In a cyber-attack scenario, inaccurate or manipulated projection over the headset may not only jeopardise a soldier’s life but also operational success as a whole. Extreme climates, such as scorching heat or freezing cold, as well as connectivity challenges in remote or contested areas, could also notably hinder their functionality. These vulnerabilities raise critical concerns about the reliability and resilience of such systems when deployed in real-world battlefield conditions. Importantly, the exorbitant costs associated with these devices could limit adoption and militarisation of this technology. Evidently, two of the most sophisticated devices in this regard – Microsoft’s HoloLens 2 and Apple’s Vision Pro – cost around USD 3500 per unit. The US Military’s IVAS system uses HoloLens devices.
High-tech advancements hinge on overcoming complexities to maximise the utility of emerging technologies. To address cognitive overload, rigorous R&D and regular trials are essential. Machine learning (ML) can play a pivotal role by filtering massive data streams and prioritizing critical information in real-time. Moreover, ML-driven encryption and advanced security protocols are vital to safeguard devices from cyber threats, with indigenisation emerging as a key solution for long-term cybersecurity resilience. Equally important is the durability of VR/AR hardware. Developing headsets with weather-resistant, adaptive materials will be crucial for their reliability in extreme operational environments. Given the high costs associated with such technology, smaller militaries would consider small-scale pilot projects and embrace Public-Private Partnerships (PPP) to share expenses and accelerate innovation.
The fusion of physical and virtual domains in warfare is not without its challenges – ethical dilemmas like in-built biases, over-reliance on technology, and reliability concerns in extreme conditions remain significant hurdles. Yet, these obstacles pale in comparison to the transformative potential of spatial computing. Dominance in the virtual realm is poised to become as crucial to victory as strategy, organisational strength, and geopolitical positioning. The sooner this reality is embraced, the better prepared nations will be for the battles of tomorrow. What once was science fiction is rapidly becoming science fact – reshaping the very nature of conflict and power.
Shah Muhammad is a Research Assistant at the Centre for Aerospace & Security Studies (CASS) in Islamabad, Pakistan. The article was first published in Modern Diplomacy. He can be reached at: [email protected]
