Military turns to AR/VR for combat and training pilots

India’s military, which has for years relied on state-owned companies for its defence equipment needs, is increasingly using products developed by startups using augmented reality (AR) and virtual reality (VR) as it seeks to upgrade its fighting capability.

The armed forces are tapping start-ups for applications ranging from training pilots to repairing aircraft, increasing the efficiency of weapons and even making tanks more resilient on the battlefield.

A see-through armour developed by Mumbai-based AR/VR startup AjnaLens, which raised ₹12 crore in a pre-series A funding last month, is a case in point. Called AjnaESAS (enhanced situational awareness system), it comprises an AR-based head-mounted display and a 360-degree camera that offers the crew inside a tank a 360-degree horizontal field of view. The camera system also has night vision and 4X zoom capabilities. “One of the limitations of tanks is visual awareness in terms of what is around the tank as they have to navigate through a small periscopic view, which makes navigation difficult,” said Abhishek Tomar, co-founder and chief technology officer, AjnaLens. “The AR-based solution can empower drivers to safely navigate through dust, sand and haze. This increases the survivability of the crew,” added Tomar who said that his company’s clients include the Indian Army, Indian Navy, and the Defence Research and Development Organisation (DRDO).

According to Pankaj Raut, co-founder and chief executive officer (CEO) of the firm, the Indian Army was looking for a solution that would allow a soldier inside a tank to “see through” its armoured body. The company created a prototype that received a grant of ₹1.5 crore from the defence ministry under the iDEX initiative for further development. “While the project is near completion and user test trials are successfully finished, the final field trials are yet to conclude. The timeline for deployment is yet to be decided,” said Raut.

However, electronic systems such as these can be hacked if connected to a network during battle, which is why the entire system is kept offline and accessible only to the tank’s crew, the company said.

The camera feed is generated and fed directly into the AR headset in real-time.

Others too are experimenting with AR and VR for defence. Krupalu Mehta, co-founder and CEO of Mumbai-based Parallax Labs, said several defence agencies have been doing proof of concepts and demos. Some of these products are now being deployed, even though the scale of implementation is still very small. Training to fly a plane and effectively using weapons are among potential growth areas where these AR/VR startups are seeing traction. For instance, Parallax has developed a VR-based personal flight simulator that offers a more cost-effective alternative to traditional flight simulators. Parallax deployed a VR simulator at a naval aviation unit in Goa in October 2021 and is in talks to deploy another at the combat army aviation training school in Nashik. Parallax is currently using its VR solution for training on Hindustan Aeronautics Ltd’s ALH Mk-III helicopters.

A spokesperson for the defence ministry didn’t immediately respond to an email seeking comments on the roadmap for adoption of AR/VR in defence.

According to Mehta, traditional simulators have a lot of hardware dependencies, including motion platforms, curved screens — all of which take up a lot of physical space. For training on a different aircraft, cockpit hardware has to be changed all the time. Training on a full flight simulator costs Rs. 1 lakh per hour. Also, multiple arms of defence rely on a single aircraft simulator in the Western Command in Bengaluru.

“Setting up a VR simulator can cost Rs.15-20 lakh. It can allow defence forces to install multiple simulators within their units. It also provides the option to have multiple aircraft and allow two pilots to collaborate on a single mission in training,” he added.

Training engineers for the repair and maintenance of aircraft is another area where VR is getting the nod. Mehta points out, every year, the government spends ₹800-900 crore on the maintenance of aircraft. For an aircraft like Sukhoi, one hour of flying requires five hours of maintenance. The operators need to be trained on maintenance and knowing the entire aircraft. VR solutions provide a view of the entire aircraft and helps discover faults faster. “We have also incorporated artificial intelligence (AI) to understand and pick up various faults and create randomized scenarios for trainees that can be put in the training module,” said Mehta. The training module is under development and will be up and running sometime in the second half of the year.

Apart from training, these technologies are also being used to enhance vision to engage with targets during combat. AjnaLens has an AR-based guiding system that can upgrade MANPAD (man-portable air defence system), which is used in portable surface-to-air missiles. These AR-based glasses are directly connected to the weapon's radar and the information relayed by them is converted into holographic images to provide an early warning on where and how far the target is, regardless of the weather conditions. 

According to Tomar, most of the weapons used in India are fair weather and not all weather. Misfirings on MANPADs can be expensive. One round can cost ₹3-4 lakh. Tomar said his startup is hoping to deploy 500 to 600 of these AR systems this year if it bags the government tender for it.

Elsewhere, in June 2021, Switzerland’s Department of Defense had commissioned a VR simulator for training. Mehta explains, getting a simulator commissioned means training on it will be counted as part of the official training hours. He expects India to commission VR simulators this year or next. In the US, Microsoft was contracted by the US Army to supply 120,000 Integrated Visual Augmentation Systems (IVAS) headsets, based on Microsoft's HoloLens. The timeline for testing and deployment has been pushed to the end of this year.