Smart teaming

Mark Ayton spoke with members of Textron’s test team about the recent manned-unmanned teaming flight demonstration

MANNED-UNMANNED TEAMING TECHNOLOGY

There is nothing new about operating UAVs. It’s been done the world over for decades. Air vehicle manufacturers continue to develop ever greater capabilities and operators similarly employ their air vehicles in ever more challenging roles, environments and with ever longer mission endurance. In all but a few flights, the UAV has until now been under the control of a land-based or seabased operator. No longer is that the case.

In July, Wichita, Kansas, became the location from where the first proof-of concept demonstration involving aircrew in an airborne aircraft controlling UAVs successfully took place. Why Wichita? Because it’s the home of Textron Aviation’s defence division, manufacturer of the Scorpion jet, the type used in the demonstration. Scorpion was, though, only half of the hardware used in the demo. Sister company Textron Systems provided its Synturian open architecture control system, the software tool that allowed connectivity to and control of UAVs from the aft seat of a Scorpion jet in the skies of Kansas.

Synturian’s legacy

The bread-and-butter work of Textron Systems is primarily UAVs, a business line since the 1980s with a product portfolio that includes tactical UAV Pioneer, Shadow and Aerosonde. All three are operated from ground control stations, so whilst they are autonomous, they do have pilots in the loop monitoring their operation and mission set. Textron Systems’ Universal Ground Control Station is used by the US Army to operate the Hunter, Shadow and Gray Eagle UAVs, and the company also produces the One System Remote Video Terminal. The latter is a handheld device that allows a user or third party to receive video and meta data from a UAV. The terminals have been used by US troops to watch video imagery of the periphery of their location for situation awareness, fed to them by a nearby UAV. Textron Systems’ expertise in command and control and providing remote situation awareness led to some smart thinking about new capabilities. According to Wayne Prender, Vice-President of Applied Technology and Advanced Programs, the company saw two things: “One, the current state of the art is generally a one-to-one relationship between a control station or a control panel and a UAV; and two, they are generally confined and not mobile. From a user interface standpoint, such systems have their legacy back in late 1990s and early 2000s, but user interfaces have really developed and matured significantly since that point in time, so we started an internal company programme to develop a system with three objectives. One, the ability to control multiple unmanned systems, not just UAVs, whether they be air, ground or maritime, and operate them simultaneously; two, to make the user interface significantly more intuitive, make it easier to operate, make it less taxing on an operator, make it more game like, without removing any capability; three, we wanted to make the system scalable in nature, make it something that functions on a tablet-sized device or a traditional ground control station, all the way up to a command and control operations centre. The three objectives generated the system now called Synturian.”

Synturian in operation. Note the halo icon on the top right of the aft cockpit display during the first demonstration flight on May 4.
All images Textron Defense Aviation

Discussions about the future of mannedunmanned teaming with partner companies Textron Aviation and Bell concluded in recognition for a need to be able to provide additional teaming capabilities for manned aviation platforms, which then spurned the initiative and demonstration with the Scorpion jet at Wichita.

Textron’s initiative demonstrates the level of confidence in where Synturian-type technology can be further developed and utilised in an operational environment. The initiative also enables Textron to respond to future requirements from the US Army, US Navy and the US Air Force, each of which has published guidance documents for the future use of UAVs and mannedunmanned teaming. Loyal Wingman is a US Air Force concept, while the US Army’s is known as Advanced Teaming. Wayne Prender reckons the Army’s terminology is a smart phrase, because advanced teaming opens up people’s minds to concepts like unmanned-unmanned teaming, as well as manned-manned teaming, enabled by unmanned technologies.

Synturian’s core capabilities and its functionality

Synturian is map-centric with a military orientated modular open architecture system that is hardware agnostic and able to operate on different operating systems, including Windows as per the Scorpion jet.

Synturian’s primary capability is to operate multiple UAVs simultaneously even when the individual UAVs are utilising different communication protocols, compared to most control stations that tend to be limited by a specific standard or communication protocol. Synturian must therefore present information to the user not only for the UAV under active control, but also the others for which he or she has responsibility. Textron engineers have developed techniques and protocols that enable this to happen easily.

A Textron Systems engineer oversees the Synturian system in operation during the May 4 demonstration flight, from a work station positioned adjacent the Scorpion aircraft which acted as a surrogate UAV.

Wayne Prender explained further display functions: “Synturian has the ability to not only have a separate screen or window for video, but also to display the image collected by the UAV, the so-called video feed. This is geo-rectified and overlaid on to the map being monitored by the user. That helps signiicantly when coordinating with ground units and helping them monitor a target, because it provides better situation awareness, especially for quick orientation when monitoring multiple vehicles or during an ISR mission where the video-fed image is in reference to the air vehicle.”

Because different UAVs use slightly different commands, as part of the concept to control multiple UAVs, Textron has developed a haloshaped moving icon in which all command and control functions are physically colocated and represents that UAV on the screen. The system removes any doubt about which of the multiple UAVs is under a particular user’s control.

The control functions are standardised to overcome the issue of different protocols between different types of UAV, thereby providing the user with seamless interaction with each vehicle.

Demonstration flights

Ahead of the two demonstration lights, Wayne Prender said the team was very confident the integration of the software and hardware solutions would go smoothly. The key accomplishment would be getting the Textron light test aircrew to demonstrate how they could utilise the capability in an operational environment: new tactics, techniques and procedures, how better to position their aircraft to conduct their part of the mission and not only what aspects of the mission could be offloaded to and undertaken by a UAV, but also how many UAVs they could manage simultaneously. Prender thinks this is where the next steps in experimentation and use of this concept needs to be. He explained that a Scorpion jet would be used as an airborne, mobile, command and communication node operating multiple UAVs in an operational environment. This will start to flush out how manned and unmanned systems would then mature over time to make the new system of systems more effective. He said: “The manned platform was designed for its mission and the unmanned platform similarly was designed for its mission. Now we are finding out how to better utilise them together. That’s all good and positive, but when you start to take them together from a fresh view or a ground-up system of system design element, there may be certain trades or different design decisions you make, because they are going to work collaboratively. This is just step one in what we believe will be a game-changing marriage of platforms and technologies.”

Outlining the two demonstration lights Dan Hinson, Chief pilot with Textron Aviation Defense, said: “The team established it had everything required in the Scorpion to control a UAV using a laptop computer loaded with the control software and fitting specific radios and an additional antenna to broadcast the signals. The scenario involved one Scorpion lying as the control node and a second Scorpion on the ground simulating a UAV, and a control station on the ground performing both control station functions and simulation of other air vehicles. That was all accomplished within four weeks of the decision being made to conduct the light.”

Wayne Prender said: “The team used simulation because lying a UAV from a control station is a well-established process. What we really wanted to flush out was the integration of the hardware, software and the datalinks used and their operational use by aircrew. We are confident that was the right first step, it allowed us to integrate very rapidly and prove the concept. The next step is to mature the system further, and start to operate live UAVs to determine how the various systems can best be optimised to work in a collaborative environment.”

For the demonstration, the Synturian systems installed on the Scorpion aircraft operated on the jet’s existing Windows-based mission computer and used the aircraft’s Vortex C-band datalink to send commands back and forth. The airborne Scorpion downlinked commands to the second Scorpion on the ground, which in response uplinked messages confirming receipt of the commands, thereby mimicking a UAV. In the words of senior light test engineer James Bearman: “It was the C-band datalink that made this possible and the first time we had two-way communications on the datalink.”

Textron’s senior light test pilot, Brett Pierson, explained the use of live virtual constructive (LVC) testing. He said: “Because of airspace limitations use of a live UAV was out of the question, so we went with a live Scorpion and beamed down to a virtual UAV. The constructive component involved controlling the sensors of the virtual drones, because the virtual drones are loaded with actual maps, so we could slew the sensors around from the cockpit and see on our screens what that drone would be seeing had it been real.

Synturian operates on the Scorpion’s existing Windows-based mission computer.
Synturian operates with one primary field of view and uses halo icons to denote each specific UAV under its control. UAV controls are provided on a pop-up graphic user interface seen in this shot at the top of the icon.

“We have accomplished everything as far as we can on the LVC piece so the logical follow-on that we would like to do is to gain access to the proper UAVs and some restricted airspace to be able to ly Scorpion in and take control of the UAVs already airborne: literally grab control of them through the software. Beyond that, the visionary aspect and true innovation would be to have the capability to tube-launch UAVs from the wing, thereby bringing the UAV to the ight on our wing, launch it, control it and use it accordingly.”

Sortie specifics

The first demonstration light on May 4 involved the aircraft lying overhead the ground-based Synturian operator (a Textron Systems engineer) who simulated launching the UAV. Brett Pierson took up the scenario: “The airborne aircrew go to a detection mode, one that detects airborne UAVs. Two systems, an Aerosonde and a Nightwarden, were detected. By clicking on each halo icon as displayed on the screen map, the system goes to a function that allows the aircrew in the back seat to take control. That process involves passing the control of the air vehicle from the ground-based operator to the aircrew in the aircraft’s back seat. An acknowledgement is made to confirm airborne Scorpion has control.

“The whole beauty of what Textron Systems has developed is that control of the UAV is all commanded by mouse clicks. For example, you command the system to ly to this point, at this altitude, at this airspeed simply by a mouse click and the software translates that to pitch, roll, yaw throttle commands. James was not physically lying the drone so much as commanding it, and that difference in workload, while the drone is transiting, allows you to call up on your mission display in Scorpion the imagery fed by the UAV sensor. We repeated the test on May 7 with Dan Hinson performing all the controls.”

Dan Hinson provided his view of operating the system from the back seat of Scorpion. He said: “If you can load the system on a laptop you can put it in our aeroplane. Learning to use the system in the back seat is very intuitive and very easy training. Controlling rather than lying the UAV was the difference that enabled us to achieve success with the demonstration.

“From a workload perspective, it’s low even when managing a couple of different systems at the same time because you can turn away from controlling the UAV and do something else knowing that if the UAV was to run out of commands it will enter an orbit at the point it has been commanded to go, even to the point when the air vehicle reaches its minimum fuel and recovers to base. That level of workload is what allows the aircrew to be a force multiplier controlling multiple air vehicles. It makes the back seat of the Scorpion a battle management node or an information collector.”

Adding to Hinson’s comments, Pierson said his colleague’s observation of acting as mission commander leads to a potential revolution in aerial warfare by extending the aircraft’s sensor horizon and therefore the weapons’ horizon with the ability for the aircraft to remain in a safe permissive airspace while UAVs are sent to do the dangerous work.

The main test objectives for the first flight were simply to get the two Scorpions communicating with each other and to take control of the UAV. James Bearman said that since that all went so well, the next test points to accomplish with a single UAV were simple tasks like changing altitude, airspeed, or the orbit point. He said: “They too were easy to accomplish, so we moved on to more complicated tasks. Can we build a flight plan and task the drone, such that if the drone cannot complete that plan can we invoke a contingency plan to fall back on?

“I created the flight plan by clicking points on the map, and sent the UAV on its way with a contingency plan in place. For example, hold over this target area and establish a 3-mile orbit. That was all completed within the first 30 minutes of us flying around and once the communication link was established between the two Scorpions.

“Synturian operates on one primary field of view and all of the controls on the halo icon follow the drone around such that I click on the particular drone I wish to control and a pop-up graphic user interface appears. You click commands on that graphic user interface; up and down is used for altitude, left and right for airspeed. There is another popup graphic user interface for building a flight plan, so Textron Systems has made the system really easy for the aircrew to use.

“Since that all went so well and we still had 45 minutes of fuel remaining, we opted to introduce a second drone. We had discussed this option, but had not done a hard brief before the flight. Once the second drone was placed on the simulator, I took control of that and it proved to be just as easy to control as the first one. From my perspective, I simply clicked between the two drones and used the respective graphic user interface to control each one.”

Further objectives

To date, Textron has not determined a limit for the number of drones that could be controlled. Nor are UAVs the limit. In future trials there is nothing that prevents the test team from flying and controlling an unmanned surface vessel (USV) on the ocean, with a drone flying above it using tactics suitable for that scenario.

Discussing future Synturian trials, Dan Hinson said he would opt to fly at medium altitude above a cloud deck, then launch UAVs and control them. He said: “The point being, Scorpion’s above the clouds controlling a system flying beneath the cloud or picking up control of a ground-launched drone, and thereby demonstrating the potential tactical benefits of manned-unmanned teaming.”

Wayne Prender provided detail about the company’s future objectives. He said: “We think the system has the ability to fundamentally change how UAV operations are conducted. One of the drivers for future UAV operations is to decrease the footprint such that one person can operate multiple UAVs to reduce the equipment and manning footprint. Then the ability for one certified pilot or UAV operator to manage multiple UAVs in an unmannedunmanned teaming scenario with the operator maintaining positive safety control, and handing offpayload and flight navigation control to a forward user.

“The fundamental shift we believe this system can provide involves multiple to multiple concept of operation which might be employed on a variety of military applications for airborne and ground assets, dismounted users or in the maritime domain.

“This also starts to open up not only multiple vehicle control but multi-domain control – for example, a US Navy vessel operating a UAV and a USV in collaboration, from the same control station. The UAV provides situation awareness over a much larger area given its altitude, but can’t actually interdict or engage a surface target; the USV can. In the case of a search and rescue mission, you now have the ability to identify a survivor in the water and deploy a small USV to go and recover them or drop offa raft. That can all be done in collaboration using a single operating picture. Synturian is not only scalable to different types of control stations, but also in the class of UAVs it can control.

Summing up, Prender said the team is very proud that it achieved the project in a very short space of time: “We took our Synturian product and the Textron Aviation Defense Scorpion and in less than four weeks we were able to put our software into the back seat of the Scorpion and move through a pretty quick integration effort and then step through a series of ground and ultimately to flight tests to prove that the concept. We were also able to showcase this to a potential customer, who was able to witness what we were doing and gave very positive feedback.”