Algorithms Turn Autonomous Agents into a Self-Learning System

As revelers rang in the new decade, Singapore’s drone show became a trending story in the Twitterverse. The show incorporated 500 drones flying in different formations, including a countdown clock, delighting spectators with 3-D animations against the night sky. The performance required exquisite control and coordination as individual drones moved in different directions at varying air speeds, each arriving at its precise location at the precise time in relation to the group.

This type of choreography is based in control theory and its applications, a specialty of Zongli Lin, Ferman W. Perry Professor of electrical and computer engineering at the University of Virginia School of Engineering. Lin’s know-how underpins signals and systems research, a strength of UVA’s Charles L. Brown Department of Electrical and Computer Engineering.

Lin’s research is inspired by biologists who study swarming behaviors. Examples include schools of fish, flocks of birds and colonies of bees or ants, as well as antelope herds and the wolf packs who hunt them. Engineers think of a swarm as a set of agents that act collectively to perform a task, with each agent taking its cue from a small number of nearby agents.

Distributed coordination of a multi-agent system, such as a drone swarm, is the specific research problem that has captured Lin’s attention. “No single agent has a complete picture. Each agent can interact with its neighbors but cannot tap into the global knowledge of the swarm,” Lin explained.

Distributed coordination depends on each agent’s ability to sense, process and communicate information, turning the swarm into a self-learning system. Connectivity and information sharing organically emerges within the system as agents come into close range or cross paths.

Lin’s work applies to drones in the air and vehicles on land and underwater. Air, land and sea form the triad of situational awareness critical to combat, peacekeeping and humanitarian missions. The search for enhanced capabilities in each space earned Lin a grant from the U.S. Army Research Office. For these missions, a system of autonomous agents might be tasked to conduct wide area surveillance, map an area, search for or pursue a target, clear a mine or carry supplies.

Lin’s algorithms lead individual agents to agree on the most desirable way to achieve a group task; to perform tasks in small groups; and to follow the agent best able to lead the group at any point in time during the mission. Beyond core functions, Lin’s algorithms allow the system and its agents to perform well in remote and harsh environments and to overcome adverse conditions such as wind gusts and ocean currents. When governed by these algorithms, the multi-agent system can complete its mission even if an agent is downed, damaged or forced off course.

Additionally, Lin’s algorithms give an agent maximum maneuverability during climbs and dives in the air and when traversing uneven terrain. Because military operations are often conducted in remote locations with limited opportunities for maintenance and supplies, Lin also needed to boost agents’ ability to conserve on-board resources. Lin proposed an event-triggered distributed control algorithm that allows the agent to operate in a reserved, steady state until pinged by a significant change in operational status.  

Lin’s research demonstrates the value of mathematical theory of self-learning systems. Advances in self-learning systems continue to improve machine learning and artificial intelligence for national defense.

“Dr. Lin’s research addresses some of the fundamental issues in the coordinated control of multi-agent systems encountered in the real world,” said Dr. Derya Cansever, program manager, multi-agent control, Army Research Office, an element of U.S. Army Combat Capabilities Development Command’s Army Research Laboratory. “Coordinated control of multi-agent systems will be instrumental in establishing autonomous systems for the Army’s use, and Dr. Lin’s research provides a step in the right direction towards their realization.”