In this video, DARPA program manager Dr. Bruce Draper describes the technology he thinks could usher in the next “do-it-yourself” revolution.
The Perceptually-enabled Task Guidance (PTG) program aims to develop virtual “task guidance” assistants that can work with different sensor platforms to help military personnel perform complex physical tasks and expand their skillsets. Unlike today’s AI assistants, PTG technology would be able to see what the user sees and hears what they hear by integrating with a microphone, a head-mounted camera, and displays like augmented reality (AR) headsets, to deliver accurate instructions.
PTG performers recently demonstrated early successes of their prototypes by using the task of cooking recipes as a proxy for unfamiliar, more complex tasks, such as battlefield medical procedures, military equipment sustainment and co-piloting aircraft.
“Today the commercial sector is pursuing new, useful ways to present data to the user but it doesn’t go far enough,” said Draper. “The gamechanger with PTG would be having perceptually-driven AI interfaces that can make sense of the real world, react to whatever the user is doing and provide advice. I'm really impressed at how quickly performing teams are making progress toward the goals.”
For more information about PTG, you can also listen to Voices from DARPA podcast episode 41: The AI Tutor. In the episode, Draper expands on the motivations behind PTG in addition to other programs in his portfolio.
PTG Performers include Kitware (Columbia University; University of California, Berkeley; University of Texas at Austin); PARC (University of California, Santa Barbara; University of Rostock); Northeastern University (University of California, Santa Barbara; Stony Brook University); New York University; University of Texas at Dallas (University of California, Irvine; University of Florida); Stevens Institute of Technology (Purdue University; University of Michigan; University of Rochester); University of Florida (Northeastern University; Topos Institute; Texas A&M University; University of Arizona); Raytheon Technologies (Valkyries Austere Medical Solutions); Northrop Grumman (University of Central Florida); Red Shred (Third Insight); MIT Lincoln Laboratory