New excavator interface lets rookies match veteran operators on day one

Skilled excavator operators make their machines seem like an extension of their own bodies. This intuition usually takes years to develop. Now, a new controller from MIT engineers helps new operators match veterans in just one session.

Excavators are common on job sites for digging, clearing, and leveling. But mastering them is hard. Standard joysticks require operators to translate every wrist movement into machine actions. This "mental model" takes years to build.

A team led by Hermano Krebs at MIT's Department of Mechanical Engineering took a different approach. Instead of training people to think like a machine, they built a machine that understands how humans naturally move.

Digging in, Faster

"This is a more intuitive way to command the machine," Krebs said. "With this new interface, we can eliminate many mental maps an operator would need to build." The team's findings appeared in the Journal of Computing and Civil Engineering.

The device is called the World-Space Interface (WSI). It's a miniature excavator arm and bucket that an operator grips. They move it like their own arm when digging by hand. A six-panel display shows a virtual excavator that instantly mirrors these movements.

"'World-space' refers to everything outside of yourself, or outside the excavator's cab," Krebs explained. "Operators normally build a mental map to manipulate things in the world-space. But now, we can just mime picking up rocks or dirt, and the computer translates that for us."

The researchers believe this won't just be for training. Krebs imagines the controller mounted directly in a real cabin. It would act like an oversized extension of the operator's arm.

"Instead of joysticks, you might have this miniature arm where the operator places their own arm, like an exoskeleton," he said. "This would let them operate the excavator in the cab." If work is in a difficult or unsafe area, an operator could sit off-site and remotely control the excavator.

To test the idea, the team ran training experiments. Both expert and novice volunteers used the WSI and a traditional joystick simulator. Researchers created 15 virtual scenarios. These included construction sites, highways, forest roads, riverbanks, mining areas, and urban and rural settings.

Tasks included scooping sand, digging trenches, clearing debris, and breaking rocks. Volunteers trained one hour a day for seven days. This mirrored a typical weeklong excavator course.

With joysticks, novices always performed worse than experts, though they did improve. With the World-Space Interface, that gap disappeared. Novices matched expert performance from the very first session. "Joysticks are a non-intuitive way to control the machine," said study co-author Joao Buzzatto. "This is the first interface that doesn't require me to command the excavator with joysticks."

The system currently lacks physical feedback. The team is now adding haptics, which simulates the sense of touch, to the mechanical arm. For example, if an operator mimes picking up rocks, the arm would generate a force. This would make it feel like the operator could feel the weight of the rocks.

"Haptics would make this an even more intuitive system," said co-author Solmon Jeong.

Other companies like Caterpillar, Hyundai, and Komatsu are also developing simulators. However, these mostly use traditional joystick controllers that still take time to learn.

This approach could lead to exoskeleton-controlled machines or fully autonomous excavators. The gap between human gesture and machine motion is clearly getting smaller.

Deep Dive & References

A World-Space Interface for Excavator Teleoperation - Journal of Computing and Civil Engineering