The first news from this year’s Automate conference comes via Alphabet X spinout Intrinsic. The company announced at the event in Chicago on Monday that it will integrate a number of NVIDIA offerings into its Flowstate robotics app platform.
This includes Isaac Manipulator, a collection of basic models for creating workflows for robotic arms. The offering launched back in March at GTC, with some of the biggest names in industrial automation already on board. The list includes Yaskawa, Solomon, PickNik Robotics, Ready Robotics, Franka Robotics and Universal Robots.
The collaboration focuses specifically on grasping (grasping and picking up objects) – one of the key modalities for both manufacturing and fulfillment automation. The systems are trained on large data sets with the aim of performing tasks that work across hardware (i.e. hardware agnosticism) and with different objects.
This means picking methods can be transferred to different environments instead of having to train each system for each scenario. Once we as humans figure out how to pick things up, that action can be adapted to different objects in different environments. In most cases, robots can’t do that – at least not at the moment.
Photo credit: Intrinsic
“In the future, developers will be able to leverage pre-built general purpose comprehension capabilities like this to significantly speed up their programming processes,” said Wendy Tan White, founder and CEO of Intrinsic, in a post. “For the broader industry, this development shows how baseline models could have a profound impact by, among other things, making it easier to address today’s large-scale robot programming challenges, creating previously infeasible applications, reducing development costs and increasing flexibility for end users.”
The first flowstate tests took place in Isaac Sim – NVIDIA’s robot simulation platform. The company’s own customer, Trumpf Machine Tools, worked with a prototype of the system.
“This universal grasping ability, trained with 100% synthetic data in Isaac Sim, can be used to build sophisticated solutions that can perform adaptive and versatile object grasping tasks in simulation and real life,” says Tan White about Trumpf’s work with the platform. “Instead of hard-coding specific grippers to grab specific objects in a specific way, efficient code for a specific gripper and object is automatically generated to complete the task using the foundation model.”
Intrinsic is also working with DeepMind, also part of Alphabet, to crack pose estimation and path planning – two other key aspects of automation. For the latter, the system was trained on more than 130,000 objects. The company says the systems are able to determine the orientation of objects in “a few seconds” – an important part of being able to record them.
Another important part of Intrinsic’s work with DeepMind is the ability to operate multiple robots in tandem. “Our teams tested this 100% ML-generated solution to seamlessly orchestrate four separate robots working on a scaled-down simulation of an automotive welding application,” says Tan White. “The motion plans and trajectories for each robot are automatically generated, are collision-free and surprisingly efficient – they perform about 25% better than some traditional methods we have tested.”
The team is also working on systems that use two arms simultaneously – a setup more in line with the emerging world of humanoid robots. We’ll be seeing a lot more of this in the next few years, whether humanoid or not. Going from one to two arms opens up a whole world of additional applications for these systems.