Toyota Research Institute Uses Generative AI to Teach Robots How to Make Breakfast

Toyota Research Institute (TRI) has developed a system using generative AI to teach robots how to perform individual tasks needed to make breakfast. Instead of relying on traditional coding methods, researchers equipped the robots with a sense of touch, plugged them into an AI model, and demonstrated the tasks to them, similar to how a human would learn. The sense of touch plays a crucial role in enabling the robots to “feel” what they are doing and gather more information.

By giving the robots a tactile feedback mechanism, they can carry out difficult tasks more easily than relying solely on visual information. The team at TRI refers to this approach as creating “Large Behavior Models” (LBMs) for robots. Similar to how language models are trained by observing patterns in human writing, LBMs would learn through observation and be able to generalize and perform new skills they have not been explicitly taught.

Using this technique, the researchers have already trained the robots to perform over 60 challenging skills, such as pouring liquids, using tools, and manipulating deformable objects. They aim to increase this number to 1,000 by the end of 2024.

Other companies, such as Google and Tesla, are also conducting similar research in teaching robots to infer tasks based on their experiences. The goal is for AI-trained robots to eventually carry out tasks with minimal instruction, similar to how humans can understand general directions.

However, the process of training robots in this way is still considered slow and labor-intensive. Providing enough training data and ensuring accuracy can be challenging. Nonetheless, with advancements in generative AI, researchers are making significant progress in teaching robots complex skills without extensive coding and debugging.

– “Toyota Research Institute using generative AI to teach its robots how to make breakfast” – The Verge
– “Researchers Teach a Robot How to Pour From a Bag” – The New York Times