Developer appearance We shared the demo of a new human-like robot that can hold a conversation in real time. The fruits of their collaboration with OpenIA have surprised the world.
We're now in full conversation with Figure 01, thanks to our partnership with OpenAI.
Our robot can:
– Describe her visual experience
– Planning future actions
– Meditate on her memory
– Explain its reasons verbally
Deep technical diving 🧵:pic.twitter.com/6QRzfkbxZY– Corey Lynch (@coreylynch) March 13, 2024
Through social media, a video clip was published showing Figure 01 interacting smoothly with a person. You can have full conversations thanks to the integration with ChatGPT.
At the beginning of the footage, a human voice can be heard asking the robot: “First figure, what do you see now?” The prototype responded immediately: “I see a red apple on a plate in the middle of the table, and a drying rack with cups.”
The company shared that this collaboration allows the model to have “fast, low-level, and dexterous robotic actions.” “Our robot can describe its visual experience, plan future actions, reflect on its memory, and verbally explain its reasons,” the model’s creator, Corey Lynch, wrote on his X account.
How it works?
During the video, the device is tested on tasks such as recognizing an apple, plates and cups. Also, collect trash and put it in a bin. He did them all.
According to the creator, its functionality is developed by feeding images from the built-in cameras that record text and speech captured by the microphone.
“The model processes the entire conversation history, including past images, to generate linguistic responses, which are fed back to the human via text-to-speech,” Lynch said.
The first form is designed to describe your environment and apply “common sense” when making decisions. The researchers and developers who led the project explained that all behaviors are driven by transformative policies.
“These networks capture integrated images at a frequency of 10 Hz and generate events with 24 degrees of freedom,” the designer said. This can lead to poor wrist position and angulation at the finger joints.
Even for their collaborators, this means an important change, because years ago they could not imagine having a full conversation with a robot while performing a task.
“Our goal is to train a model for the world to operate humanoid robots on a billion-unit scale,” said Brett Adcock, co-founder and CEO of Figure.
:format(jpeg):focal(347x204:357x194)/cloudfront-us-east-1.images.arcpublishing.com/gfrmedia/EQMJZSFQN5FHRGWTLJDOMHBSFI.png&description=Video%3A%20This%20artificial%20intelligence%20robot%20that%20amazed%20the%20world){kind=link}