Robots with spatial and temporal memory

Robots need to know where they stand.

The researchers used OpenCloud to give robots something extremely important for any form of intelligence, spatial and temporal memory, in practice this means that the robot begins to build a model of the world around it, instead of just recording images or isolated sensors, the system combines data from lidar, RGB cameras, stereo cameras and motion sensors, with that, the robot creates a detailed three-dimensional map of the environment.

The big difference is that this map not only records where things are, but also when events occurred. Researchers call this capacity people's spatial memory, that is, the robot begins to understand relationships between objects, places and moments in time. This allows you to answer questions like, where did I see a certain object? Who walked into this room yesterday when something was last moved? To build this complex memory, the system uses a technology called Special RED, which organizes space into small three-dimensional blocks known as Voxels.

Each of these blocks contains information about geometry, objects, time and semantic context. The result is something similar to a three-dimensional database of the physical world and perhaps the most surprising part is that all this was made available as open source on Ginhub, this means that the system can be adapted to different platforms, from humanoid robots like Unitree G1 to drones or quadruped robots. In theory, any machine equipped with solidar visual sensors could use this develop advanced spatial perception.

Many researchers believe that this approach represents an important step towards so-called embodied intelligence, because for robots to coexist with humans on a daily basis, it is not enough to see the world, they need to understand what happened in it and mainly remember it later, but they also need to control their own body with extreme precision.