A new chip developed by MIT researchers could help small, low-power UAVs avoid obstacles as they go around tight corners inside industrial HVAC systems to check for gas leaks.
The chip allows small autonomous robots and other battery-limited devices to create detailed 3D maps of their environment in real time using only the power equivalent of a single LED. A robot can use such a map to plan a collision-free path to reach its goal.
In general, generating such complete maps requires power-hungry systems and a lot of memory to create and store 3D representations of obstacles in the robot’s environment.
MIT researchers took a different approach by combining a high-performance mapping algorithm with special hardware designed to speed up its operation, which reduces memory and power consumption.
This system-on-a-chip consumes only about 6 milliwatts of power, half the power required by other systems.
This low-power operation may also make the chip well-suited for lightweight augmented reality headsets that can be worn for extended periods of time, for applications such as medical teaching simulations or detailed repair and assembly work.
“This paper shows an important example of how you can use the collaborative design of algorithm and hardware to push energy efficiency. Although there has been a lot of work looking at compact 3D maps, what stands out about this work is that it also ensures that the process of producing those maps is as efficient as possible. Our chip allows you to store very large maps in a very small space,” said Svien energy, Vizeeneener professor in the Department of Electrical Engineering and Computer Science (EECS), member of the Research Laboratory of Electronics (RLE), and senior author of the paper on the chip.
He is joined on the paper by lead authors and MIT graduate students Zih-Sing Fu and Peter Zhi Xuan Li and Sertac Karaman, professor of astronomy and astronomy and director of LIDS. The work was recently presented at the IEEE Largest Integrated Circuits Conference.
A very compact map
For a robot, generating a 3D map that includes obstacles in its environment usually requires a lot of energy because it has to store the images taken by its camera, and process all the 3D pixels in each image multiple times.
Instead of representing the environment using 3D pixels, which are cubes called voxels, the MIT researchers used a method that maps obstacles in space using ellipsoid blobs called Gaussians.
The size, shape, and thickness of these ellipsoids can be adjusted, so they fit better with curved shapes than one would with solid, cube-shaped voxels.
Importantly, the map captures obstacles and free space around the robot, and together these allow the robot to plan a safe, collision-free path. Mapping constraints and free space with voxels often consumes a lot of memory, making traditional methods power hungry. Because Gaussians cannot fit the geometry well, one long ellipsoid can represent an area that can occupy many voxels, so the busy and free areas are captured more closely.
For their new Gleanmer program, the researchers used an algorithm their lab developed called GMMap that efficiently generates a 3D map of the robot’s environment using Gaussians to represent obstacles.
With traditional methods, the robot would need to load and process each depth image several times to adjust the size and shape of the ellipsoids. The system would normally build Gaussians by comparing all the pixels in the image to each other. But the amount of memory and power required to do this remains too high for most edge devices.
To solve this problem, MIT researchers developed a method that can generate highly accurate Gaussians in depth images with only one pass, after which they can discard the images, so the chip does not want to store the entire image at once.
Instead of comparing each pixel with other pixels in the 3D image, their algorithm assumes that adjacent pixels are the same Gaussian, so it only needs to compare each pixel with its neighbors.
“At any point in time, we only need to store a few pixels in memory, which greatly reduces the memory required for our algorithm,” Li said.
Using co-design
But as the robot moves through space, it often sees the same object from different perspectives. If it generates Gaussians, some will overlap because they represent the same thing. This can make the 3D map too large to be stored on the peripheral device.
Combining overlapping Gaussians makes the map more coherent, but doing so often requires an algorithm to process many raw pixels stored in memory. The researchers developed a new way to perform this clustering process directly on overlapping Gaussians, without needing to revisit the original pixels. Since Gaussians are more compact than pixels, this greatly reduces memory and power requirements.
The same principle applies to its algorithm – most calculations work directly on the concatenated Gaussians instead of the original pixels, which allows for energy efficiency.
The researchers used this principle to design a chip that stores active Gaussians within the chip’s small, fast memory next to the computing units. This is only possible because the Gaussian map is very dense.
The Gaussians the robot needs to work on next are waiting in on-chip memory units, so they don’t need to be fetched from a remote, power-hungry, off-chip location.
“By having a dedicated memory that only stores the things you’ve seen in the last few frames, you can access data very efficiently,” Fu explained.
They tested the system-on-a-chip by reconstructing a range of different, pre-existing 3D surfaces. The chip can also reconstruct obstacles and free space directly from live data transmitted from the iPhone’s camera.
Gleanmer produced detailed 3D maps in real time while using about 6 milliwatts of power. It requires only about 2.5 percent of the power that an existing mapping chip would require.
By reusing Gaussians along the path as it plans, the chip allows the robot to hit a safe route using only about 20 percent of the energy it would need.
“We reduce the memory usage by making sure that the algorithm is working. Then we speed up the work done by that efficient algorithm, so in the end, our chip works as efficiently as possible,” said Li.
The researchers plan to further improve energy efficiency by moving the processing units on the chip closer to the sensors that collect environmental data. They can also explore additional applications, such as the use of Gaussians to represent schematics. This can help AI systems think about complex plans more effectively.
“Real-time 3D mapping has been a missing piece for small, autonomous systems. A drone that inspects a pipe or AR glasses that navigate a room both need to understand the environment – quickly, continuously, and almost for free. Gleanmer makes that possible for the first time on a chip that you can hold between your fingers,” said Karaman.
This work is supported, in part, by the MIT-MathWorks Fellowship, Amazon, the US National Science Foundation, and Intel.
