Science and engineering in their never-ending quest continue to wow and intrigue with the invention and improvements of ways of doing things. The other day, I was browsing through the pages of the internet, and it was terrific to see the advances recorded over time from the robotic labs of the Boston Dynamics. The human-looking robot or humanoid robot, Atlas standing 1.5m and weighing 75kg can now easily jump on top of obstacles in a manner capable of impressing a parkour expert. While the robot's coordination may be a source of wonder, there is a team working on something that may equally rival that in the field of computer vision.
A computer graphics showing penumbr shadows. Image credits: By Wikipedia Commons from CVF Link]
A team of scientists from the prestigious Massachusetts Institute of Technology are now working on ways to see what lies ahead in the corner ushering in a new field of science less than a decade old; non-line-of-sight imaging. Which, in other words, is the science of deciphering what we may not naturally see without the aid of imaging technology such as that inherent in our camera.
The idea to begin such a daring quest started when one of the research team leaders, Antonio Torralba saw shadows on the wall of the hotel room when he was on vacation in Spain. A shadow which at first was a surprise as it apparently has no origin. However, on close observation, it was a shadow from one of the windows which inadvertently produced a pinhole camera effect of the activities outside the hotel.
A tripod with camera standing against a wall with images A and B not in view. Image credits: By OpenAccess Computer Vision Foundation from CVF Link]
The "camera" which was the hotel window enabled the vacationer to see the usually hidden objects. In much the same way we have the edges of walls and floors often act as a camera via its depiction of a partially shaded area as a result of shadow which an otherwise invisible object creates at the hidden view of the object.
For instance, imagine a camera directly placed against a wall. The red and blue objects, A and B respectively are obviously out of sight for the camera. However, if you look closely, you can see the subtle varying light disturbance or penumbra left on the floor by object A and B. If you follow the direction of the arrow, you will notice changes in the light gradient as you increase the angle of view.
The MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers made up of Katherine L. Bouman, Vickie Ye, Adam B. Yedidia, Fredo Durand, Gregory W. Wornell, Antonio Torralba, and William T. Freeman are using computer algorithm to track these subtle changes that occur in the penumbra from objects hidden from view in real time making use of ordinary camera such as that found in our smartphone devices.
The information gleaned from the penumbra video which the team aptly nicknamed CornerCamera, the team's algorithm can create a one-dimentional (1-D) images obtained from the highly magnified subtle, floor's penumbra changes captured by the RGB cameras.
The information that this dynamic one-dimension image contains is enough to extract both the colour of the hidden object and identify the location and movement. This method is more compact, robust, and is far superior in that it can work in broad daylight, dust, and in the rain. The ability to work very well in these three conditions makes it superior to the more expensive laser systems which works by firing lasers, and calculating the time it takes for it to return and digitally "builds" a picture of the hidden object.
Automous vehicle. Image credits: By Wikipedia Commons from CVF Link]
The application for this technology is vast. It will be particularly useful in navigation in self-driven cars, search and rescue operation, remotely monitor occupants in a location; an issue I'm sure the military will love in tactical [rocucres. Also, apart from the self-driven vehicle, it will help reduce collision and blind spots when driving around corners. You can see the applications are vast and important. At the moment we can only wish the team well as they explore means
References
- Atlas:The World's Most Dynamic Humanoid
- Seeing around corners without looking
- CVF: Turning Corners into Cameras: Principles and Methods
- Accidental pinhole and pinspeck cameras: revealing the scene outside the picture
- IEEE:Smartphone Cameras Peek Around Corners by Analyzing Patterns of Light
- CSAIL Computer Code
- CSAIL MIT
- It's not science fiction, it's penumbra!
- Video