If you think snapping high-definition pictures on your smartphone is the pinnacle of digital photography, think again. At the Columbia Vision Laboratory, engineers are designing cameras with 360-degree fields of view, cameras that power themselves, and cameras that capture more detail than is visible to the human eye.
What’s the point of a photograph filled with invisible detail?
“You may not be able to see all the visual data it contains, but a computer can,” says Shree K. Nayar, the computer-science professor who directs the laboratory, at Columbia’s school of engineering. “This technology could be useful in creating new types of medical-imaging devices, space telescopes, or microscopes.”
Nayar and his colleagues at the Columbia Vision Laboratory are at the forefront of a nascent field called computational photography. This is different from what is commonly known as digital photography; the digital cameras that are popular today collect light through a single hole positioned behind a curved lens, just as traditional cameras do, but computational cameras often use complex arrays of mirrors or movable sensors to collect enormous amounts of data before generating an image.
“In computational photography, there is no ‘image’ to speak of until the very end of the process, when your software has assessed the jumble of digital data you’ve gathered,” says Nayar, who is widely regarded as a pioneer of the field. “This enables you to achieve some pretty remarkable things in terms of image quality and energy efficiency.”
Some of Nayar’s inventions have already been incorporated into commercial cameras. He helped Sony develop a new type of high dynamic range (HDR) sensor that can detect and correct for imbalances of light on a pixel-by-pixel level; this technology is now used in many smartphone cameras. But Nayar’s projects are typically more experimental. The camera that produces extremely high-resolution images, for instance, began as a test of Nayar’s theory that spherical lenses could overcome traditional limitations in photographic resolution. A prototype that he and former graduate student Oliver Cossairt ’11SEAS built uses a glass sphere and a concave sensor behind it to gather light from different angles.
“Nobody asked us to build this thing,” says Nayar. “But there’s an insatiable appetite for high-resolution imaging in academia and in many industries, and this is one way to achieve it.”
A solar-powered video camera that Nayar’s lab built also grew out of a technical theory he had dreamed up.
A few years ago, it occurred to him that the sensors that cameras use to record light are structurally similar to those in solar panels that convert light into energy. He wondered: could his team design one sensor that did both? Last year, Nayar, working together with research engineer Daniel Sims ’14SEAS and several other collaborators, succeeded in creating a sensor that can toggle between the two functions. The result is a camera that can operate indefinitely while shooting at least one image per second, outdoors or in a well-lit room. Nayar thinks it would be useful in situations where it is difficult to replace a camera’s battery.
“I got a call from people at the Wildlife Conservation Society saying they’d love to have a bunch of these cameras installed in remote locations for observing endangered species,” says Nayar. “How cool would that be?”
Nayar doesn’t expect that these inventions will be adopted widely in their current forms. The prototypes, which he acknowledges are bulky and look rather inelegant, are intended merely to demonstrate their technological principles. Nayar hopes that camera companies will eventually work with him to create sleeker, more commercially viable versions.
“We’re inventors, not product designers,” he says. “But we believe that our technologies could be miniaturized fairly easily.”
Which is not to say that the team’s prototypes don’t possess a certain retro charm.
“Doesn’t this one resemble an old daguerreotype camera from the nineteenth century, with its big light box?” says Nayar of the self-powered camera. “I love that. I love how these objects recall the dawn of the age of photography while at the same time pointing the way forward.”
