Why is this important: Google Research is working on a new AI image noise reduction technology that could radically change low-light photography. The tool is able to reconstruct a dark scene with powerful denoising and minimal artifacts, comfortably outperforming existing denoising tools.
Computational photography has come a long way and is now widespread in smartphones and post-processing software. Noise reduction is arguably one of the most valuable tools, as even the best camera sensors are not free from image noise, especially when used in a darker environment. Google Research has unveiled an exciting new technology that uses artificial intelligence to remove image noise from darker scenes, allowing photographers to “see in the dark”.
They call this new tool RawNeRF and it is part of their open source project known as MultiNeRF. RawNeRF is able to help photographers capture darker scenes specifically. It uses AI that has unprecedented denoising power, and what’s really impressive is that denoising seems to happen with minimal quality loss and far fewer artifacts than comparable tools.
NeRF (Neural Radiance Fields) is a view synthesizer capable of digitizing a collection of images and reconstructing an accurate 3D rendering. Google researcher Ben Mildenhall explains that RawNeRF “combines images taken from many different viewpoints to jointly denoise and reconstruct a scene.” So it’s not just a denoiser, but it can be used to vary the camera position and view the scene from different angles. Scenes are reconstructed in a linear HDR color space, which means it can handle details like varying exposure, using tone mapping, and changing focus.
As seen in the video above, Mildenhall uses a smartphone photo of a candlelit table to demonstrate the power of RawNeRF. It applies minimal post-processing and brightening, and while the resulting image is more detailed, it has a significant amount of sensor noise. It shows that passing the image through a state-of-the-art deep denoiser leaves unsightly artifacts, but with RawNeRF the results are truly astounding, especially with respect to image quality and lack of artifacts. The reason it works so well is because the AI is trained on raw image data rather than post-processed JPEGs.
It’s exciting to think that we could see this technology incorporated into our cameras and smartphones very soon, and it’s likely to be a game-changer for professional photographers and enthusiasts alike. There is no existing noise reduction tool that even comes close to these results.