“Blind” Raspberry Pi camera takes pictures using sounds
A Mexican artist and computer researcher has created a Raspberry Pi-powered “blind” camera that uses artificial intelligence (AI) to convert sounds into images.
Created by Diego Trujillo Pisanty, the device features a horn that is aimed at a sound source in much the same way you would point a camera in the direction of the scene or object you wish to capture.
The heavy lifting is done by a custom-made AI neural network (ANN) that Pisanty trained using his own video recordings around Mexico City.
The ANN can make links between a sound and an image, like how the word “dog” can refer to an image of a furry animal and a barking noise, Pisanty explained.
“Each video frame was associated with its previous second of sound,” said Pisanty.
“The network was then trained to encode the sound into a vector, decode it back to the matching image, and try and convince another network that the resulting image is a photograph,” said Pisanty.
“[The] Blind Camera thus combines elements of Autoencoders and Generative Adversarial Networks,” Pisanty stated.
Pisanty records the sound of a subway train in Mexico City
One of the frames from the video generated by the Blind Camera’s ANN
Pisanty said that developing the AI with bespoke training data rather than using an existing model provided flexibility and control to trigger certain image aesthetics based on his artistic intent.
The resulting ‘photo’ was indexical of the surrounding soundscape, not the scene in front of the camera.
“The code is as much an artwork as the object and its resulting images,” he stated.
“The process of producing Blind Camera’s code prompted reflection about how AI written by artists would work and how it could differ from currently available models.”
Pisanty said because of the ANN’s limited source data, the results were intentionally biased to interpret the world from a local [Mexico City] perspective.
“The Blind Camera will therefore produce images matching its Mexican identity,” Pisanty stated.
He explained because the ANN was only trained in the city, using it in areas like a park filled with trees and grass would still produce a city-like image.
Now read: Garmin unveils new premium smartwatches — with up to 139 days battery life
Loading ...