Researchers at the University of Washington have created an AI called Audeo that can reproduce realistic music simply by analyzing silent piano videos.
Now even artificial intelligence is able to digitally reproduce any sound of a piano in a realistic way, simply by analyzing a silent video.
A team of researchers at the University of Washington has created the artificial intelligence called Audeo that incorporates a kind of intelligent software previously trained with 172,000 frames of video of pianist Paul Barton playing music by classical composers such as Mozart or Bach.
Audeo AI is capable of playing piano virtually
Simply by analyzing a silent video, Audeo AI is able to carefully learn which keys are pressed and what are their order. This way it determines the individual notes and their arrangement. It is also able to sense how hard each of the keys is played and how long they are held down, this allows it to calculate the intensity of each note and how long it lingers under the sound of subsequently played notes. As if that were not enough, the AI is also able to understand the distinctive acoustic characteristics of pianos.
This data is converted into a format that can be understood by a digital synthesizer, and when that synthesizer plays the music file it is very close to the original piano sound.
In one of its amazing tests, the Audeo AI was able to learn a song through silent videos of composer Barton while he was playing distinctive pieces of music.
Then music recognition applications such as SoundHound analyzed such productions performed by the artificial intelligence, being able to recognize the piece of music with an accuracy of around 86%. For comparison, when this application analyzed the original piano audio its recognition accuracy rose to 93%.
