The experiment explores the artist’s synesthesia, a neurological condition that causes one sense to trigger another. In Kandinsky’s case, he’d see colors and shapes when he heard music, and hear music when he painted. The abstract art pioneer tried to evoke the sensation through his artworks and writings: Google’s collaboration with Paris’ Centre Pompidou imagines the sounds he heard in 1925 as he painted “Yellow, Red, Blue,” a striking display of geometric art. [Read: How Polestar is using blockchain to increase transparency] The Google Arts & Culture team describes the lithograph as a small instrumental work: The researchers first trained Google’s Transformer neural network on music from Kandinsky’s time. They then used the system to generate new scores inspired by the sounds. Sound artist Antoine Bertine, who worked on the project, said the initial idea was to use machine learning as an archeological tool: The experiment resulted in Play a Kandinsky, an interactive tool that lets you explore Kandinsky’s synesthesia and generate your own musical compositions based on his work. You can hear the sounds he associated with different colors and shapes, or listen to the entire masterpiece as the artist may have heard it. But the best part lets you become the composer of the painting, by clicking on colored shapes to produce the music that you see. In my case, the string ensemble of the red triangle caused an acute sense of restlessness, but I attribute much of that to this interminable lockdown. The feeling subsided slowly when I tapped on the peaceful violins behind the multi-colored plane. It doesn’t replicate the connections between sights and sounds that a synaesthetic feels, but it’s an intriguing exploration of the connections between music and painting.
