SUBSCRIBE TO NEWSLETTER

Mind reading: Innovators capture Pink Floyd song by decoding brain activity

From PLOS 16/08/23

Brain activity analysis of patient while listening to the Pink Floyd song. CREDIT: PLOS

Researchers led by Ludovic Bellier at the University of California, Berkeley, US, demonstrate that recognizable versions of classic Pink Floyd rock music can be reconstructed from brain activity that was recorded while patients listened to the song.

Published August 15th in the open access journal PLOS Biology, the study used nonlinear modeling to decode brain activity and reconstruct the song, “Another Brick in the Wall, Part 1”. Encoding models revealed a new cortical subregion in the temporal lobe that underlies rhythm perception, which could be exploited by future brain-machine-interfaces.

Past research has shown that computer modeling can be used to decode and reconstruct speech, but a predictive model for music that includes elements such as pitch, melody, harmony, and rhythm, as well as different regions of the brain’s sound-processing network, was lacking.

The team at UC Berkeley succeeded in making such a model by applying nonlinear decoding to brain activity recorded from 2,668 electrodes, which were placed directly on the brains of 29 patients who then listened to classic rock.

Anatomical location of song-responsive electrodes. (A) Electrode coverage across 29 patients on MNI template (N = 2,379). (B) Locations of significant encoding electrodes for song acoustics (Nsig = 347) using STRF prediction accuracy. Marker color and size denote anatomical label and prediction accuracy. (C) No. of significant electrodes per anatomical region; darker hue = right hemisphere. (D) Avg. STRF prediction accuracy per anatomical region. Other regions pooled. Error bars: SEM. Data: https://doi.org/10.5281/zenodo.7876019. CREDIT: PLOS.

Brain activity at 347 of the electrodes was specifically related to the music, mostly located in three regions of the brain: the Superior Temporal Gyrus (STG), the Sensory-Motor Cortex (SMC), and the Inferior Frontal Gyrus (IFG).

Analysis of song elements revealed a unique region in the STG that represents rhythm, in this case the guitar rhythm in the rock music.

To find out which regions and which song elements were most important, the team ran the reconstruction analysis after removing the different data and then compared the reconstructions with the real song. Anatomically, they found that reconstructions were most affected when electrodes from the right STG were removed.

By decoding electrical activity in the brain, UC Berkeley neuroscientists were able to reconstruct the song a person was listening to. In this case, it was a phrase (All in all it was just a brick in the wall) from the Pink Floyd song “Another Brick in the Wall, Part 1” from the 1979 album The Wall. CREDIT
Ludovic Bellier and Robert Knight, UC Berkeley
Original song waveform transformed into a magnitude-only auditory spectrogram, then transformed back into a waveform. CREDIT: Bellier et al., 2023, PLOS Biology

Functionally, removing the electrodes related to sound onset or rhythm also caused a degradation of the reconstruction accuracy, indicating their importance in music perception.

These findings could have implications for brain-machine-interfaces, such as prosthetic devices that help improve the perception of prosody, the rhythm and melody of speech.

Bellier adds, “We reconstructed the classic Pink Floyd song Another Brick in the Wall from direct human cortical recordings, providing insights into the neural bases of music perception and into future brain decoding applications.”

More info

Paper

You may also be curious about:

Tagged , , , , , , ,

60 responses

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to our weekly newsletter

Recieve the latest innovation, emerging tech, research, science and engineering news from Superinnovators.