For the first time, machine learning imitates the brain to process sounds, and can distinguish lyrics and song classifications

Do you have such a question about software like Spotify: "Spotify, what do you think when you put music?" In fact, such software may think like you.

A new study from the Massachusetts Institute of Technology shows that scientists have built a machine learning system that can process sounds like humans, identify lyrics, or classify music by genre.

It is the first man-made system that mimics the brain to interpret sound, and is comparable in accuracy to humans. The study, published in the journal Neuron, provides an attractive new way to study the human brain.

Machine learning systems are ubiquitous, such as software with music recommendation features. However, software engineers often don't know how these systems "think" or whether the internal workings of the software are similar to the human brain.

The researchers' models are based on the well-known deep neural network – inspired by human neurons or brain cells. It can process information through tiering, and the deepest layer performs the most complex work. Scientists can train these models to "learn" human behavior, such as analyzing sound.

The researchers set two goals for the model. First, play a two-second speech clip to test the words that appear in the model recognition speech. Second, play two seconds of music to test how the model classifies the music. In addition, the researchers also set noise at each test to increase the difficulty of model recognition.

The experiment required thousands of cases to train the model, but in the end, the performance of the model was as good as the human brain. The model recognizes dozens of music types, such as it can recognize dubstep from ska or gothic rock. However, when playing a segment of a city's voice, like the human brain, it makes mistakes.

But researchers are still not sure if the model being built can process signals like the brain – or it has its own way to solve the same problem. So they need to look at the human brain.

The first author of the article, Alex Kell, from the Massachusetts Institute of Technology, studied data from the fMRI scanner and observed which areas of the brain were most active when people heard a series of natural sounds.

He then enters these sounds into the model. He found that when the model deals with some relatively basic information (such as the frequency of sounds or patterns), it corresponds to a certain area of the brain. When taking on more complex tasks (such as recognizing the meaning of music), the model corresponds to another area of the human brain.

This shows that the model can process information in the same way as the human brain from the simplest to the most complex hierarchy.

Andrew Pfalz, a Ph.D. associate with experimental music and digital media at Louisiana State University (which uses neural networks to apply music to music), says the ability to connect the inner workings of deep neural networks to the brain is incredible.

Machine learning systems are ubiquitous, such as software with music recommendation features. But software engineers often don't know how these systems "think" or how the internal workings of software are similar to human brains.

“This is a black box,” Pfalz said. “Interestingly, we trained these models and saw that they were correctly classified and predicted, but we didn’t know what was going on inside.”

But after exploration, researchers at the Massachusetts Institute of Technology can clearly understand which layers of the system are in what state, and how the model handles the same sound as the human brain.

The machine learning system (hence the name "Neural Network"), originally inspired by brain structure, is now helping scientists better study the brain. Pfalz thinks this idea is very interesting.

However, Ching-Hua Chuan, a computer scientist at the University of North Florida who mainly studies the use of machine learning systems to create music, emphasizes the breadth of this statement. "[Neural networks] have never intended to simulate how our brains work," she added, adding that the difficulty of spying on the internals of "black box" suggests that more research is needed to prove that the model does simulate the brain.

The MIT team believes they are nearing this goal. Josh McDermott, a senior author of the study, at the Massachusetts Institute of Technology, said that if they are right, the model can help scientists understand and simulate how the brain processes sound and other sensory signals. Moreover, because testing on a model is faster, safer, and less costly than performing experiments on a real brain, this may accelerate some of the advances in neuroscience.

Kell said that computing power and neural network technology are not always able to simulate part of the human brain, but the past five years have created a new era. "In the field of machine learning, many insurmountable problems in history have actually been solved."