For nearly a decade, a team of researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) has sought to discover why certain images persist in people's minds, while many others fade. To do this, they set out to map the spatio-temporal brain dynamics involved in the recognition of a visual image. And now, for the first time, scientists have harnessed the combined strengths of magnetoencephalography (MEG), which captures the timing of brain activity, and functional magnetic resonance imaging (fMRI), which identifies active regions of the brain, to precisely determine when and where brain processes are taking place. a memorable image.
Their open access study, published this month in Biology PLOS, used 78 pairs of images corresponding to the same concept but differing in their memorability scores: one was very memorable and the other easy to forget. These images were presented to 15 subjects, with skateboarding scenes, animals in various environments, everyday objects like cups and chairs, natural landscapes like forests and beaches, urban scenes of streets and buildings and faces displaying different expressions. They discovered that a more distributed network of brain regions than previously thought is actively involved in the encoding and retention processes that underlie memorization.
“People tend to remember some images better than others, even when they are conceptually similar, such as different scenes of a person riding a skateboard,” says Benjamin Lahner, an MIT doctoral student in electrical and computer engineering, affiliated with CSAIL and first author of the study. study. “We identified a brain signature of visual memorization that appears approximately 300 milliseconds after seeing an image, involving areas of the ventral occipital cortex and temporal cortex, which process information such as color perception and object recognition. This signature indicates that highly memorable images elicit stronger and more sustained brain responses, particularly in regions like the early visual cortex, which we previously underestimated in memory processing.
While highly memorable images maintain a higher, more sustained response for about half a second, the response to less memorable images diminishes quickly. This idea, Lahner explained, could redefine our understanding of how memories form and persist. The team believes that this research holds potential for future clinical applications, particularly in the early diagnosis and treatment of memory-related disorders.
The MEG/fMRI fusion method, developed in the laboratory of Aude Oliva, principal investigator at CSAIL, skillfully captures the spatial and temporal dynamics of the brain, overcoming traditional constraints of spatial or temporal specificity. The fusion method benefited from the help of its friend machine learning, to better examine and compare brain activity when looking at various images. They created a “representation matrix,” which looks like a detailed chart, showing how similar neural responses are in various regions of the brain. This chart helped them identify patterns of where and when the brain processes what we see.
Selecting pairs of conceptually similar images with high and low memorability scores was the crucial ingredient to unlocking this memorization information. Lahner explained the process of aggregating behavioral data to assign memorability scores to images, where they curated a diverse set of high and low memorability images with balanced representation across different visual categories.
Despite the progress made, the team notes some limitations. Although this work can identify brain regions showing significant effects on memorization, it cannot elucidate the function of these regions in terms of how they contribute to better memory encoding/retrieval.
“Understanding the neural underpinnings of memorization opens exciting avenues for clinical advances, particularly in the diagnosis and early treatment of memory-related disorders,” says Oliva. “The specific brain signatures we identified for memorization could lead to early biomarkers of Alzheimer's disease and other dementias. This research paves the way for new intervention strategies finely tuned to the individual's neural profile, potentially transforming the therapeutic landscape of memory disorders and significantly improving patient outcomes.
“These results are exciting because they give us insight into what happens in the brain between seeing something and storing it in memory,” says Wilma Bainbridge, an assistant professor of psychology at the University of Chicago, who did not participate in the study. “Researchers detect here a cortical signal that reflects what is important to remember and what may be forgotten early on.”
Lahner and Oliva, who is also director of strategic industry engagement at the MIT Schwarzman College of Computing, director of the MIT-IBM Watson AI Lab and principal investigator of CSAIL, join Western University assistant professor Yalda Mohsenzadeh and York University researcher Caitlin Mullin. On paper. The team acknowledges a Shared Instrument Grant from the National Institutes of Health, and their work was funded by the Vannevar Bush Research Fellowship through an Office of Naval Research grant, a National Science Foundation award, an award from the 'Multidisciplinary academic research initiative through a grant from the Army Research Office. , and the EECS MathWorks Scholarship. Their article is published in Biology PLOS.