Preprint · CC-BY
via bioRxiv
PAM: Predictive attention mechanism for neural decoding of visual perception
Dado, T., Le, L., van Gerven, M., Güclütürk, Y., Güclü, U.
biorxiv · 2025
Abstract
In neural decoding, reconstruction seeks to create a literal image from information in brain activity, typically achieved by mapping neural responses to a latent representation of a generative model. A key challenge in this process is understanding how information is processed across visual areas to effectively integrate their neural signals. This requires an attention mechanism that selectively focuses on neural inputs based on their relevance to the task of reconstruction -- something conventional attention models, which capture only input-input relationships, cannot achieve. To address this, we introduce predictive attention mechanisms (PAMs), a novel approach that learns task-driven "output queries" during training to focus on the neural responses most relevant for predicting the latents underlying perceived images, effectively allocating attention across brain areas. We validate PAM with two datasets: (i) B2G, which contains GAN-synthesized images, their original latents and multiunit activity data; (ii) Shen-19, which includes real photographs, their inverted latents and functional magnetic resonance imaging data. Beyond achieving state-of-theart reconstructions, PAM offers a key interpretative advantage through the availability of (i) attention weights, revealing how the models focus was distributed across visual areas for the task of latent prediction, and (ii) values, capturing the stimulus information decoded from each area.
◌ CITATION ONLY
Full text is not openly licensed for redistribution here. Read it at the source:
Provenance
- Source
- bioRxiv
- DOI
- 10.1101/2024.06.04.596589
- Canonical
- link ↗
- Fetched
- 2026-10-09 MST
Cite this
APA
T., D., L., L., M., V.G., Y., G., & U., G. (2025). PAM: Predictive attention mechanism for neural decoding of visual perception. <em>biorxiv</em>. https://doi.org/10.1101/2024.06.04.596589
Vancouver
T. D, L. L, M. VG, Y. G, U. G. PAM: Predictive attention mechanism for neural decoding of visual perception. biorxiv. 2025. doi:10.1101/2024.06.04.596589.
BibTeX
@unpublished{dado2025PAMPre,
title = {PAM: Predictive attention mechanism for neural decoding of visual perception},
author = {Dado, T. and Le, L. and van Gerven, M. and Güclütürk, Y. and Güclü, U.},
journal = {biorxiv},
year = {2025},
doi = {10.1101/2024.06.04.596589},
}
Research neighborhood
References, citing works, and semantically nearest findings. Click a node to open it.
Related findings
biorxiv 2024
Preprint · CC-BY
Individual variability in neural representations of mind-wandering
Robert Brummer 2023
Open access · US-GOV
Comparing the Efficacy of Micro-encapsulated Lactocaseibacillus Rhamnosus Vs Free Probiotic in Powder to Affect Brain Connectivity
biorxiv 2024
Preprint · CC-BY
Spatial and temporal pattern of structure-function coupling of human brain connectome with development
biorxiv 2025
Preprint · CC-BY
One Hundred Neural Networks and Brains Watching Videos: Lessons from Alignment
biorxiv 2024
Preprint · CC-BY
Alpha phase-coding supports feature binding during working memory maintenance
Virginia Polytechnic Institute and State University 2023
Open access · US-GOV