Background: It is increasingly recognized that the complex functions of human cognition are not accurately represented by arbitrarily-defined anatomical brain regions. Given the considerable functional specialization within such regions, more fine-grained studies of brain structure could capture such localized associations. However, such analyses/studies in a large community-dwelling population are lacking. Objective: To perform a fine-mapping of cognitive ability to cortical and subcortical grey matter on magnetic resonance imaging (MRI). Methods: In 3,813 stroke-free and non-demented persons from the Rotterdam Study (mean age 69.1 (±8.8) years; 55.8% women) with cognitive assessments and brain MRI, we performed voxel-based morphometry and subcortical shape analysis on global cognition and separate tests that tapped into memory, information processing speed, fine motor speed, and executive function domains. Results: We found that the different cognitive tests significantly associated with grey matter density in differential but also overlapping brain regions, primarily in the left hemisphere. Clusters of significantly associated voxels with global cognition were located within multiple anatomic regions: left amygdala, hippocampus, parietal lobule, superior temporal gyrus, insula and posterior temporal lobe. Subcortical shape analysis revealed associations primarily within the head and tail of the caudate nucleus, putamen, ventral part of the thalamus, and nucleus accumbens, more equally distributed among the left and right hemisphere. Within the caudate nucleus both positive (head) as well as negative (tail) associations were observed with global cognition. Conclusions: In a large population-based sample, we mapped cognitive performance to cortical and subcortical grey matter density using a hypothesis-free approach with high-dimensional neuroimaging. Leveraging the power of our large sample size, we confirmed well-known associations as well as identified novel brain regions related to cognition.