Objective: To investigate the relationship between amyloid-β (Aβ) deposition and markers of brain structure on cognitive decline in oldest-old individuals with initial normal cognition. Methods: We studied cognitive functioning in four domains at baseline and change over time in fifty-seven cognitively intact individuals from the EMIF-AD 90+ study. Predictors were Aβ status determined by [18F]-flutemetamol PET (normal = Aβ − vs. abnormal = Aβ+), cortical thickness in 34 regions and hippocampal volume. Mediation analyses were performed to test whether effects of Aβ on cognitive decline were mediated by atrophy of specific anatomical brain areas. Results: Subjects had a mean age of 92.7 ± 2.9 years, of whom 19 (33%) were Aβ+. Compared to Aβ−, Aβ+ individuals showed steeper decline on memory (β ± SE = −0.26 ± 0.09), and processing speed (β ± SE = −0.18 ± 0.08) performance over 1.5 years (P < 0.05). Furthermore, medial and lateral temporal lobe atrophy was associated with steeper decline in memory and language across individuals. Mediation analyses revealed that part of the memory decline observed in Aβ+ individuals was mediated through parahippocampal atrophy. Interpretation: These results show that Aβ abnormality even in the oldest old with initially normal cognition is not part of normal aging, but is associated with a decline in cognitive functioning. Other pathologies may also contribute to decline in the oldest old as cortical thickness predicted cognitive decline similarly in individuals with and without Aβ pathology.