Introduction: We aimed to establish sex differences in vascular brain damage of memory clinic patients with possible vascular cognitive impairment (VCI). Methods: A total of 860 memory clinic patients (aged 67.7 ± 8.5; 46% female) with cognitive complaints and vascular brain damage (ie, possible VCI) from the prospective TRACE-VCI (Utrecht-Amsterdam Clinical Features and Prognosis in Vascular Cognitive Impairment) cohort study with 2-year follow-up were included. Age-adjusted female-to-male differences were calculated with general linear models, for demographic variables, vascular risk factors, clinical diagnosis, cognitive performance, and brain magnetic resonance imaging markers. Results: We found no difference in age nor distribution of clinical diagnoses between females and males. Females performed worse on the MMSE (Mini-Mental State Examination) and CAMCOG (Cognitive and Self-Contained Part of the Cambridge Examination for Mental Disorders of the Elderly). Females had a larger white matter hyperintensity volume, while males more often showed (lacunar) infarcts. There was no difference in microbleed prevalence. Males had smaller normalized total brain and gray matter volumes. During follow-up, occurrence of cognitive decline and institutionalization was comparable, but mortality was higher in males. Discussion: Our results suggest that susceptibility and underlying etiology of VCI might differ by sex. Males seem to have more large vessel brain damage compared to females that have more small vessel brain damage.