Chronic obstructive pulmonary disease (COPD) is associated with substantial health impact that may already become apparent in early disease. This study aims to examine the features of subjects with COPD in a Dutch population-based sample and compare their physical status, mental status, and social status to non-COPD subjects. This study made use of Longitudinal Aging Study Amsterdam (LASA) data. Demographics, clinical characteristics, self-reported diseases, post-bronchodilator spirometry, physical, mental, and social status were assessed. A number of 810 subjects (50.5% male, mean age 60.5 ± 2.9 years) were included. Subjects with COPD (n = 68, mean FEV1 67.6 [IQR 60.4–80.4] %.) had a slower walking speed than non-COPD subjects, p = 0.033. When compared to non-COPD subjects, COPD subjects gave a lower rating on their health (physical subscale of SF-12: 15 [IQR 16.0–19.0] vs. 18 [IQR 11.0–17.0] points) and life (EQ5D VAS: 75 [IQR 70.0–90.0] vs. 80 points [IQR 65.0–85.5]) surveys. COPD subjects also had a more impaired disease-specific health status (CAT: 9.5 ± 5.9 vs. 6.7 ± 5.2, respectively), were less likely to have a partner (69% vs. 84%, respectively) and received emotional support less often (24% vs. 36%, respectively) compared to non-COPD subjects (All comparisons p < 0.001). In a population-based sample, subjects with COPD had a reduced physical performance, a more impaired disease-specific health status and were more socially deprived compared to non-COPD subjects. These impairments need to be taken into consideration when setting up a management program for patients with mild COPD.