Background: Which neighbourhood factors most consistently impact on depression and anxiety remains unclear. This study examines whether objectively obtained socioeconomic, physical and social aspects of the neighbourhood in which persons live are associated with the presence and severity of depressive and anxiety disorders. Methods: Cross-sectional data are from the Netherlands Study of Depression and Anxiety including participants (n = 2980) with and without depressive and anxiety disorders in the past year (based on DSM-based psychiatric interviews). We also determined symptom severity of depression (Inventory of Depression Symptomatology), anxiety (Beck Anxiety Inventory) and fear (Fear Questionnaire). Neighbourhood characteristics comprised socioeconomic factors (socioeconomic status, home value, number of social security beneficiaries and percentage of immigrants), physical factors (air pollution, traffic noise and availability of green space and water) and social factors (social cohesion and safety). Multilevel regression analyses were performed with the municipality as the second level while adjusting for individual sociodemographic variables and household income. Results: Not urbanization grade, but rather neighbourhood socioecononomic factors (low socioeconomic status, more social security beneficiaries and more immigrants), physical factors (high levels of traffic noise) and social factors (lower social cohesion and less safety) were associated with the presence of depressive and anxiety disorders. Most of these neighbourhood characteristics were also associated with increased depressive and anxiety symptoms severity. Conclusion: These findings suggest that it is not population density in the neighbourhood, but rather the quality of socioeconomic, physical and social neighbourhood characteristics that is associated with the presence and severity of affective disorders.