Exposure to stress is one of the best-known negative regulators of adult neurogenesis (AN). We discuss changes in neurogenesis in relation to exposure to stress, glucocorticoid hormones, and inflammation, with a particular focus on early development and on lasting effects of stress. Although the effects of acute and mild stress on AN are generally brief and can be quickly overcome, chronic exposure or more severe forms of stress can induce longer lasting reductions in neurogenesis that can, however, in part, be overcome by subsequent exposure to exercise, drugs targeting the stress system, and some antidepressants. Exposure to stress, particularly during the sensitive period of early life, may (re)program brain plasticity, in particular, in the hippocampus. This may increase the risk to develop cognitive or anxiety symptoms, common to brain diseases like dementia and depression in which plasticity changes occur, and a normalization of neurogenesis may be required for a successful treatment response and recovery.