Major depressive disorder (MDD) has a higher prevalence in women with supraphysiologic androgen levels. Whether there is also an association between depression and androgen levels in the physiological range, is unknown. This study examined if women with current MDD have higher androgen levels compared to women who have never had MDD, and if androgen levels are associated with onset and remission of MDD. In 1659 women (513 current MDD, 754 remitted MDD, and 392 never MDD), baseline plasma levels of total testosterone, 5α-dihydrotestosterone, and androstenedione were determined with liquid chromatography-tandem mass spectrometry, and dehydroepiandrosterone-sulfate and sex hormone binding globulin (SHBG) with radioimmunoassays. Free testosterone was calculated. MDD status was assessed at baseline, and at 2 and 4 years follow-up. Women were aged between 18 and 65 years (mean age 41) with total testosterone levels in the physiological range (geometric mean 0.72 nmol/L [95% CI 0.27–1.93]). After adjusting for covariates and multiple testing, women with current MDD had a higher mean free testosterone than women who never had MDD (adjusted geometric mean 8.50 vs. 7.55 pmol/L, p = 0.0005), but this difference was not large enough to be considered clinically meaningful as it was consistent with statistical equivalence. Levels of other androgens and SHBG did not differ and were also statistically equivalent between the groups. None of the androgens or SHBG levels predicted onset or remission of MDD. Our findings support the idea that plasma androgens within the physiological range have no or only limited effects on depressive disorders in women.