Introduction: Cosmetic results and quality of life (QoL) are increasingly important in the treatment of breast cancer. This study was designed to determine the relationship between QoL and both subjectively and objectively measured cosmetic outcomes of breast-conserving therapy (BCT), and its course over time. Methods: A total of 128 breast cancer patients who underwent BCT as part of a prospective randomized controlled trial were included. QoL was measured using the EORTC QLQ-C30 and QLQ-BR23 at baseline, 3, 6, 12, and 36 months. Cosmetic outcome was determined by patient self-evaluation, panel evaluation, and BCCT.core software. Results: By 36 months, all QoL factors except arm symptoms had returned to baseline or improved. After adjustment for patient and tumor characteristics, a significantly better QoL in terms of body image, pain, and arm and breast symptoms was found for good/excellent cosmetic outcomes compared to fair/poor outcomes, as measured by both patient self-evaluation and panel evaluation. The BCCT.core showed no correlation with any QoL factors. Conclusion: There is a high correlation between poor cosmetic outcome and low scores on quality of life indicators, underlining the importance of achieving a good cosmetic outcome after BCT.