Purpose: Correct identification of tumour receptor status is important for treatment decisions in breast cancer. [18F]FES PET and [18F]FDHT PET allow non-invasive assessment of the oestrogen (ER) and androgen receptor (AR) status of individual lesions within a patient. Despite standardised analysis techniques, interobserver variability can significantly affect the interpretation of PET results and thus clinical applicability. The purpose of this study was to determine visual and quantitative interobserver variability of [18F]FES PET and [18F]FDHT PET interpretation in patients with metastatic breast cancer. Methods: In this prospective, two-centre study, patients with ER-positive metastatic breast cancer underwent both [18F]FES and [18F]FDHT PET/CT. In total, 120 lesions were identified in 10 patients with either conventional imaging (bone scan or lesions > 1 cm on high-resolution CT, n = 69) or only with [18F]FES and [18F]FDHT PET (n = 51). All lesions were scored visually and quantitatively by two independent observers. A visually PET-positive lesion was defined as uptake above background. For quantification, we used standardised uptake values (SUV): SUVmax, SUVpeak and SUVmean. Results: Visual analysis showed an absolute positive and negative interobserver agreement for [18F]FES PET of 84% and 83%, respectively (kappa = 0.67, 95% CI 0.48–0.87), and 49% and 74% for [18F]FDHT PET, respectively (kappa = 0.23, 95% CI − 0.04–0.49). Intraclass correlation coefficients (ICC) for quantification of SUVmax, SUVpeak and SUVmean were 0.98 (95% CI 0.96–0.98), 0.97 (95% CI 0.96–0.98) and 0.89 (95% CI 0.83–0.92) for [18F]FES, and 0.78 (95% CI 0.66–0.85), 0.76 (95% CI 0.63–0.84) and 0.75 (95% CI 0.62–0.84) for [18F]FDHT, respectively. Conclusion: Visual and quantitative evaluation of [18F]FES PET showed high interobserver agreement. These results support the use of [18F]FES PET in clinical practice. In contrast, visual agreement for [18F]FDHT PET was relatively low due to low tumour-background ratios, but quantitative agreement was good. This underscores the relevance of quantitative analysis of [18F]FDHT PET in breast cancer. Trial registration: ClinicalTrials.gov, NCT01988324. Registered 20 November 2013, https://clinicaltrials.gov/ct2/show/NCT01988324?term=FDHT+PET&draw=1&rank=2.