The cause of prolactin alterations in transgender persons is often assigned to oestrogens, but the precise cause and time course during different phases of cross-sex hormone treatment (CHT) remain unclear. In this study, we prospectively examined prolactin levels in 55 female-to-males (FtMs) and 61 male-to-females (MtFs) during the first year of CHT. Because long-term prolactin data were not available in this population, we studied these levels in a retrospective population of 25 FtMs and 38 MtFs who underwent gonadectomy. FtMs were treated with testosterone and MtFs with estradiol, with or without the anti-androgen cyproterone acetate (CPA) (after gonadectomy CPA is cessated). During the first year of CHT, prolactin decreased with 25% (95CI: −33%, −12%) in FtMs and increased with 193% (95CI: 156%, 219%) in MtFs. Eighteen MtFs developed hyperprolactinemia (≥0.6 IU L−1). In the retrospective population, post-gonadectomy levels in FtMs were lower than baseline levels (−39%; 95CI: −51%, −20%) while in MtFs post-gonadectomy levels and baseline levels were comparable (−6%; 95CI: −24%, 15%). No hyperprolactinemia was found after gonadectomy. In conclusion, in FtMs, prolactin decreased consistently during CHT and in MtFs, prolactin increased during pre-surgical CHT but normalised after gonadectomy. It is likely that CPA induces increasing prolactin levels in MtFs.