Raloxifene is a selective estrogen receptor modulator that may delay the onset of mild cognitive impairment in elderly women. Effects of raloxifene treatment on mental performance in males remain to be investigated. In a previous functional magnetic resonance imaging (fMRI) study, we showed that raloxifene treatment enhanced brain activation in elderly males during encoding of new information (faces) into memory. The current study used fMRI in the same group of subjects to screen for effects of raloxifene treatment on brain function during face recognition. Healthy elderly males (n = 28; mean age 63.6 years, SD 2.4) were scanned at baseline and after 3 months of treatment with either raloxifene 120 mg (n = 14) or placebo (n = 14) in a randomized, double-blind, placebo-controlled study design. Functional data were analyzed in an event-related fashion with respect to correct hits and correct rejections using FSL software. Performance data were analyzed with respect to recognition accuracy, latency, and response bias. Functional effects of treatment were found on brain activation related to correct hits only. When compared to placebo treatment, raloxifene treatment enhanced brain activation in the left posterior parahippocampal area (Z = 3.9) and right inferior prefrontal cortex (Z = 3.5). Recognition accuracy scores remained stable in the raloxifene group, whereas the placebo group showed a small but significant decrease in accuracy scores (p = 0.02). No significant effects were found on response bias or latency. In conclusion, raloxifene treatment affects brain function during memory performance in a way that may reflect increased arousal during initial encoding, with downstream effects on brain function during retrieval of information. Behaviorally, such neurofunctional effects may actively block decreased memory performance as a result of context-dependency. The validity of these predictions can be tested in large-scale clinical trials.