An important dimension of individual differences, independent of general cognitive ability (GCA), is specialization for verbal or spatial ability. In this study we investigated neuroanatomic, network, and personality features associated with verbal vs. spatial ability. Healthy young adults (N = 244) were evaluated with (1) a cognitive battery yielding measures of verbal and spatial ability independent of GCA, (2) structural MRI scans providing measures of surface area, cortical thickness, and DTI scans allowing calculation of diverse network metrics, and (3) Big-5 personality measures. Sex differences were found for cognitive, personality, anatomic, and network measures. In men only, cortical surface was significantly, differentially related to the cognitive variables, predicting spatial but not verbal ability. Similarly, in men only, neuroticism and quirkiness (the overall deviation from the mean across scales) were significantly, differentially related to the two cognitive variables. Different graph metrics predicted spatial ability in men (overall connectivity) and women (clustering). Verbal-spatial specialization was related to sex, cortical surface area, network organization, personality, and vocational interests. Most of the identified correlates of this cognitive specialization were found only in men, and mostly for spatial ability. Taken together, these results identify a suite of neurobehavioral features whose covariance is partially sex-specific.