Social withdrawal is one of the first and common signs of early social dysfunction in a number of important neuropsychiatric disorders, likely because of the enormous amount and complexity of brain processes required to initiate and maintain social relationships (Adolphs, 2009). The Psychiatric Ratings using Intermediate Stratified Markers (PRISM) project focusses on the shared and unique neurobiological basis of social withdrawal in schizophrenia, Alzheimer and depression. In this paper, we discuss the working definition of social withdrawal for this study and the selection of objective and subjective rating scales to assess social withdrawal chosen or adapted for this project. We also discuss the MRI and EEG paradigms selected to study the systems and neural circuitry thought to underlie social functioning and more particularly to be involved in social withdrawal in humans, such as the social perception and the social affiliation networks. A number of behavioral paradigms were selected to assess complementary aspects of social cognition. Also, a digital phenotyping method (a smartphone application) was chosen to obtain real-life data.