Social inclusion policies often assume that community integration is beneficial for all people with disabilities. Little is known about what actually happens in encounters between people with and without severe intellectual disabilities in the public space. Based on social-constructionist and responsive-phenomenological insights, we performed participant observation, semi-structured interviews and researcher reflexivity to study encounters between Harry (pseudonym), a man with a severe intellectual disability, his neighbours, fellow service-users, support professionals, and the first author. A thinking-with-theory strategy was adopted to interpret and deepen observations and reflections. We argue for more ‘experimental-relational spaces of encounter’ between people with and without severe intellectual disabilities, wherein the latter put down their verbal and cognitive weaponry. This proved to be more appropriate for spending ‘quality time’ with Harry then typical satisfactory neighbourhood interactions–often embedded in verbality, habits, routines and rationalizations that do not reflect the existence of people with severe intellectual disabilities. Points of interest An increasing number of people with severe intellectual disabilities live in homes in a neighbourhood setting. However, in everyday neighbourhood life people with and without severe intellectual disabilities hardly ever encounter each other in a way both of them like. The research shows that this lack of pleasant encounters can be related to powerful rules about how we should interact in the public space in the 21st century. In the neighbourhoods we studied, people without intellectual disabilities are in control of these rules. They prefer verbal and cognitive interactions in the public space. Non-verbal and less cognitive approaches are often seen as inappropriate. The research recommends that people without intellectual disabilities should open up for more fitting ways to communicate with people who cannot speak (for themselves).