The social brain hypothesis proposes that the complexity of human brains has co-evolved with increasing complexity of social interactions in primate societies. The present study explored the possible relationships between brain morphology and the richness of more intimate 'inner' and wider 'outer' social circles by integrating Bayesian hierarchical modeling with a large cohort sample from the UK Biobank resource (n=10,000). In this way, we examined population volume effects in 36 regions of the 'social brain', ranging from lower sensory to higher associative cortices. We observed strong volume effects in the visual-sensory network for the group of individuals with satisfying friendships. Further, the limbic network displayed several brain regions with substantial volume variations in individuals with a lack of social support. Our population neuroscience approach thus showed that distinct networks of the social brain show different patterns of volume variations linked to the examined social indices.