Small is Bountiful
This is where it's important to identify X. If X is responsibility, then a large group enables people to avoid individual responsibility by distributing it into the collective. That's where 'everybody does it' and other famous excuses for reprehensible behaviour come in.
Conversely, the larger the group, the less nimble it is. That is why you increase the rate of chemical reaction or the flexibility of a business by breaking a large bloc down into smaller units. The activity of a catalyst is much enhanced by distributing it more evenly in smaller particles.
When I look at social phenomena like mobs and megachurches, I think a lot about this. It is all too easy for those embedded in a large body to be insulated from the reactive environment; it is all too easy for those at the edge to have no connection or communication with those at the centre.
The human mind is meant to be part of a network, with strong connections to very few other nodes and weaker connections to a few others. When the group is too large, the human mind turns in on itself to consolidate its 'picture' of the network.
This gives us a natural size for group effectiveness — it's the number of people-concepts that typical members can hold in their own individual brains at any one time, because that's how humans interact, by simulating other humans and their reactions (the 'What Would X Do?' idea). That's the basis of empathy.
So yes, small is better, unless you want to diminish individual responsibility, or hide something, or make a large bunch of people who don't really know or feel for each other. That's your call.