In animals and plants, survival and reproduction are strongly influenced by population size. Fitness decrease in overcrowded populations has been a keystone research paradigm for decades, but the awareness that small populations are at high extinction risk recently urged population biologists to focus on fitness reductions at low numbers, and in particular, on a phenomenon known as the Allee effect. We applied these ecological concepts to scientific production, in order to assess whether size mattered in scientific groupings. For this, we assumed that researchers are distributed across distinct populations of different sizes (universities) and transmit memes (scientific findings) via publications, and analyzed scientific production in North America, via the Science Citation Index Expended database. We show that memetic fitness, estimated via the number of citations gained per individual, is lower in small populations than is large ones. The underlying cause is not a lack of interactions within populations, but rather, lower memetic diversity at low population size. These results, suggesting lower fitness in smaller populations, highlight a surprising parallel between the biology of small populations and the functioning of scientific research.