Questions: We investigated the changing role of climate, forest fires and human population size in the broad‐scale compositional changes in Holocene vegetation dynamics before and after the onset of farming in Sweden (at 6,000 cal yr BP) and in Finland (at 4,000 cal yr BP). Location: Southern and central Sweden, SW and SE Finland. Methods: Holocene regional plant abundances were reconstructed using the REVEALS model on selected fossil pollen records from lakes. The relative importance of climate, fires and human population size on changes in vegetation composition was assessed using variation partitioning. Past climate variable was derived from the LOVECLIM climate model. Fire variable was reconstructed from sedimentary charcoal records. Estimated trend in human population size was based on the temporal distribution of archaeological radiocarbon dates. Results: Climate explains the highest proportion of variation in vegetation composition during the whole study period in Sweden (10,000-4,000 cal yr BP) and in Finland (10,000-1,000 cal yr BP), and during the pre‐agricultural period. In general, fires explain a relatively low proportion of variation. Human population size has significant effect on vegetation dynamics after the onset of farming and explains the highest variation in vegetation in S Sweden and SW Finland. Conclusions: Mesolithic hunter‐gatherer populations did not significantly affect vegetation composition in Fennoscandia, and climate was the main driver of changes at that time. Agricultural communities, however, had greater effect on vegetation dynamics, and the role of human population size became a more important factor during the late Holocene. Our results demonstrate that climate can be considered the main driver of long‐term vegetation dynamics in Fennoscandia. However, in some regions the influence of human population size on Holocene vegetation changes exceeded that of climate and has a longevity dating to the early Neolithic.