Woking had almost a dozen holiday homes in 2021, new figures show.
The 2021 census figures from the Office for National Statistics highlight the growing problem holiday lets are having on the housing market, reducing supply and driving up house prices, especially in tourist areas, such as Wales and the South West of England.
Generation Rent, a campaign group aimed at protecting renters' rights, said the high concentration of holiday homes is "taking properties out of the reach of people who need somewhere to live".
The figures show there were approximately 10 holiday homes in Woking when the census was taken in March 2021.
Holiday let numbers are rounded to the nearest five and any under 10 are suppressed.
It meant there were 0.2 holiday homes for every 1,000 dwellings in the area.
The Isles of Scilly had the highest proportion of holiday homes in the UK, though it has a very small number of total households.
Excluding the Isles of Scilly, South Hams in Devon had the highest rate of holiday homes, at 44.1 per 1,000 dwellings. This was followed by similarly touristy areas – Gwynedd, which contains Snowdonia in north Wales, North Norfolk and Anglesey.
In the UK, more than 75,000 addresses were used as holiday homes in 2021 – of these, around 11,995 were based in the South East, equating to 3 per 1,000 dwellings.
Dan Wilson Craw, deputy director at Generation Rent, said: "Airbnb has made owning a holiday home more lucrative and we saw a huge increase over the pandemic when international travel was suspended for much of 2021.
"There is fierce competition for the limited number of homes coming on the market and it is pricing out people who grew up in these communities and want to work in the tourist industries Airbnb is meant to be supporting."
Mr Wilson Craw called on local authorities to license holiday let operators and introduce higher council tax on second homes. He also urged the Government to remove "tax perks which make holiday lets even more profitable than normal tenancies".
"This, combined with a renewed effort to build more houses nationwide, is the only way that renters in holiday hotspots will be able to feel they are not at risk of being driven out of the communities they want to build their lives, stay with their families and contribute to society in," he added.
A Government spokesperson said it has already introduced a higher rate of stamp duty for second properties, closed tax loopholes on holiday lets in April, and will give councils the power to apply a council tax premium of up to 100% on second homes through the Levelling Up and Regeneration Bill.
They added: "We are taking action to combat the adverse impact that second homes can have on local communities, particularly in tourist areas."