Graffiti has been scratched on buildings and stones pulled from walls at a rarely seen abandoned village.
"Unprecedented" numbers of visitors have flocked to Ladybower Reservoir in Derbyshire since low water levels revealed the ruins of Derwent.
But park rangers say they have had to stop people trying to remove material from the site.
The Peak District National Park authority said it was shocked by the damage to the "iconic" structures.
Derwent was flooded in the 1940s to make way for Ladybower Reservoir, which was built between 1935 and 1943.
But this year's long dry spring and summer has reduced the water levels so low that remains of the village have been exposed, which had drawn crowds of people in recent weeks.
One visitor, Clare Whittaker, was left "upset and mad" when she saw both children and parents taking "beautiful" stones from the walls and throwing them in to the mud.
"My Sunday visit to Ladybower left me very upset," she said. "We're very lucky to get to see some good history but people were deliberately destroying that history."
Steve Rowe, team leader of the Edale Mountain Rescue Team, said he had also seen vandalism at the site.
"There's a fair amount of graffiti and defacement on the ruins. It's a huge part of our history and now 'Cheryl' and 'Steve' have scratched their names in the rock.
"We need to look after it, we have a responsibility like you would at any historical site."
Anna Badcock, the park authority's cultural heritage manager, said: "Whilst we understand that people are fascinated by the appearance of these usually hidden ruins, the structures remain an iconic archaeological feature of the Peak District National Park.
"As we wouldn't expect people to vandalise any of the National Park's many heritage buildings or other archaeological features, the remains of the homes and other submerged buildings are no exception.
"We urge people to leave these features intact to open a valuable window onto history, not just today, but for future generations to enjoy."
People are also being urged to look at the ruins of Derwent from the banks of the reservoir, rather than walking on the mud, which can be dangerous.
Edale Mountain Rescue said they had already saved one man who was stuck up to his chest for almost two hours.
Mr Rowe said: "It's exciting and interesting; come and have a look but appreciate the risk.