For a long time parts of the town belonged to the manor of Vaivara. Later from these grounds have been created separate manors. In the city there used to be 3 manor houses: Peuthof – church manor, Sillamägi-mõis, Türsel (Türsamäe) – knights manor.
It is known that Türsamäe manor was built in 17th century, near the village, where there was a windmill. Remains of the windmill are still near the town. Tursamäe manor was a knight manor and belonged to nobility. Tõnu Walk (Waldman) was the last owner of this manor.