Dude, before the colonizers came, some SL women did not wear a top..
This is related to culture. Cultures and their values evolove with the time and the current cultural standards do not approve this. And during the colonial times, only the low castes didn't wear anything to cover their upper body. If a high caste woman did the same, that would be considered a disrespect to the religion and culture.
Same applies when visiting the Vatican which most of the so called modern thinkers in Sri Lanka don't cricize.
When it comes to the humans, we can symbolise our thought with our attire. If we want to disrespect someone, we can do that with what we wear also.
e.g. Visiting someone's wedding at a 5star hotel wearing a T-shirt and a short with pair of thongs could easily give the bad impression that you did not take them seriously. But a beggar passing by the hotel will not give the same impression because he is a beggar.