More than five lakh people are poor