he does not know your phone number, they must have a whole database of those numbers maybe even from whatsapp groups