Federer explained why he did not refuse to sign autographs for fans