Whether or not Twitter should expose people to a variety of views is an interesting question, but it has nothing whatsoever to do with free speech. The free speech issue here is protecting Twitter's right to decide what its platform says. How it exercises that right is simply not a free speech issue.
In fact, arguing that people should exercise their right to free speech different is the very opposite of an actual free speech argument.
Should Twitter be permitted to ban whoever it pleases? That's a free speech issue. And unless you think the answer is no, then who it should ban is not a free speech issue any more than whether Milo should talk about racism or bunnies is a free speech issue.
RE: While I may not be a fan of Milo Yiannopoulos, we should not silence him. The erosion of free speech has begun.