Don Brash says Auckland Mayor Phil Goff's decision not to let far-right speakers use council venues was wrong, and is part of a group that's raised $50,000 to take the council to court.
Free Speech Coalition consists of 10 academics, lawyers and former politicians from "across the political spectrum".
The group was formed in response to Auckland Mayor Phil Goff's decision not to allow Canadian far-right figures Stefan Molyneux and Lauren Southern speak at council venues.
Now that the group has raised $50,000, Free Speech Coalition's official website says they will be going ahead with legal action to "defend free speech".
Speaking to Wendyl Nissen on RadioLIVE, Dr Brash says the issue is part of a wider argument around freedom of speech.
"If free speech is only permission to say something nice about something no one much cares about, it's not really free speech at all," he said.
"I think Phil Goff was entirely wrong to say taxpayer or ratepayer funded facilities cannot be used by people whose views he disagrees with."
- Auckland 'alt-right' event cancelled due to 'health and safety'
- Winston Peters would've let controversial far-right speakers into NZ
He says he's not familiar with Mr Molyneux or Ms Southern, who have been accused of hate speech for their views on Islam, feminism and gender.
"I don't know what their views are, I've never heard them speak, I've never heard anything they've written," he says.
"Presumably they cannot say, 'Please go and kill somebody' because that would be illegal. But if they say 'we don't like particular groups', well that's free speech. I think they should be allowed to say that."
Dr Brash says he's wary of condemning certain opinions as hate speech.
"It is a perfectly accurate statement to say that in the Koran, gays are to be executed - simply stating that as fact should surely be allowed," he explained.
"It's quite a different thing to say therefore we should do something awful to Muslims. They're two different things."
He says Free Speech Coalition includes people associated with left-wing politics, such as commentator Chris Trotter - not just right-wing figures like himself.
RadioLIVE host Tony Amos agreed with Dr Brash's views, saying that to allow politicians to "dictate" who can and can't speak at public facilities is "moving into very dangerous territory".
"I don't want to be 'protected' from what I may hear by a politician," he says. "That's not the role for my parliamentarians or my local government."
Mr Amos says the line keeps shifting in regards to what opinions are deemed acceptable.
"It's all very well to express good sympathy or empathy to protect people who may be vulnerable, but when it comes at the expense of everybody's freedom to say what they believe or what their opinion is, that's worse."
He claims New Zealand already has good legislation that protects people from being attacked, and that we don't need more.
Newshub.
