The Government is about to turbocharge its fight against online child exploitation material by partnering with a UK watchdog.
It's using artificial intelligence that helps the powerful filter block up to 30,000 harmful sites per day.
But as AI prevents the spread of exploitative imagery, it is also helping to create it.
It's a terrifying new form of online bullying - pornographic deepfakes of students by students, using artificial intelligence.
"Yeah, I'm aware of a few of my colleagues who have had to deal with that sort of issue as recently as in the last fortnight," Secondary Principals' Association president Vaughan Couillault said.
Deepfakes create a sexual image of a person doing or saying things they haven't actually said or done.
The Secondary Principals Association said it's happening in New Zealand.
"There have been cases on both sides of the Tasman that target 50 to 60 young girls in the senior secondary space or indeed some staff so it's certainly increasing in quantum," Couillault said.
Justice Minister Paul Goldsmith told Newshub that in most circumstances non-consensual pornographic deepfakes are an offence under the Harmful Digital Communications Act. However, critics say the law only covers instances where police can show offenders' intention to cause a high level of harm.
Makes Sense, an organisation advocating for a safer digital space for children, believes New Zealand's regulation of harmful sexual content online is inadequate.
"We are significantly behind other countries and in a severe issue like children being exploited… It's unacceptable," Makes Sense co-founder and sex therapist Jo Robertson said.
But the Government on Tuesday unveiled one significant improvement.
"I am very pleased to announce that we have struck a new deal with a UK provider to increase how much material is removed from the internet that is child abuse material or child exploitation material," Internal Affairs Minister Brooke van Velden said.
The Government's new deal will see the number of blocked URLs increase from around 700 to up to 30,000 per day. It is adopting the Internet Watch Foundation, something Makes Sense has long been advocating for.
However, it won't stop material like deepfakes from being created.
"Technology is moving extremely fast. Where I'm hoping to focus my efforts is on that criminal activity, the worst of the worst content. There will be more information that I'm sure our Government will need to grapple with as technology moves fast, I'm open to having those conversations," van Velden said.
The Secondary Principals Association said we need to be teaching students about the harm of non-consensual deepfakes.
"Education is always the key so that students, those people, who are doing the bad stuff understand what impact it's having on those who are the victims," Couillault said.
Because the impact can't be deleted.
