Western Australia Police have concluded a five-month trial of real-time facial recognition technology, during which the system scanned 1,380,882 faces across 100 deployments at 43 regional and metropolitan locations. The trial, which ended on Saturday, produced 122 arrests.
Under the trial guidelines, cameras mounted on or positioned near marked police vehicles scanned faces in designated public areas and matched them against an alert list drawn from police databases. Police said images of people who were not on the list were pixelated in real time and deleted immediately without being stored.
Alongside the arrests, the force said the alerts led officers to engage with 146 registered sex offenders and to locate 10 people for a welfare check.
Three operations at the Perth Royal Show accounted for 223,749 of the scans on their own, producing 15 alerts, three arrests and seven interactions with people on the alert list.
A police spokesperson confirmed the system generated nine incorrect alerts against 1,380,882 scans, which the force put at an error rate of 0.00065 per cent.
Addressing the false matches at a media conference on Sunday, the Western Australia police commissioner, Col Blanch, confirmed that officers had engaged with each individual. “I want to be very clear on those false alerts,” he said. “No one was arrested. No one made a complaint. Each one did result in an interaction with that person and asking them relevant questions.”
“It’s a very, very, very low number. It’s not zero. We’d love it to be zero. But I think the important thing for us is to engage with our community just like we would if a human thought someone was a wanted person,” Blanch said. He acknowledged the technology was “not perfect” and that authorities “knew this going into this trial”.
Blanch said the trial had “focused on using modern technology responsibly to identify people who presented a risk, while allowing the vast majority of people to go about their day without any impact”.
Civil liberties advocates disputed the way the figure had been framed. Bill Rowlings, chief executive of Civil Liberties Australia, said the force was “severely gilding the lily” and suggested the rate should instead be calculated by comparing how many of the arrests resulted in convictions. “We will only know what the incorrect rate is when the 122 people arrested have their day in court,” he said, adding that “if half the 122 arrested … are not convicted, the ‘incorrect’ rate will be 50%”. He said the scans had been performed “with no reasonable suspicion whatsoever”.
When the trial was announced, the state’s independent privacy watchdog, the Office of the Information Commissioner WA, said it had not been involved in implementing it and warned of “collective privacy harms”. The surveillance vans have now been removed from public deployment while the force conducts an internal evaluation of how the technology performed and its community impacts, including “reviewing the safeguards and whether all requirements were met”. Concerns had previously been raised about algorithmic bias, with facial recognition systems historically producing higher false-match rates for people with darker skin tones.
The arithmetic deserves attention, because it is the most interesting thing the trial produced. Nine incorrect alerts were divided by 1,380,882 scans. The denominator is not a measure of the system’s work. It is a measure of how many people walked past a van.
This has an attractive property. The force can improve its accuracy without touching the software. Every person who passes a camera and is correctly ignored is added to the bottom of the fraction, and the error rate falls. Accuracy, on this definition, is manufactured by pedestrians.
The implications for deployment planning are considerable. A van parked outside a railway station at rush hour is more accurate than the same van on a quiet rural road, running the same algorithm, on the same afternoon. A van parked where nobody walks has an error rate that cannot be calculated at all.
It follows that the cheapest way to reduce facial recognition errors in Western Australia is to increase foot traffic. The Department of Unnecessary Research notes that a well-chosen public holiday would do more for the figures than a software update, and costs the force nothing.
Perfect accuracy is also within reach. Scan the whole population of the state often enough and nine errors become a vanishingly small share of a very large number. The errors do not go away. They are simply outvoted.
Nine people were questioned who should not have been. Everyone else walked past a van and became part of the error rate. On the force’s own arithmetic, that is the achievement.