Evaluate the roadside AI trial after the flag

A roadside camera can flag a driving pattern for an officer to investigate.

The Devon and Cornwall trial announced in December 2024 used those observations to direct nearby officers toward vehicles. Further enquiries could include roadside testing for drink or drugs. Evaluating that use requires following the flag into the investigation it prompted. The original trial announcement

Revisiting the story in November 2026, I want to know whether those flags improved the work officers could do afterward.

Keep the trial dates visible

The police ethics committee's February 2025 minutes describe a proof of concept on the A396 near Tiverton. It ran on December 13 and 20, 2024. The minutes record four stops for impaired driving across those dates, with the supplier taking the collected data away for evaluation. Police ethics committee minutes

Four stops give a count of police activity during a small pilot. Calculating detection performance would also require the number of correct flags with the missed cases, then a comparison with ordinary officer deployment.

The police's January 2025 disclosure also described a trial awaiting evaluation. That account establishes its status at the time. A claim about nationwide deployment in 2026 would require evidence of a later deployment. Police trial disclosure

Evaluate each handoff

The camera records driving behavior before the system produces a flag. An officer then chooses how to investigate. A finding about the driver depends on the evidence obtained through that investigation.

Consistent classification can still produce too many unhelpful stops. A useful flag may arrive too late for an officer to act. Changing roads or conditions may also change performance. Each possibility affects the outcome even if the preceding step worked.

The announcement describes information reaching nearby officers in real time. That handoff needs location accuracy sufficient to identify the intended vehicle. Its latency also affects whether an officer can investigate the observation.

I'd evaluate the path through to investigation. Processing an image creates one intermediate result; the evaluation needs to establish what officers could use that result to do.

Keep results with the task they measure

Vision Zero South West later expanded its camera initiative for mobile-phone use and seatbelt detection. Its announcement describes AI with human reviewers, extending coverage to the M5 with support through April 2027. The camera initiative expansion

That deployment concerns objects visible in an image. The impairment pilot infers a possible condition from a driving pattern. Each task needs its own performance evaluation.

Reported changes in phone-use or seatbelt detections describe those deployments. Measuring the impairment system's effect on crashes would require an outcome study of that system.

The pilot records document its design with the recorded stops. They also record a request for evaluation. A numerical estimate of lives saved remains a separate claim requiring evidence beyond those records.

Count false flags as workload

Consider a hypothetical evaluation with 10,000 vehicles, of which 100 meet the target condition. Suppose the system flags 80 of those 100. It also flags 5% of the remaining 9,900 vehicles.

The result is 80 relevant flags alongside 495 false flags. About one in seven flags meets the target condition. These numbers illustrate the arithmetic; they were chosen for this example, with no claim about Acusensus or the trial's measured performance.

An uncommon target can therefore produce substantial investigation work even with a low false-positive rate. The evaluation needs a denominator for observed vehicles, followed by counts at each stage. It also needs a defensible method for estimating missed cases.

Investigating only flagged vehicles leaves the unflagged group largely unexamined. A method for checking that group is needed before the evaluation can estimate how often the system missed its target.

Give reviewers evidence for disagreement

The ethics committee asked about drivers whose disabilities might affect their driving pattern. Vehicle adaptations raised a related concern. It requested more detail about positive results and false positives for later discussion. The minutes record questions for evaluation, leaving any measured disparity to be investigated. The committee's evaluation request

A reviewer needs context for the flag. They also need the ability to consider another explanation before disagreeing with the classification. A review process that rewards quick acceptance can undermine that check even when a person is formally involved.

I'd also specify how observations are handled. Access needs an owner, with responsibility assigned for evaluating the pilot. Those decisions make it possible to examine how the collected data serves the trial's stated purpose.

An evaluation should show which flags led to proportionate investigations, then report the outcomes those investigations established. That record would let readers judge the system's contribution to road safety.