Road Safety News
 

Academic challenges TfL camera claims

Tuesday 15th December 2015

Transport for London (TfL) is standing by its claim that fixed speed cameras have cut killed and serious injuries (KSIs) by 58%, despite a prominent academic concluding that the cameras have delivered no casualty benefit (Local Transport Today (LTT)).

The LTT article (subscription only), published on 11 December, says TfL officials were told their claims were incorrect in 2014 by Mike Maher, an honorary professor at University College London, and member of the Institute of Mathematics and its Applications.

Writing in LTT, Mr Maher said: “What TfL did not say was that this percentage fall in KSIs at camera sites matched that across London generally over the same period; that is, no allowance was made for trend.”

Mr Maher’s article suggests that TfL has repeatedly claimed that the capital’s fixed speed cameras have delivered, on average, a 58% reduction in KSIs at camera sites and helped prevent 500 deaths or serious injuries annually.

He adds that TfL has used this claim to help justify an investment programme to replace hundreds of wet film fixed cameras with new digital equipment.

The papers Mr Maher presented to TfL officials, which have been released following a Freedom of Information request, highlight findings from his analysis of casualty data for hundreds of fixed camera sites in the capital.

Mr Maher concluded: “Although the estimates of the average camera effect from these variants of the basic model are different, they all agree that there is no discernible beneficial effect from the fixed cameras on either slight (casualties) or KSIs.”

Mr Maher told LTT he is “very neutral” on the issue of speed cameras and is not an “anti-camera lobbyist”. He also says there is evidence of “significant benefit” from the use of red light cameras, with a 17-20% reduction in KSIs as a result of their deployment.

In a statement to LTT, TfL said: “On average, a reduction of 58% in fatal and serious collisions was observed in the three-year (after) period compared to the three-year period before safety cameras were installed and this was the figure quoted in papers seen by TfL’s finance and policy committee.

“We value the contribution made by Professor Mike Maher to the debate on speed camera effectiveness. We disagree that the effect of speed cameras is negligible but we are of course interested in any new statistical analysis on this topic.”


Comments

Comment on this story
Report a reader comment

What's your view - comment on this story:

I confirm that I have read and accept the moderation policy and house rules relating to comments posted on this website.
Your comment:
Your name and location:
Your email:
Captcha [What is this?]

I think it would be better if the authorities made no claims of collision reductions at speed camera sites at all - better still, conceal the cameras altogether and don't publicise their locations. As long as there are sufficent speeders to catch on a road that, should be the criterion.

The overall benefits would remain unchanged and best of all, people wouldn't have to get bogged down in statistical games and mind-mumbing analysis.
Hugh Jones, Cheshire

Agree (4) | Disagree (6)
-2

This perpetual debate about the effectiveness of location specific and highly visible cameras plays right into the hands of those who don't seem to want speed control at all.

The wider debate can be distilled down to the following:

Do you agree with limiting the speed of vehicles by mandatory and enforceable speed limit?

If so, do you believe that speed limits should be adhered to wherever they may be?

If so then why not have randomly placed, moveable, covert cameras? This would maximise the compliance across the widest area and for all speed limits.

I would suggest that what happens at an individual camera location site may be interesting but the cumulative effect of cameras on compliance across the whole road network is what is far more important.
Rod King, 20's Plenty for Us

Agree (6) | Disagree (5)
+1

Good questions, Bob. In order:

1) Not really. Collision numbers at a location may consist of a random element and a systematic element.

2) It doesn't. Those factors contribute to general trends and need accounting for separately (as Professor Mike Maher did)

3) RTM is an effect caused by "selection" and is therefore likely to occur whenever collision histories are known within the site-selection process.

There is a more detailed explanation here:
http://speedcamerareport.co.uk/04_rtm.htm
Dave Finney, Slough

Agree (5) | Disagree (2)
+3

Don't worry Bob, you are not on your own as many people do not understand statistical variation either!

Are collisions random events? If they were random events then we would be able to find no patterns in the data, but the accident data is rich with many patterns that tell us a great deal about the accident causation process. According to an analysis of Stats19, accidents are directly correlated with traffic density at around 96% so as the traffic density increases the number of accidents increases along with it. Speeding on the other hand (the subject of this thread) is inversely correlated with traffic density to almost the same degree so as traffic density (and accidents) increases the incidence of speeding decreases and vice versa. There is however a bump in the data at pub kicking out time which would only be expected.

All the elements in the road transport system exhibit a degree of variation that is either inherent in the element itself or as a result of variation in one or more of the other elements. All these variables aggregate and disaggregate in various ways to form the overall properties of the system that we can observe either at particular times or over a spread of time. Most unwanted events (usually between 85-99%) in systems happen when the normal variation in the system aggregates in unexpected or unusual ways which is usually lumped together under the term 'common causes'. Sometimes though there is abnormal variation introduced into the system from 'special causes' that are not an inherent part of the system function and are usually non-quantifiable. Our problems start when interventions that only work for the elimination of special causes are used to try and eliminate common causes.
Duncan MacKillop. No surprise - No accident.

Agree (7) | Disagree (0)
+7

Having spent in excess of 10,000 hours on camera data I understand RTM at least as well as anyone else! It is a statistical effect arising from the laws of chance, and understood in general for >100 years:

6000 people throw dice, it is inevitable that~1000 score 1, ~1000 score 2 etc and that the average score is ~3.5. Now select the 1000 who scored 6 to throw again - do they all throw 6 again? Of course not, the odds against that are near infinite. They are bound to average ~3.5.

GB accident numbers are clearly not random, but how many happen in particular tiny areas and when is close enough to random for the same rules to apply to camera sites- as I have proved using Stats19 data.

The degree of selection bias and inevitable matching fall is a factor of data volatility /accident rate and selection criteria. But there is no need whatever to measure it, nor is it possible in any case because it varies all over the place - all that matters is to understand it and that the RTM fall happens at the stroke of midnight on the last day of the selection period - no shades of grey, numbers are either abnormal because they were selected as such, or normal because they were not - like dice scores.

And as Dave Finney pointed out in 2010 inevitable data acquisition, selection, planning and logistic delays mean that RTM is dead and buried well before cameras can have any effect.

More recently analysts have used Dave's method, or something like it, to ignore selection bias and RTM and project forward data from before bias started and trend adjust it to determine the "normal" level for comparison with post-installation data. Most but not all adjust for trend in the particular police area, but because they fail to realise that if trends vary between areas - as they certainly do - they will also vary within each area. Ditto by speed limit, huge variations. For those reasons accurate trend adjustment - over at least 10 years is not remotely possible so they still end up with totally unreliable results.

My own method, using Stats19 data for 32 police areas, showing bias and RTM but ignoring them, and recording what actually happened at 1000's of sites, not relative to what happened elsewhere but as any change of trend at those same sites, provides accurate results - no measurable effect.
Idris Francis Fight Back With Facts Petersfield

Agree (5) | Disagree (1)
+4

The Road Safety Good Practice Guide is archived here:

http://webarchive.nationalarchives.gov.uk/20070129122531/http:/www.dft.gov.uk/pgr/roadsafety/laguidance/roadsafetygoodpracticeguide

Look at section 5: measuring effectiveness 5.119 Regression to the mean - Mayer and Mountain 1988 are referenced.

Also interesting is 5.124 'Accident migration.'

So, anyone involved with evaluating speed camera effectiveness and effects should have read this document years ago and factor RTTM into any claims for casualty reduction.
Paul Biggs, Tamworth

Agree (5) | Disagree (3)
+2

Could one of the RTM specialists that are active here explain a few things to me:

Does RTM therefore assume that all collisions are random events?

How does RTM take into account changes in other factors (development, population, traffic demand, etc. e.g. things not connected to cameras) that might result in more collisions during the site selection period?

Does that mean that RTM might not apply automatically in all places or that RTM cannot account for these other changes?

I'm a bit confused as I'm not in road safety or stats.
Bob, North West

Agree (4) | Disagree (0)
+4

If there were 'adverse effects', wouldn't such a thorough study as the professor's, have highlighted them - with evidence naturally? I can't think of any, let alone any having an influence beyond the site of a camera.
Hugh Jones, Cheshire

Agree (3) | Disagree (7)
-4

Hugh is right in claiming that "the effect of enforcement and penalties goes beyond the site of the speed camera and is therefore un-trackable and incalculable" but what he seems not to realise is that this applies to adverse effects as much as benefits.
Idris Francis Fight Back With Facts Petersfield

Agree (7) | Disagree (7)
0

We wouldn't need a debate on effectiveness if we routinely adopted appropriate study designs to determine the effectiveness of road or other social interventions. As recommended by the Cabinet Office Behavioural Insights Team http://www.behaviouralinsights.co.uk/publications/test-learn-adapt-developing-public-policy-with-randomised-controlled-trials/

In other words, we could demand the kind of evidence to help prevent you having a road injury that is mandatory for all the medical interventions they use in A&E to repair you after a road injury.

For what it's worth, if TfL are interested in "interested in any new statistical analysis on this topic" they *could* make the data open access and we could have a new study every week to talk about. The problem isn't that "statistics can prove anything" but that in the absence of control of confounding variables we can all project our own assumptions on the data. There are casual methods and counterfactual methods that can help quantify this, but it would still be quicker and cheaper to use a properly designed study in the first place.
Paul Hewson, Plymouth

Agree (8) | Disagree (2)
+6

One last comment please - it is a pity, but not surprising, that the media (including RSGB sadly) chose to highlight the negative aspect of Prof Maher's study (speed cameras) and not the positive i.e "He also says there is evidence of “significant benefit” from the use of red light cameras..." Why not focus on that instead? It might have been a better headline for a road safety forum!
Hugh Jones, Cheshire

Agree (7) | Disagree (9)
-2

No doubt there will be more about the analysis of TfL speed cameras in the 8th January LTT. I don't know if Hugh (or Rod King for that matter) is a qualified statistician, but his last comment seems speculative, contradictory and makes little sense.
Paul Biggs, Tamworth

Agree (9) | Disagree (12)
-3

If I'm allowed a further comment, I can only reiterate what I have said on other occasions, that the effect of enforcement and penalties goes beyond the site of the speed camera and is therefore untrackable and incalcuable. Rod King has already pointed this out and more importantly, both Professor Maher and TfL should have thought of it.
It's short-sighted to assess the effect of motoring prosecutions only with respect to the site of the offence. Speeders tend to speed everywhere they can, so where they're detected is not as important as the fact that they have been detected and prosecuted.
Hugh Jones, Cheshire

Agree (12) | Disagree (9)
+3

I would urge people to read Professor Mike Maher's report, or at least read the articles in Local Transport Today. The report uses the Four Time Periods method and it is probably the most accurate official analysis of speed cameras to date.

I would like Transport for London to clarify their response. Have they failed to understand the report, or are they just refusing to acknowledge the conclusions and act on them?

Remember that the government have introduced a new rule that authorities must publish "the impact of speed cameras on road safety". TfL must therefore do this but what are the penalties if they don't? TfL could, of course, take the simplest and cheapest option of all, just run all their speed cameras within scientific trials. Surely the public have a right to expect such an evidence-led approach?
Dave Finney, Slough

Agree (10) | Disagree (12)
-2

Apologies for appearing facetious or cynical, but as a professor and a member of the Institute of Mathematics, could Mr Maher quantify 'very neutral', to us non-academics? Is that 110% of normal neutral? The gear shift on my car does not have several neutral positions for me to chose from - just the one.

Good that the professor acknowledges the effectiveness of red-light cameras though, but I wonder if he has taken into account the EuroNCAP ratings of the vehicles involved, which as someone has suggested may be the real reason for significant drop in KSIs near cameras.
Hugh Jones, Cheshire

Agree (10) | Disagree (9)
+1

Since 95% of accidents, 84% of fatalities are by non-speeding drivers, with 63% of pedestrian injuries and 43% of fatalities due to pedestrian failing to look properly, the maximum possible gain from cameras would be 5%-14% even if cameras were perfectly placed exactly where an accident would have occurred.
Andrew Mather, Kent

Agree (15) | Disagree (11)
+4

This pretty much confirms the Andrew Mather analysis of Stats19. Funny how their surnames should be so close and they both come to the same conclusions.
Duncan MacKillop. No surprise - No accident.

Agree (10) | Disagree (8)
+2

I don't think there's hard evidence for that Rod, otherwise speed cameras wouldn't catch any speeders, but we know they catch many thousands. Speed 'surfing' between fixed cameras is a well known phenomenon too. Most accidents happen when vehicles are travelling under the posted speed limit anyway.
Paul Biggs, Tamworth

Agree (14) | Disagree (12)
+2

It seems to me that the existence of speed cameras does influence general compliance with limits across the whole network as well as the immediate vicinity of the camera sites themselves. Hence casualty trends at speed camera sites and any general trend in casualties would seem to both an output of having a speed camera regime that influences both site-specific and general compliance. Any similarity in trend is therefore not evidence of failure.
Rod King, 20's Plenty for Us

Agree (15) | Disagree (9)
+6

Yes Peter. Any comprehensive analysis of 'before' and 'after' data should include 'contributory factors' and the EuroNCAP safety ratings for the vehicles involved. I remember reading on the EuroNCAP website that each safety star awarded equates to a 12% reduction in KSI - hence the 5 star maximum could equate to a 60% reduction in KSI.
Paul Biggs, Tamworth

Agree (10) | Disagree (10)
0

Nice to have what me and many others have been saying for over a decade confirmed by someone objective and qualified. Prof Mayer also has an aptly titled viewpoint article in LTT: 'If we want safer roads, we need higher standards in casualty data analysis.'
Paul Biggs, Tamworth

Agree (11) | Disagree (14)
-3

The story is academic and could be about any road safety campaign or strategy. As the Tory politician George canning said "You can make statistics say anything you like except the truth." Other influencing factors need to be included into the mix and even then one will never know what affected what.
Peter Westminster City

Agree (10) | Disagree (15)
-5