Badman statistics

[Ali Admin]

A professional statistician has contacted AHEd with concerns about the use of statistics in Badman's report.

His verdict is nothing short of damning.

Yet another nail in the coffin.

[Ali Admin]

Reproducing the statistician's verdict here in full for the search engines.

Statistician's Comment:

From W. Wallace

Mr Wallace has worked in local government as a statistician and also as a university statistics lecturer and research fellow.

Dear [AHEd]

Quite frankly I can't believe that you received the Appendix as an FOI request. I had been looking for a statistical / methodological appendix to the Badman report but had not found one. The methodology as shown does not stand up as plausible or acceptable statistically, apart from all the other issues concerning the precise information used i.e. abuse, disability services or known to SS for a variety of possibly unsubstantiated reasons.

The use of a sample median to gross up to a national value requires that all LA's have the same number of EHE children, which they do not.

It is not easily possible to estimate the statistical error introduced by doing this but suffice it to say that the standard error of such an estimate would be so large that it would not be worth using the statistic.

Also quoted is 477 registered EHE children known to the 25 LA's that responded out of the 90 asked to respond as part of the review. We have no way of knowing how representative the 25 are of the 150 LA's and this needs to be checked before any statistics can be quoted. Are we comparing like with like? If we were confident that these 25 reasonably reflected the total 150 then we might take the 477 and divide by the total number of EHE children in the 25 LA's. This would give us an estimate of the proportion of EHE children known to SS per LA. This is what they could have done but did not.

There are at least two main flaws to be noted:

1. The inappropriate use of one measurement instead of the target measure. Using 'known to SS' rather than recorded abuse (often termed an error of operationalisation).

2. Using a non-probability sample. No standard errors or confidence intervals can be computed. Only some qualitative value may be possibly obtained. This is error in sample selection.

I can say without any hesitation that the information on methodology casts grave doubt on any use of the results from the Badman Review.

What small amount of the Appendix that is shown is enough to bring a case of maladministration. Refusing to show any further details for whatever reason is not going to support their case one bit. It is very serious that statistical methods can be misused to try to support a case that does not exist.

W. Wallace BSc MSc MPhil FSS AFIMA