Statistical significance

All estimates based on a sample are subject to what is known as ‘sample variation’. However tightly controlled, the answers from one sample of people will differ somewhat from another sample of people drawn in exactly the same way.

There are various statistical tests which can be used to assess whether changes in PAMCo data from period to period (or between publications) are actually real changes rather than the result of sample variation. For any estimate taken from PAMCo it is possible to work out a ‘confidence limit’. This is a way of expressing how much variation we might expect. For instance:

  • Magazine X has estimated readership of 1,900,000 adults (3.5% of adults)
  • The 95% confidence limits for this estimate are +/- 120,000
  • This means we can be 95% certain that the true estimate for Magazine X is between 2,020,000 and 1,780,000 (3.3% to 3.7% of adults)

PAMCo has created two easy-to-use spreadsheets to:

  • Find out the confidence limits for a particular estimate, be it readership or any other estimate on the survey.
  • Find out whether the difference between two estimates is statistically significant

All users need to do is to enter the estimates they wish to test into the boxes shown on the spreadsheet, along with the population and sample size on which the estimates are based.