NRS Ltd handed responsibility for governance to PAMCo from January 2016.

The disruption caused by Covid from March 2020 onwards made it necessary to evolve the PAMCo methodology. Even before Covid struck there were plans to take a small proportion of interviews online in 2021 (as a follow-up to improve response) and to test taking a larger portion of interviews online via an ‘online first’ mixed methodology.

As a response to restrictions on face-to-face interviewing following the onset of the Covid pandemic in March 2020, plans to test ‘online first’ were accelerated. What followed was an intensive period of testing in 2020. 12 separate tests were conducted, culminating in a large ‘live’ test which started in September 2020 and was completed, post lockdown, in April and May 2021.

The new method was approved by the PAMCo Board, with the advice of the PAMCo Technical Group, as “the best possible in the circumstances”. Now restrictions on face-to-face interviewing have been lifted, the option of a face-to-face interview is also available in many cases. However the new method remains primarily online self-completion.

There are major differences in both interview mode and sample. The interview mode is self-completion, completed either online or using a paper self-completion questionnaire. Previously the interview was conducted in-home face-to-face by an Ipsos interviewer.

Around 60% of interviews are completed online. The remainder are CAPI interviews conducted by an interviewer or paper self-completion questionnaires. There are different rules as to who can be interviewed than was the case with pre-Covid PAMCo. Previously specific households were sampled, and then specific individuals aged 15+ within the household, with no substitutions allowed. For the new method specific households are sampled in a similar way, but the invitation is for up to two household members to take part, without restriction as to who they are, other than that they are aged 15 or over.

Response rates are different to the previous method, both overall and by particular demographics. This has implications for the readership data obtained and is discussed more in response to the next question.

The effects for the data collected are as follows:  

• As participants go through the questionnaire themselves, they are more likely to identify the full range of titles they read, including infrequently read titles. 
• However, the self-completion method is more prone to title confusion, particularly for certain types of publication, especially those with generic sounding names.  
• It is not possible to collect all data in full using a paper questionnaire.  For instance, it is not possible to ask source of copy and attitude statement data for individual brands.  This is collected generically for types of publication and then allocated to titles with average issue readership claims.   
• Inevitably, not all questions will be answered in the paper questionnaire.  
• It is particularly challenging to collect the information required for allocating social grade via a self-completion questionnaire.  Detailed social grade classification requires a sequence of questions which it is much easier for interviewers to administer, and interviewers receive special training how to elicit the information required.  Inevitably there will be some gaps in the information collected by the new method either because participants are unwilling to give it, or because it is not possible to anticipate follow-up questions required.  Full social grade information is not available for about 13% of the weighted population.  This gap has been filled by a modelling process based on other information given in the interview.    
• A considerable focus of the development testing was in improving questionnaire design to minimise the impact of the issues above, particularly in respect of the paper self-completion questionnaire.  

Although over 90% of the population use the internet, not everyone is able to, and there are also some users who would not be comfortable completing a detailed questionnaire online.

It is important that households are sampled in such a way as to ensure everyone has the chance to be selected and the sample is therefore representative of the whole population. Using the postal address file to select addresses is the best way of doing this. There isn’t an equivalent comprehensive database of email addresses to use for sampling purposes. Where email databases exist they will be incomplete, get out-of-date quickly and be subject to GDPR restrictions.

It is not possible to carry out the field stage (i.e., the in-person visits to non-responsive households) if lockdown restrictions are in place. For this reason if restrictions are imposed fieldwork must be suspended in the affected area.

If the restrictions are in a very specific area this should not interrupt reporting and the sample in that area will be re-scheduled for when restrictions are lifted. More widespread restrictions are likely to affect the reporting of fresh print data. The digital audience figures are unaffected.

No. The differences in sample and method mean that it is not possible to make a like-for-like comparison or look at trends between the two.

The new method makes it more challenging to get those in DE social grades to take part, and has brought other changes to the sample profile before weighting is applied, including an increased proportion of C1s.

It is also worth noting the challenge of collecting the information required for allocating social grade via a self-completion questionnaire. Interviewers are trained to elicit the detailed information required. Full social grade information is not available for about 11% of the sample and 13% of the weighted population. A predictive CHAID model has been used to assign social grade where it is missing. The CHAID model is informed by other information given in the interview such as Chief Income Earner income, working status, terminal education age, household tenure etc.

Long before the Covid situation PAMCo had decided to move to a two-year reporting database based on a sample of 44,000 from January 2021 onwards.  This replaces the previous one-year database of 36,000.

• To save on fieldwork costs as 22,000 interviews rather than 36,000 would be required per year.  Fieldwork accounts for the majority of PAMCO’s costs.
• Moving to 44,000 sample two-year database gives a larger reporting sample for small brands relative to the previous 36,000 sample one year database.
• The change was in the context of a shifting focus, with a changing balance between print and digital readership in many cases.

PAMCo changed methodology to ‘online first’ in Q.4 2020. The decision to change the methodology arose from a need to adapt to Covid times (which meant it was no longer possible to rely on face-to-face interviewing in home) and to ensure that the measurement solution for print remains future facing, flexible and efficient as digital growth continues.

As expected, when there is a major change in methodology there will also be a step change in the readership figures. The ‘online-first’ approach brought considerable change to both the sample and interviewing mode compared to the previous in-home face-to-face method. More information about the methodology change can be found in the FAQs (13-15).

As a generalisation, most print titles measured by PAMCo recorded higher print readerships using the new ‘online first’ method, despite declining circulations. PAMCo’s digital figures have not been impacted. The change to the print figures was an anticipated outcome from the methodological changes and given the research challenges faced one that was supported by both the PAMCo Board and Technical Group as best possible readership figures in the circumstances.

Higher print readership has resulted in higher readers-per-copy (RPC) figures for most paid-for titles, however the degree of increase in RPC for a number of news and magazine titles has raised some questions. For more information on RPC and reasons for variation in RPC click here.

The PAMCo Technical Group is currently investigating several options to see if improvements to the measurement can be made. These include a review of survey prompts and questions for all titles, to emphasise further that participants are being asked about PRINTED copies. A test of changes to the readership question for weekday newspapers is also being considered. An increasing proportion of interviewer administered interviews (as an alternative to self-completion) may also help.