PAMCo Bridge FAQs

PAMCo Bridge FAQs

SECTION 1: METHOD

1. Why has the PAMCo method changed?

The disruption caused by Covid from March 2020 onwards has made it necessary to evolve the PAMCo methodology.  Even before Covid struck there were plans to take a small proportion of interviews online in 2021 (as a follow-up to improve response) and to test taking a larger portion of interviews online via an ‘online first’ mixed methodology.   

Covid restrictions meant that from March 2020 onwards it was not possible to continue conducting the face-to-face in-home interviews to collect print readership, demographic and lifestyle information.  The pre-Covid PAMCo interview involved interviewers and participants both looking at the same tablet screen.  There are many visual prompts to look at, so the interview is not suitable for doorstep completion. 

The plans to test ‘online first’ were accelerated. What followed was an intensive period of testing in 2020.  12 separate tests were conducted, culminating in a large ‘live’ test which started in September 2020 and was completed, post lockdown, in April and May 2021. 

The new method has been approved by the PAMCo Board, with the advice of the PAMCo Technical Group, as “the best possible in the circumstances” given that it will not be possible to return to face-to-face in-home interviewing at scale for the foreseeable future.  

The significant changes in method and sample have inevitably changed the estimates obtained, and more information can be found about what to expect in the following FAQs.  Given the level of change, it is not possible to make direct comparisons with pre-Covid PAMCo estimates. 

At the same time there have also been major changes to the digital audience estimates, which PAMCo sources from the UKOM approved supplier and integrates with the print readership estimates by means of a data fusion.  This is because UKOM changed approved supplier from January 2021 to Ipsos iris, and the estimates are based on a very different method to that of the previous supplier.  More information can be found in the following FAQs. 

2. What is PAMCo’s new method?

PAMCO’s new method for collecting the print readership estimates is known as ‘online first’.  It is a two-phase method: 

  • It starts with a postal phase – sampled addresses are mailed and up to two household members are invited to complete an online (or paper) self-completion questionnaire   
  • This is followed by a field stage when non-responsive households are visited in-person by Ipsos ‘facilitators’ to encourage completion online or paper.  Participants can also be offered a tablet to complete the interview on, if that suits them better, or a video interview  

Tests showed in-person visits to non-responsive households are essential to obtain a representative sample.  Around half the completed questionnaires come from this field phase, having achieved half at the postal phase.  

Participant incentives are offered to encourage completion (£20 per completed questionnaire). 

Around three-quarters of the self-completion questionnaires are completed online.  It is important to offer an alternative for those who are not willing or able to complete online, particularly as this group will contain a disproportionate number of readers for certain types of publication. The alternative offered is a paper self-completion questionnaire. About a quarter of the sample opt to complete the interview using the paper self-completion questionnaire. 

For a one page summary of the new methodology please click here

3. What are the main differences between the new method and the pre-Covid method?

There are major differences in both interview mode and sample. 

The interview mode is self-completion, completed either online or using a paper self-completion questionnaire.  Previously the interview was conducted in-home face-to-face by an Ipsos interviewer. 

Around three-quarters of interviews are completed online.  Having an online questionnaire which can be completed on any device gives PAMCo scope to develop the survey further going forward.   

There are different rules as to who can be interviewed than was the case with pre-Covid PAMCo. Previously specific households were sampled, and then specific individuals aged 15+ within the household, with no substitutions allowed. For the new method specific households are sampled in a similar way, but the invitation is for up to two household members to take part, without restriction as to who they are, other than that they are aged 15 or over. A procedure to select a specific person within the household for interview is still, however, used at the field stage.

Response rates are different to the previous method, both overall and by particular demographics.  This has implications for the readership data obtained and is discussed more in response to the next question. 

4. What are the pros and cons of a self-completion methodology?

The effects for the sample are as follows: 

  • It requires more motivation for the participant to complete the questionnaire themselves (even if they have promised they will) and this affects response rates 
  • Not everyone feels confident/comfortable/willing to complete a self-completion questionnaire, whether online or on paper, and we can see this reflected in the sample.  There is a tendency to over-represent better educated and more literate participants, and conversely to under-represent the less literate.  
  • The online option improves representation of the under 55s and those living in certain high-status areas relative to the previous PAMCo method.  However, the new method is not as good at representing those in C2DE social grades and who have not been educated beyond the age of 16.  Adding the option of a face-to-face interview from October 2021 onwards may help address this. 

The effects for the data collected are as follows: 

  • As participants go through the questionnaire themselves, they are more likely to identify the full range of titles they read, including infrequently read titles. 
  • However, the self-completion method is more prone to title confusion, particularly for certain types of publication, especially those with generic sounding names. 
  • It is not possible to collect all data in full using a paper questionnaire.  For instance, it is not possible to ask source of copy and attitude statement data for individual brands.  This is collected generically for types of publication and then allocated to titles with average issue readership claims.  
  • Inevitably, not all questions will be answered in the paper questionnaire. 
  • It is particularly challenging to collect the information required for allocating social grade via a self-completion questionnaire.  Detailed social grade classification requires a sequence of questions which it is much easier for interviewers to administer, and interviewers receive special training how to elicit the information required.  Inevitably there will be some gaps in the information collected by the new method either because participants are unwilling to give it, or because it is not possible to anticipate follow-up questions required.  Full social grade information is not available for about 13% of the weighted population.  This gap has been filled by a modelling process based on other information given in the interview.  It is hoped adding the option of face-to-face interviews back into the methodology mix from October 2021 will reduce the gap to be filled. 
  • A considerable focus of the development testing was in improving questionnaire design to minimise the impact of the issues above, particularly in respect of the paper self-completion questionnaire. 

5. Why has this method been adopted despite the drawbacks in respect of sample and completion?

Given that it is not possible to conduct the PAMCo survey face-to-face and in-home at scale for the foreseeable future, the PAMCo Board, advised by the PAMCo Technical Group, have judged the new method to be the ‘best possible in the circumstances’.  

Although it is not perfect, particularly in respect of sample representation there is no alternative available which would do better.  Response rates to telephone surveys are very low, for instance, and would not deliver a better sample.   

PAMCo continues to make a very considerable investment in sample quality in order to achieve the most representative sample possible by this method.  This is not a sample recruited online or from an ‘opt-in’ online panel.  The sample is a random sample of households designed to represent all adults in Great Britain. Every effort is made to maximise participation via the use of participant incentives and field stage of visiting non-responsive households in person.  

6. Why have a paper self-completion questionnaire? Wouldn’t it be better to use only online self-completion?

Although over 90% of the population use the internet, not everyone is able to, and there are also some users who would not be comfortable completing a detailed questionnaire online.  Without an alternative to the online self-completion questionnaire, it would not be possible to include these people in the survey.  This would mean excluding a proportion of print readers, and for certain types of publication a disproportionately high proportion of their print readers e.g., TV magazines and some newsbrands.   

Around a quarter of the participants complete using the paper self-completion.  In practice this will include some who could complete online if they wished but prefer the paper option. 

7. What about video interviewing?

Interviewers are able to offer the option of a video interview via Microsoft Teams, with an interviewer taking the participant through the online questionnaire.  However, less than 1% of interviews are taken in this way. 

Testing prior to implementation indicated take-up was likely to be low.  Video interviewing is best suited to participants who are not technologically confident, but paradoxically requires a degree of ‘know-how’ and confidence to set-up on the appropriate platform.  For those confident with online video platforms, the online self-completion questionnaire is a more efficient (and less costly) option. 

8. Why mail invitations to take part to postal addresses rather than using email addresses?

It is important that households are sampled in such a way as to ensure everyone has the chance to be selected and the sample is therefore representative of the whole population.  Using the postal address file to select addresses is the best way of doing this.  There isn’t an equivalent comprehensive database of email addresses to use for sampling purposes.  Where email databases exist they will be incomplete, get out-of-date quickly and be subject to GDPR restrictions. 

9. How do you encourage participants to take part? What are the incentives?

Participants are offered a £20 incentive for completing a questionnaire.  This is given in the form of an Amazon or Love2Shop voucher (the latter can be used in numerous high street stores). Alternatively, the participant can opt to donate the sum to a choice of three charities. 

The previous PAMCo method did offer incentives in London and the South-East where it was more challenging to get response, but not elsewhere in the country.  With the switch to a self-completion methodology, it is necessary to offer incentives throughout the country. 

10. What are the response rates?

Response rates are still settling but around 35% of households take part, and an average of 1.2 adults per household.  It is hoped there can be further improvements particularly when the option of a face-to-face interview is added from October 2021 onwards. 

The new method response rates are not directly comparable to the previous PAMCo method, due to the change in sampling, but for reference the previous method achieved an interview in around 50% of sampled households. 

11. Will PAMCO go back to face-to-face interviewing as Covid recedes?

From October 2021 the plan, subject to any remaining or new Covid restrictions, is that when Ipsos visit non-responsive households in person an in-home interview can be offered if the participant prefers.  We hope this will encourage those who are less confident about completing the questionnaire themselves to take part. If this proves to be the case it will help address some of the skews in the sample and potentially improve response rates. 

The take-up of this option is entirely dependent on public sentiment and participant willingness to invite an interviewer into their home, as the interview is not suitable for doorstep completion.  In practice the proportion of interviews conducted in this way is unlikely to be above 10-15%. 

Longer term what happens as Covid’s implications are controlled is also dependent on public attitudes.  At this stage it seems unlikely that the survey will return to a primarily face-to-face in-home methodology, but one of the benefits of having a ‘mixed methodology’ is that this can be reviewed when appropriate. 

12. What happens if there is another lockdown?

It is not possible to carry out the field stage (i.e., the in-person visits to non-responsive households) if lockdown restrictions are in place.  For this reason if restrictions are imposed fieldwork must be suspended in the affected area.     

If the restrictions are in a very specific area this should not interrupt reporting and the sample in that area will be re-scheduled for when restrictions are lifted. More widespread restrictions are likely to affect the reporting of fresh print data.  The digital audience estimates are unaffected. 

13. Given that the new method is primarily online why doesn’t it cost less than the pre-Covid face-to-face interview method?

Online surveys can be relatively inexpensive compared to other types of survey, particularly if the sample comes from an online panel composed of participants who have chosen to be panel members.  PAMCo is not sourcing sample from an online panel.  Although the PAMCo interview is now completed largely online, the sample is recruited offline to ensure it is as representative as possible, which adds to cost.   

The two key elements of cost are having ‘facilitators’ from Ipsos visit non-responsive households and offering all participants an incentive of £20 for completion, whereas the previous method only offered incentives in London and the South-East.   

PAMCo did test whether it was possible to conduct the survey without the field stage, which would make a considerable difference to costs.  Without the field stage, the sample is not sufficiently representative, as it is too ‘self-selecting’ and skewed to readers, producing readership estimates which are not credible.  The field stage generates around half the completed questionnaires and considerably improves sample quality. 

SECTION 2: THE SURVEY ESTIMATES

1. What differences in the print readership estimates can we expect?

The new method delivers different estimates to the pre-Covid method because of the substantial change to method and sample. Although there will also be changes in actual reading behaviour since the pre-Covid data were last collected in March 2020, the change in method will have a greater influence on the differences observed. It is not comparing like-for-like measurement techniques and therefore not appropriate to make direct comparisons for commercial purposes.

Some general patterns noted are as follows:

• Claims to be a Read Past Year print reader of specific titles are generally higher using the new method, particularly for magazines, with more participants are claiming to read at least one magazine or newspaper in the past year.
• As a sector, magazines are picking up around 30% more Average Issue Readership (AIR), claims mostly due to a bigger percentage of the population claiming to be an average issue reader of at least one magazine, and then a small increase in the average number of titles claimed by each reader from 2.1 to 2.4.
• Overall average issue readership of newspapers looks more in line with the pre-Covid method at a top line level but there are a number of notable changes. In particular, a higher level of AIR claims for Saturday newspapers and some Sunday newspapers is being picked up. Not surprisingly the number of claims for free weekday papers is down, though this is likely to change as commuting increases.

The above points are generalisations; differences vary by type of publication and individual title

2. Why are we seeing these differences? Why has print readership generally gone up when circulation has gone down?

There are three main reasons for changes in the data relative to the pre-Covid:
• Real changes in reading since March 2020
• Differences in the sample used to collect the data. In particular, the self-completion methodology appeals more to literate better-educated participants who are more likely to be readers. It is more difficult to get less-literate participants and those in the DE social grades to participate. Adding the option of a face-to-face interview from October 2021 onwards may help address this
• Differences in the interview mode. The self-completion mode has pros and cons. On the plus side it means participants are less likely to miss claiming titles they actually read. This is one reason why Read Past Year estimates have increased, with more infrequent readers being identified. On the other hand, the self-completion method can be more prone to confusion without interviewer guidance.

As a generalisation one would expect readership to have fallen since March 2020 because circulation has.
However, as mentioned above, the main reason for differences is the change in method rather than changes in real behaviour. Readership levels are generally higher because of the change in sample and method, as described above. We are starting from a different level. For this reason, it is not possible to make a direct comparison with the PAMCo data collected before March 2020.

3. What about the readership estimates for free distribution titles, which are showing decreases?

Titles for which circulation and readership is largely based on free distribution were particularly affected by the Covid lockdowns and restrictions. The free distribution titles PAMCo reports on are Metro, The Evening Standard and Time Out.

Readership estimates collected in the initial batch of new method (i.e. online first) data between October 2020 and June 2021 are lower for such titles than those collected by the pre-Covid methodology. This is not surprising, as during this period the titles concerned had reduced circulation, due to restrictions to the commute and work from home guidance.

Again, it is not possible to make a direct ‘like-for-like’ comparison with the pre-Covid readership estimates, as the two methodologies are so different.

We will be monitoring commuter titles throughout Bridge releases to better understand the effects of the new methodology and possible changes in readership patterns in the quarterly data.

4. What about the readership profiles? Will the new sample change my title’s profile?

As noted to an earlier question, there are some changes in the sample. For instance, the pre-Covid method sample was too skewed towards those aged 55 or over, whereas with the new method sample the proportion of over 55s in the sample is more in line with the proportion of over 55s in the population. The new method also gets interviews with a lower proportion of C2DEs relative to population than the pre-Covid method.

It is important to stress that picking up fewer 55+s or C2DEs in the sample does not disadvantage titles that are aimed specifically at over 55s or C2DEs, either in respect of their overall estimates or their profile data. The sample is weighted to population targets to avoid this. The weighting is designed to correct any skews in the sample towards (or against) specific demographic groups and ensure as far as possible a ‘level playing field’.

Of course, it is not possible to weight every aspect of the data, and inevitably there will be some shifts to profile data as a result of the changes to sample and method for some titles. Early analysis indicates there is no fundamental change to the profiles (e.g. predominantly ABC1 titles remain ABC1 titles and so on) but individual title profiles will continue to be monitored as sample sizes build.

5. What sort of titles are prone to title confusion?

There are a number of reasons why title confusion and over-claim can happen in a readership survey, for instance:
• Titles with very similar names
• Participant uncertainty e.g. “I know I’ve read a TV guide, not sure which one”
• Participant desire to make a positive claim if they have read a title not shown on the list, claiming a different title with a similar or generic sounding name
• Participant desire to make a positive claim to read more often than they do because they strongly identify with a particular title and want to be seen as a reader
• Strong brand names which may also be associated with other content/contact points besides the publication itself
• Confusion between print and digital reading

No readership survey based on participant recall will be completely free of title confusion. The PAMCo Technical Group have identified a handful of specific instances where there may be more opportunity for title confusion and over-claim than was the case for the pre-Covid method. More detail with regard to the titles concerned can be found in a separate note here, which also details the changes being made to reduce the risk of confusion going forward

6. Can I compare the pre-Covid data with data from the new method?

No. The differences in sample and method mean that it is not possible to make a like-for-like comparison or look at trends between the two.

7. Do the skews towards more literate participants mean we should be cautious about the print readership data?

It is important to be aware of the skews in the sample in order to understand why PAMCo data have changed and are not directly comparable with what has gone before.

Obviously, it is preferable to minimise skews in the sample and maximise response rates. Every effort has been made to do this, and further measures are in hand, such as offering the option of a face-to-face interview from October 2021 onwards.

Given the reality that it will not be possible to return to face-to-face interviewing at scale for the foreseeable future, the PAMCo Board, advised by the PAMCo Technical Group, have judged the new method as the ‘best possible in the circumstances’.

8. What are the implications for social grade?

The new method under-represents C2DEs in the sample, particularly DEs. This will not be fully corrected by weighting.

In addition, those C2DEs interviewed appear to be more likely to be readers than we would expect, particularly for magazines.

It is hoped that re-introducing the option of a face-to-face interview will make the interview more attractive to those who are less willing or confident to fill in a self-completion questionnaire. A review of survey materials and incentives is also underway to make the survey as attractive as possible to the C2DE participants.

It is challenging to collect the information required for allocating social grade via a self-completion questionnaire. Interviewers are trained to elicit the detailed information required. Full social grade information is not available for about 11% of the sample and 13% of the weighted population. A predictive CHAID model has been used to assign social grade where it is missing. The CHAID model is informed by other information given in the interview such as Chief Income Earner income, working status, terminal education age, household tenure etc.

SECTION 3: THE DIGITAL ESTIMATES AND TOTAL BRAND REACH

1. Why are the digital audience estimates and Total Brand Reach so different to the last PAMCo release?

The digital audience estimates have changed as well as the print readership estimates because of the change in the UKOM approved supplier.  

PAMCo uses the digital estimates provided by the UKOM approved supplier and uses a data fusion to integrate them with print readership estimates in order to give publishers a measure of total cross-platform brand reach.  Ipsos iris became the UKOM approved supplier in January 2021.   

As Ipsos iris uses a very different methodology to Comscore there are considerable differences in the estimates provided.  PAMCo has also had to develop a new data fusion to integrate the estimates.  More details are provided about this in the answers to later questions.  

2. What are the differences between Ipsos iris and the previous supplier?

Ipsos have summarised the main differences in method as follows: 

  • Universe definition: adults aged 15+ (versus 6+ desktop & 13+ smartphone/tablet)   
  • Single source panel of 10,000+ people and 25,000+ devices (versus four different panels)  
  • Differences in structure and categorisation of websites 
  • Traffic allocation based on tag keyword allocation (versus URL classification) 
  • Differences in panel weights and balancing  
  • Differences in reporting embedded browser traffic 
  • Differences in sample sizes  

For more information contact Jing Yeow on jing.yeow@ipsos.com at Ipsos Mori

3. Why is there less duplication of reading between print and digital platforms – especially for magazines?

A new data fusion process has had to be developed to integrate the iris estimates. 

For newsbrands, duplication of reading in both print and online is similar in relative terms to the levels see in the PAMCo fusions with Comscore (though the absolutes have changed because of the differences in size of the digital audience estimates).   

For magazine brands generally less duplication is observed in relative terms.  This is in part because Ipsos are not using duplication targets from the PAMCo single-source panel which measured PAMCO respondents’ reading across their digital devices.  The single-source data were collected in 2017/2018, and so now are somewhat aged, especially given the disruption to the market in 2020.  

If required there is an opportunity to collect and apply fresh duplication target data.  This would be done by asking iris panellists to complete a PAMCo survey to measure their print reading behaviour and observe the duplication with their passively measured digital reading behaviour. 

However, it is important to note that the main factors driving Total Brand Reach are the size of the digital audience and, usually to a lesser extent, the size of the print audience.  The level of duplication makes much less difference.  To put this in context, if the print and digital duplications were increased in line with the levels observed in the fusions with Comscore Total Brand Reach would decrease in the region of 1-6%, depending on brand.

SECTION 4: REPORTING

1. When will the new data first be reported?

The first data release to include data from the new method will be on 20th October 2021, embargoed until 29th October.  This will include print data collected between July 2019 and June 2021, with a data fusion to the July 2021 Ipsos iris estimates.  

The print data will span a two-year period, in line with PAMCo’s planned move to a two-year reporting database from 2021.  It will combine sample obtained using the pre-Covid face-to-face method between July 2019 and March 2020 with sample obtained using the new method between September 2020 and June 2021.    

The next two releases after that, shown in the table below, will also include a mix of pre-Covid and new method sample, with the latter gradually replacing the former.  

To mark the transitional nature of these releases they will be labelled as ‘PAMCo Bridge’. 

The first release based entirely on new method data will be in June 2020, based on the period September 2020 to March 2022.  From this point trending can begin. 

 

Data Release Date of release Print data use Digital data use
PAMCo Bridge 3 2021 October 2021 Jul ’19 – Jun ’20 Ipsos iris July ‘21
PAMCo Bridge 4 2021 December 2021 Oct’19 – Sep ‘21 Ipsos iris Sept ’21
PAMCo Bridge 1 2022 March 2022 Jan ‘20 – Dec ‘21 Ipsos iris Nov ’21
PAMCo 2 2022 June 2022 Sep ‘20 – Mar ‘22 Ipsos iris Mar’ 22

2. Why is PAMCo combining data collected pre-Covid with data from the new method, especially as the two methods are so different?

In an ideal world it would be technically preferable not to combine pre-Covid sample with new method sample given that the two methods are so different.  However, as PAMCo now publishes on a two-year database it would not be possible to collect enough sample for a standalone release reporting on all titles until mid-2022. 

A gap of over two years since there was last fresh print data in the market is not a viable proposition.  In this context, the PAMCo Board have decided that it is better to put fresh data into the market as soon as possible by combining pre-Covid sample with new method sample.  

3. What will be the effect of combining the new method data with the pre-Covid data in the ‘Bridge’ releases?

The effect of combining the two datasets will be to soften the impact of the changes from the new method data.  These changes will become more apparent with each successive PAMCo Bridge release, as an increasing proportion of the sample is based on the new method.   

For instance, if the new method indicates a hypothetical magazine has an AIR +34% higher than the pre-Covid method, an increase of +15% will be noted in the first Bridge release, as the estimates are only partly based on the new method.  If the new method continues to record a similar difference in the level of AIR for that magazine, an increase of +22% will be noted in the second Bridge release and +29% in the third.  When finally in June 2022 there is a data release based solely on the new method the full increase of 34% will be apparent.   

If a Saturday newspaper had a +18% difference, this would show as +8% in the first Bridge release, +12% in the second, +15% in the third and finally +18% when the data are solely based on the new method.  Again, this assumes nothing else changes in the estimates the new method is delivering. 

The same is also true for negative differences in readership. For instance, the free weekday newspapers are not surprisingly showing negative differences relative to the pre-Covid data. This will in part be in changes due to real behaviour in the period to June 2020, though as with all titles it is not possible to make like-for-like comparisons due to the changes in method and sample.  The full extent of the negative differences will not be apparent in the first Bridge release and it is likely that print readership of these titles will decrease over successive Bridge releases, even if commuting journeys are increasing across this period. 

4. Why did PAMCo decide to move to a two-year reporting database?

Long before the Covid situation PAMCo had decided to move to a two-year reporting database based on a sample of 44,000 from January 2021 onwards.  This replaces the previous one-year database of 36,000. 

The reasons for doing so were: 

  • To save on fieldwork costs as 22,000 interviews rather than 36,000 would be required per year.  Fieldwork accounts for the majority of PAMCO’s costs. 
  • Moving to 44,000 sample two-year database gives a larger reporting sample for small brands relative to the previous 36,000 sample one year database.  There is the option for larger brands to report on a one-year database in due course, in order to provide the most recent data available.  
  • The change was in the context of a shifting focus, with a changing balance between print and digital readership in many cases.    

Skip to content