Fieldwork and response rates
The results on the following page are for the data collected using the new survey methodology between 5 September 2020 and 27 June 2021 as part of the PAMCo Bridge 3 2021 release.
Fieldwork for the new methodology consists of a postal stage in which invitations are sent by post to take part in a self-completion survey, either online or by post, followed by a field stage in which interviewers call at all addresses from which neither a complete survey nor a refusal has been obtained. At the field stage, and prior to the addition of in-home interviewing in October 2021, interviewers acted mainly as facilitators to invite potential participants to complete the survey, in one of the following ways:
- Online self-completion on a smartphone, tablet or pc/laptop of their own and in their own time
- Online self-completion on a tablet lent to them by the interviewer – again, in their own time
- Through an MS Teams interview in which the interviewer shared their screen and completed the survey from a remote location
- Self-completion with a paper questionnaire
The response rate shows the proportion of the target sample who were actually interviewed. It is an indication of quality and how representative the sample is.
At the postal stage, we can state a household response rate (the number of households from which any complete survey was received as a percentage of the households invited (minus those subsequently found to be ineligible at the field stage). Although there can be up to two surveys completed per household, we cannot translate this into a response rate among all aged 15+ as we do not use a random method for the selection of individuals.
The field stage PAMCo response rate can be a complete response rate because it reflects both of the following:
- The proportion of the sample that is successfully contacted in the first place
- The proportion of identified participants who complete an interview
We believe that the single most important factor in maximising the response rate is the use of face-to-face interviewers, who establish a rapport with participants and encourage their cooperation.
A great deal of effort goes into maximising response rates, e.g.:
- Two reminder waves after the first wave of postal invitations
- Fieldwork structured over different days and times of day (7 days a week, including evenings)
- Numerous call-backs by interviewers if necessary
- Explanatory letters and information about the survey available
- Incentives for each completed survey to the value of £20
The following table summarises response at address level across all stages for the sample surveyed between September 2020 and June 2021 using the new methodology.
The household / address response rate was 37% of all eligible addresses issued.
Addresses were classed as ineligible if they were non-residential, not a main residence, demolished, derelict or not yet built or were vacant / empty. No substitute addresses were used in these circumstances.
Households ‘unable to participate’ included those where illness, disability, language difficulties or being away from home through the full fieldwork period prevented participation.
|Response Rate Information at address level||No. of addresses||% of total eligible addresses||% of eligible addresses issued at each stage|
|Postal Stage||Field stage|
|Total addresses issued at postal stage||17530|
|Addresses in postal stage found to be ineligible at field stage||928|
|Eligible addresses issued at postal stage||16602||100%|
|Addresses issued at field stage (including 3806 not issued at postal stage)||17918|
|Addresses found to be ineligible at field stage||1221|
|Eligible addresses issued at field stage||16697||100%|
|– Refusals to office (from advance letter)||373|
|– Refusals in field||6474|
|– No contact||2940|
|– Household unable to participate||257|
|– Agreed to participate but no interview||2567|
|Total addresses issued||21336|
|Total eligible addresses||20115||100%|
|Addresses with any complete at postal stage||3350||16.7%||20.2%|
|Addresses with any complete at field stage||4086||20.3%||24.5%|
|Addresses with any complete at either stage||7436||37.0%|
At the postal stage, an average of 1.37 individuals per address completed the survey, while at the field stage an average of 1.19 per address participated.
Since random person selection was not used in all points during part of the data collection period between October 2020 and mid May 2021, we cannot yet report a complete response rate that takes account of contact rate and response among selected persons. However, we can report on the number of individuals who agreed to take part and the proportion who went on to complete the survey.
The table below shows that almost 60% of those who agreed to take part actually went on to complete the survey, with the paper questionnaire more likely to be completed than the online survey.
|Field Stage – Summary of completion rate after agreement to participate||No. of addresses||% completion rate|
|Addresses in which any agreement to participate||6653|
|Persons agreeing to participate||8208||100.0%|
|– Online SCQ||6802|
|– Paper SCQ||1406|
|Persons interview achieved||4905||59.8%|
|– Online SCQ||3813||56.1%|
|– Paper SCQ||1092||77.7%|
For future data releases we will be able to report the field stage response rate as a complete response rate reflecting both the proportion of the sample that is successfully contacted in the first place and the proportion of randomly selected participants who complete an interview.
Any questionnaire failing to come up to the standard required, for reasons of omissions or inconsistencies, was rejected. Just under 2% of the total sample could not be included in the final reporting sample due to having no age band information, other types of poor completion or there being more than two surveys completed at the same address. The following table summarises the usability of the completed interviews.
|Postal stage||Field stage||Total|
|Usable sample after edits||4450||4817||9267|