Following the Delphi surveys, 19 recommendations achieved overall consensus and were proposed by the steering group for inclusion; 8 recommendations only achieved consensus in one or more stakeholder groups and required discussion at the consensus meeting; and 6 recommendations did not reach consensus in any stakeholder group and were proposed for exclusion. Further details of the voting and decisions at the consensus meeting can be found in Fig. 1 and Supplementary Information. The 19 recommendations were ratified for inclusion during the consensus meeting (there was consensus to merge two of these recommendations). Of the eight recommendations that were individually discussed, two were voted in for inclusion as standalone recommendations; and there was consensus to merge two other recommendations with recommendations that had been ratified. Two recommendations were merged based on suggestions received on the initial draft of the manuscript. Further details are provided in Supplementary Information.
The flow chart illustrates the process, which culminated in 19 recommendations on how to address respondent burden associated with PRO assessment.
The final consensus statement provides 19 recommendations for consideration by anyone involved in designing and implementing PRO assessment strategies for healthcare research and clinical practice (Table 1). An elaboration describing each recommendation with supporting evidence is presented below. The recommendations are presented in accordance with the categories from the published review1, namely (1) rationale and schedule for PRO assessment; (2) measure selection; and (3) measure delivery.
Rationale and schedule for PRO assessment
Recommendation 1: involve patients, clinicians and other relevant stakeholders in the formulation of the PRO research question(s) or clinical objectives to ensure that they are important and relevant
Effective involvement of patients, clinicians and other relevant stakeholders in the formulation of PRO research question(s) and clinical objectives can help ensure the assessment of outcomes that are relevant and valued by all stakeholders in healthcare research and clinical practice25,26,27. Stakeholder involvement will vary depending on the context of PRO use. For example, patients, caregivers and clinicians can provide valuable perspectives both in healthcare research and in clinical practice settings, while early input from regulatory agencies may be particularly useful in clinical trials of investigational medicinal products. PRO assessments may be perceived as less burdensome if the research questions or clinical objectives are considered relevant and important by patients and other relevant stakeholders28,29.
Recommendation 2: consider the degree of burden that any data collection may impose on respondents and carefully balance this with the quantity and quality of data required
The rationale for collecting PRO data for healthcare research and clinical practice should be evidence-based and should demonstrate that the data collection justifies the burden and potential risks of data collection, such as the time required, emotional angst, distress or fatigue30.
Recommendation 3: ensure that patients, clinicians and other relevant stakeholders are involved in decisions about the PRO assessment schedule and the frequency of assessment
Consultation with patients, clinicians and other relevant stakeholders will help ensure that the PRO assessment schedule captures clinically relevant periods during treatment or clinical management25. The schedule of PRO assessments, including overall duration of assessment, will depend on the research or clinical objective; however, the potential respondent burden should be considered, while maximizing the collection of clinically relevant data31. The assessment schedule may not necessarily be tied to clinic visits; considerations for the mode of administration are described in Recommendation 16 (ref. 25).
The frequency of PRO administration should consider disease trajectory and balance this with respondent burden. Data should only be collected if they are essential to addressing the research objective or informing patient care. In a clinical practice setting, patients with a stable disease/condition may require less frequent PRO administration. Long-term monitoring may be burdensome and may lead to reduced PRO completion rates, but may be warranted in some instances (for example, for chronic disease monitoring or real-world evidence generation)32.
Consider the time points for assessing PRO measures within an allowable window and their relationship to clinical events (for example, treatment, clinical assessments and other assessments). Depending on the research or clinical objective, it may not be necessary to deliver all PRO measures at every time point28. A modular approach could be taken, in which different assessment frequencies are selected to reduce patient burden9. For example, more general quality-of-life aspects (for example, social or emotional well-being) may be assessed less frequently than the presence and severity of symptoms.
Measure selection
Recommendation 4: review the literature to identify relevant concept(s) of interest
To minimize burden, the concept(s) measured by the PRO measure should be relevant to the target population, disease setting and context of use (healthcare research or clinical practice). A literature review and/or surveys or qualitative work (Recommendation 5) can be conducted to identify concept(s) of interest.
Recommendation 5: qualitative and quantitative methods may be used to obtain input from patients and clinicians on selecting or developing PRO measures so that they are fit for purpose
Patient, clinician and other stakeholder input may be obtained using qualitative and/or quantitative methods, including interviews, focus groups and surveys28,33. Patient engagement and involvement is helpful to inform selection of PRO measures that capture meaningful outcomes, while reducing burden26,34. This may help avoid overly paternalistic approaches that are clinician- or researcher-driven35.
Recommendation 6: consider the complexity of the format of PRO measures and their instructions
PRO measures with greater complexity that require more cognitive effort to understand, such as those with complicated instructions, phrasing and reverse response options, may be more burdensome for respondents11,17,33. Discussions with patients from the target population may be used to explore these issues and ascertain the level of burden that may be associated with the PRO measures being considered33. The use of multiple measures with different formats may further increase complexity and should be avoided if possible.
Recommendation 7: consider the literacy level of respondents
Where possible, promote inclusion of individuals with all levels of reading, writing and problem-solving abilities11,35. Ensure that PRO content and training is easy to understand for respondents with different literacy levels and educational experience by conducting relevant readability assessments (for example, Flesch–Kincaid grade level or SMOG (simple measure of gobbledygook) index score)35. It is recommended that PRO items be at the reading level of 11–12 years of age or lower; however, this criterion should be contextualized to the intended target population and justified16.
Recommendation 8: ensure that the selected PRO measures are culturally and linguistically relevant for the target population
If PRO measures are translated into other languages, ensure that they have undergone linguistic validation with cognitive debriefing27,36. Linguistic validation is the testing of translated PRO measures with patients or lay individuals who are representative of the cultural group intending to use the measure to check understandability, interpretation and cultural relevance of the translation37.
Recommendation 9: consider the length of PRO measures and decide whether the use of a relatively longer measure is justified
Measure length is often considered as a contributing factor to PRO respondent burden; however, measure length should be balanced with patient and clinician input on what outcomes are most relevant to the population and context30. Relatively shorter measures may reduce respondent burden and increase patients’ willingness to complete forms38; however, brevity should not outweigh the utilization of PRO measures with appropriate measurement properties (reliability and validity) to assess outcomes that are relevant to key stakeholders, the research question(s)/PRO objectives and purpose of collection30,39,40.
There is evidence that the length of a PRO measure may not necessarily be associated with respondent burden16,18,30 and high response rates could be achieved with administration of relatively longer PRO measures if they are meaningful to respondents41. Furthermore, patients may prefer longer forms to shorter versions if they capture concepts that matter to them and can meaningfully inform care1,17,18. Ultimately, evaluations of PRO measure length should consider the context of use of the data, the views of those living with the condition and those responsible for using the data. Early patient involvement in selection of the measures is crucial (Recommendation 5).
Linked to the issue of PRO measure length is estimated completion time. The needs of the target population (for example, age, disease severity and comorbid conditions) and aspects of design (for example, mode and place of PRO measure administration), may impact overall completion times. Relevant stakeholder input should be sought on the anticipated completion time and its appropriateness in terms of the research or clinical context and the patient population. For instance, a PRO measure that generally requires more time to complete might not be suitable for use in a busy outpatient clinic. The same PRO measure might be appropriate for use if completed remotely, before clinical appointments. In terms of patient population, a PRO measure that requires less time to complete may be preferable for patients with osteoarthritis of the hand. In a research context it has been suggested that completion time of baseline PRO assessments should ideally be limited to 20 min and 10–15 min for subsequent assessments10,22.
Recommendation 10: if selecting more than one PRO measure, avoid overlapping constructs
The use of more than one PRO measure requires careful consideration to avoid duplication, overlap or redundancy of constructs9,42. The administration of several PRO measures may lead to respondent burden and a higher likelihood of missing data in those measures administered later, particularly if the constructs overlap. For research purposes, it is advisable that measures to support the primary and/or secondary outcomes are prioritized over those supporting exploratory outcomes.
Recommendation 11: consider the recall periods for measures, as longer timeframes may be burdensome for some respondents
When selecting PRO measures, it is important to consider the recall period (for example, ‘In the last 7 days…’) and whether characteristics of the disease/condition will affect the respondents’ ability to recall the information easily and correctly11,43. The majority of PRO measures will often have a specified validated recall period, which should not be changed without consultation and approval from the instrument developer. If multiple recall periods have been validated by developers for a particular measure, then input from relevant stakeholders, including clinicians and patients, is recommended to decide which is most appropriate for respondents and the disease/condition of interest43.
Measure delivery
Recommendation 12: ensure that respondents understand why the data are being collected, who will have access, how it will be used and why it is important for them to complete the PRO measures
It is important to inform patients about why PRO data are being collected, making it clear how the data they report could help improve their own care in clinical practice and the future treatment of patients in healthcare research10,26,44. Perceptions of the intrusiveness of items and their usefulness may influence respondents’ perception of burden14. Explanations of the importance of PRO collection and the challenge of missing PRO data, may encourage respondents to complete PROs on a regular basis26. This recommendation applies not only in healthcare research settings, where informed consent is formally obtained, but also in clinical practice where PROs are being used as part of standard care (and patients typically do not sign consent forms).
Recommendation 13: provide clear instructions, training and support for respondents on the completion of PROs as needed
It is important that patients are provided with clear instructions on how to provide their PRO responses and be given ongoing support as needed. This may enhance the quality and completeness of the data collected.
Recommendation 14: provide training and guidance for research staff and clinicians in clinical practice so that they understand the value of PROs and respondent burden
PROs may be perceived to be burdensome by research personnel, clinical teams and research ethics committees, particularly if there are numerous measures or participants are very ill12,26. Qualitative interviews suggest that trialists may be reluctant to collect PROs due to the perceived respondent burden, even when participants may be willing to complete them26. Appropriate training for staff might help alleviate their concerns and avoid an overly paternalistic approach, and may help them address any questions raised by participants regarding PRO collection. It may also help them provide information on the importance and value of data collection, which may motivate participants to complete PRO measures.
For clinical trials, site manuals or protocols should provide specific guidance on PRO administration and management and highlight the importance of facilitating adherence and completeness of data31.
Recommendation 15: specify the level or type of support that can be provided to respondents to facilitate the completion of PRO measures
For respondents unable to complete PRO measures on their own, consider and specify what help can be provided to support completion by the respondent (for example, holding a pen, assistance with a telephone or computer keyboard, scrolling/turning pages or reading out text)11. Responses to the PRO questions should be decided by the patient and not an assisting person.
Recommendation 16: offer flexible modes of administration to meet the needs of target populations and underserved groups
Modes of PRO administration may include paper, mobile device applications (apps), web-based completion, telephone interviews, interactive voice response, audio-computer-assisted interviews and other modes10,28,36. The needs of the target population and their individual preferences should be considered, such as paper or electronic delivery and whether multiple modes are needed to reach all respondent groups36. For example, in older people or respondents with low literacy or visual impairment, interactive voice response, provision of grip-pens or interviewer-administered PRO measures could be considered to reduce the burden10. Patients can also provide feedback on the acceptability of a bring-your-own device versus provisioned devices and additional options such as tablets or paper versions in the waiting room for those who cannot complete PROs electronically at home. Practical ways to reduce the burden at clinic or study visits should be considered28.
Consider the implications of using different modalities when preparing data for analyses. If multiple modes are used for data collection to minimize burden and facilitate diversity and adherence, consider how data from different sources will be integrated. For more information on measurement comparability, see the updated recommendations from The Professional Society for Health Economics and Outcomes Research (ISPOR) Task Force on measurement comparability between modes of data collection for PRO measures45.
Recommendation 17: where possible, consider the use of ePROs, which may help reduce respondent burden, but must be balanced with the needs and preferences of the target population
With patient populations who have access to and are comfortable with electronic devices, the use of electronic PROs (ePROs) may offer additional functionality, which could help reduce burden and improve adherence in healthcare research and clinical practice46. This could include allowing completion on their own devices, with real-time reminders, notifications and responses from the research or clinical team, either to thank them for completion or to respond to issues identified on the PRO, depending on the context of use. Furthermore, ePROs make the use of innovations such as computerized adaptive testing possible. ePROs may also facilitate symptom monitoring between visits12; however, patients may face barriers to using digital services, including a lack of digital skills/low computer literacy or lack of access to reliable information technology infrastructure. Estimates suggest that 37% of the world’s 7.8 billion population are digitally excluded, with older people, people on low incomes and other marginalized groups most likely to be affected35. It is important that these potential barriers and the preferences of the patient population in terms of mode of collection (as described Recommendation 16) are carefully considered with patient input when developing PRO strategies, to ensure that PRO assessments are as inclusive and equitable as possible35.
Recommendation 18: if developing new ePRO systems or modifying an existing one for a new context of use, involve patients and clinicians in the co-design of the ePRO system
Ensure that the patients providing input to the development or modification of the ePRO system include representatives from the target population and are diverse in terms of computer literacy and internet access, considering attributes as appropriate to the research question or clinical context47. Examples may include but are not limited to: sex; gender; socioeconomic background; race/ethnicity; age; health literacy; computer literacy and internet access; and disease characteristics. There may be country-specific regulatory expectations and requirements that may also need to be considered when developing ePRO systems48. The Electronic Clinical Outcome Assessment Consortium has published best practices for the electronic implementation and migration of paper PRO measures to ePROs49.
Recommendation 19: explore the functionality of ePROs with diverse representatives from the target population where possible
Several ePRO features may facilitate completion and help to minimize burden46. Patient involvement in the study co-design and usability testing with the target population can be used to identify appropriate formats50,51. Depending on the context and where permissible, consider providing the following elements in the platform: estimated completion time, progress tracker, graphical results that are easy to interpret, positive messaging/reminders, completion rate and a thank you message after completion42,46. In terms of the format consider using underlining and capitalization where appropriate, easy-to-read fonts and font sizes, one question per screen, back/next buttons and location, branching logic and adaptive web design (where multiple versions of a web page are created to fit different devices)11,42,49. Seek patient preferences on how they receive requests and reminders to complete ePRO measures (for example, emails and/or text messages).
link