Delphi Process

What is a Delphi Process?

The Delphi process is an iterative consensus method involving a series of questionnaires that are sent to relevant stakeholder participants (generally experts for a specific health condition).[1] Participants in the INTEGRATE-Pain core outcome set (COS) development Delphi process will be asked to rate the importance of outcome domains on a questionnaire created by our steering committee. The goal of the process is reach consensus on the most important domains for all forms of acute pain, the transition from acute to chronic pain, episodic/breakthrough pain, and chronic pain. The result will be four new overarching pain COS – one each for acute pain, the transition from acute to chronic pain, episodic/breakthrough pain, and chronic pain. Most Delphi studies involve two-to-three rounds of voting. The Delphi technique is the most common method for rating the importance of outcomes and outcome measures for inclusion into a COS.[2] The image below provides an overview of how the Delphi process works for producing a set of patient reported outcome (PROs) domains for a COS.

Why use a Delphi process?

The iterative nature of the Delphi process will allow for as many rounds as necessary to narrow down the list of outcome domains for the new overarching pain COS, which ensures the team can account for as many outcome domains as needed. Additionally, if a participant strongly believes a pain domain is missing from the list extracted from the literature review, they will have a chance to add it in for voting during the Delphi process. Participants in the process will use the group’s overall ratings from previous rounds to refine their outcome ratings, ensuring that the resulting COS reflect the collaborative nature of the effort. Responses will be reported back to the group anonymously, so that we can ensure a free and open discussion without fear of criticism over a particular selection. This process will also ensure that patients are included in identifying the domains for the overarching COS for pain.

What are the steps of the Delphi process?

  1. The INTEGRATE-Pain steering committee conducted a systematic literature review to develop a list of possible outcome domains (and their definitions) for inclusion in the COS. These will be presented at the pain domain meeting on June 14, 2022 (broken down by pain categories).
  2. Each participant will be sent a questionnaire/survey with the list of pain domains for the pain sub-group which they represent (acute, transition from acute to chronic, episodic/breakthrough, or chronic). Stakeholders will rank or score the domains according to importance. Participants can add in additional domains at this stage that they feel are missing.
  3. Participants will send their anonymous surveys back to the research team, who summarize the responses from the whole group and send a new questionnaire back to participants for ranking.
  4. Participants will be asked to score/rank the pain domains again, considering the results from the overall group and keeping in mind their previous ratings. Participants will send their responses back again, and the research team will collate the information again. The Delphi process usually involves 2-3 rounds of voting. However, the number of rounds will depend on how long it takes each pain category to reach consensus among the participating stakeholders.
  5. After the INTEGRATE-Pain steering committee collects and analyzes the results from each group, meeting participants and voting stakeholders will have an opportunity to discuss the results.
  6. The research team will write a journal article which summarizes the Delphi consensus process and articulates the recommended pain domains (and definitions), as agreed upon by the stakeholders. These will become the overarching core outcome sets (COS) for acute pain, the transition from acute to chronic pain, episodic/breakthrough pain, and chronic pain.

INTEGRATE-Pain Consensus Criteria for Overarching Core Outcome Sets

References

  1. Remus, A., Smith, V, & Wuytack, F. Methodology in core outcome set development (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study. BMC Medical Research Methodology 21, 10 (2021). https://doi.org/10.1186/s12874-020-01197-3
  2. COMET Initiative. Delphi plain language summary. https://www.comet-initiative.org/assets/downloads/Delphi%20plain%20language%20summary%20for%20COMET%20website.pdf. Accessed January 14, 2022.
  3. Vector illustration. Colorful lines intertwined in arrow. Dimensions 16:9. Stock illustration. https://www.istockphoto.com/vector/vector-illustration-colorful-lines-intertwined-in-arrow-dimensions-16-9-gm1281047321-379188949. Accessed January 14, 2022.