INESSS-ONF Guideline Development Process

1. Scoping review and evaluation of existing CPGs

The Guidelines Adaptation Cycle process by Graham & Harrison (2005) was used to develop the INESSS-ONF guideline. As suggested in this model, to minimize repetition of previously completed work, the INESSS-ONF Guideline Development Team first reviewed existing TBI guidelines in order to extract relevant recommendations that could be adapted to the Canadian context.

A scoping review was performed to search for existing clinical practice guidelines (CPGs) published in English or French within the last 14 years (2000-2014) that were relevant to moderate to severe TBI and addressed multidisciplinary rehabilitation care. This was conducted using bibliographic databases (CINAHL, Pubmed, Embase and APA Psyc NET). The following key words and their corresponding concepts were used: brain injury and practice guideline. Documents obtained via the scoping review were excluded if they: (1) were published before 2000, (2) addressed only mild TBI, (3) addressed only acute care or (4) addressed only care for children.

Each CPG was then evaluated individually by four appraisers using the Appraisal of Guidelines for Research and Evaluation II (AGREE II; http://www.agreecollaboration.org); The AGREE II instrument evaluates the guideline development process and quality of the guideline across 6 domains including: (1) Scope and purpose, (2) Stakeholder involvement, (3) Rigour of development, (4) Clarity of presentation, (5) Applicability, and (6) Editorial independence. Each guideline was given a standardized score ranging from 1-100 (100 representing a strong score) by the reviewing expert. The AGREE scores were summarized and presented to the expert panels at the conference in November 2014. Only one existing CPG did not meet the quality requirements at the outset of the AGREEE evaluation process and was therefore excluded. Eight (8) clinical practice guidelines met our inclusion criteria and were of adequate quality were retained for the adaptation process (see Table 1).

Table 1: Clinical Practice Guidelines Evaluated for INESSS-ONF Guideline

2. Validation of the end-users' needs and expectations

A needs assessment survey was developed to obtain feedback and concerns by end-users of the CPG to guide the development strategy of the CPG particularly the nature, scope and format of the desired end product. The survey was developed to document: 1) the actual level of use and knowledge of CPG, 2) the core subjects to be covered by the CPG, and 3) the preferences toward CPG implementation strategies. A total of 487 rehabilitation stakeholders participated in the survey, including clinicians, coordinators and program managers from acute, inpatient and outpatient rehabilitation. Although 47% of the participants confirmed they know at least one CPG for the rehabilitation of individuals with TBI, and most of the participants indicated a positive or neutral opinion of CPG, very little use it to support their activities. The most useful topics for the participants were: intervention to improve cognitive function (87%) and to manage difficult behavior (87%), Intensity and frequency of intervention (79%) and Social participation and community life (68%). Training, use of the CPG by the team members and time to read and understand the CPG were among the most popular strategies to facilitate the implementation of the CPG. The results enabled the guidelines development team to identify the priority topics to be addressed in the CPG, the format and the desired supporting tools, as well as the preferred implementation strategies.

3. Synthesis of all existing documentation and evidence

The guideline development strategy was based on the identification and adaptation of currently published, high quality recommendations most clinically relevant for TBI rehabilitation in Canada. A recommendations matrix was created to allow comparison of the recommendations and their level of evidence, extracted from existing guidelines. “Statements of evidence” taken from the Evidence-Based Review of Moderate To Severe Acquired Brain Injury (ERABI) (2016) https://erabi.ca/ were added to the synthesis matrix to permit the formulation of "de novo" recommendations when there were no existing recommendations or where the existing ones were insufficient. This material was regrouped by clinical topic and provided to the expert panel at the Consensus Conference held in November 2014.

4. Coordination of a consensus process amongst expert

A 2-day consensus conference was held in Montreal, Quebec, Canada on November 27 and 28, 2014 to develop the clinical practice guideline for the rehabilitation of adults with moderate to severe traumatic brain injury. There were 60 participants and consisted of individuals from Ontario and Quebec with a range of clinical, research, policy, decision-making, lived experience, and knowledge translation expertise.

The objectives of the consensus conference were to:

• Review the findings from a quality assessment of previously published best practice guidelines in the TBI rehabilitation field;
• Review evidence for treatments through expert input and the Evidence-Based Review of Acquired Brain Injury (ERABI) by Teasell et al.;
• Develop evidence-informed practice recommendations that could be implemented in TBI rehabilitation programs in Québec and Ontario during the next phase of the project;
• Identify potential performance indicators to measure practice related to the recommendations.

The two primary themes used to organize the work were: 1. Organization of rehabilitation services, and 2. Rehabilitation of specific brain impairments. The members of the consensus panel were assigned to working groups to address the many sub-topics within these two themes. Each working group completed a recommendations worksheet for their topics. The recommendations were either adopted with the original wording or revised/reworded based on current evidence/settings. New recommendations were also generated by consensus based on research and clinical expertise.

5. Adaptation of the recommendations and production of the guideline

Recommendations worksheets were transcribed and compiled after the consensus conference. Working group members were then provided with specific reviewing and editing instructions and asked to propose indicators and provide resources. Additional working groups in the area of “Neuropharmacology” and “Length of Stay-Intensity” were formed to develop recommendations on these topic areas.

The project committee adapted, refined and compiled all of the recommendations and proposed edits, in addition to addressing specific comments and concerns identified during the conference and through the follow-up work.. The expert panel members individually voted using an online survey to narrow down the set of recommendations to the most important and relevant recommendations. The survey was composed of two different rounds: Round 1 which surveyed participants as to “which recommendations should be included in the INESSS-ONF CPG?” and Round 2 which surveyed participants “from the ‘included recommendations', which should be identified as a “key” recommendation for implementation?”. Responses from the first round were summarized to form the basis of the second round. Recommendations for which more than 20% of the panel members voted ‘not to keep’ in Round 1 were further reviewed and were excluded based on analysis. The remaining recommendations were compiled for the second round. After the voting had closed, the Guideline Development Team finalized the list of guideline recommendations and circulated to the expert panel to provide a final opportunity to review the recommendations.

Post-voting, the 266 recommendations included in the CPG have all received at least 80% agreement by the expert panel as important to include in the guidelines. The final set of recommendations is divided in two large sections: Section I: Components of the Optimal TBI Rehabilitation System includes 71 recommendations (35 New, 36 Existing recommendations) while Section II: Assessment and Rehabilitation of Brain Injury Sequelae includes 195 recommendations (91 New, 104 Existing recommendations). A total of 126 new, original recommendations were formulated - highlighting the relevance of producing a new CPG in order to respond to the needs and context of practice in Quebec and Ontario, with an emphasis placed on informing and standardizing practice while also providing practical, implementable guideline recommendations.

The project team and the facilitators from the conference finalized the supplementary text accompanying the recommendations. This included a brief rationale for the recommendations, system implications, key structure and process indicators, suggested tools resources as well as a summary of the evidence for each section.

A initial set of indicators was first proposed by the members of the expert panel during the Consensus conference. Those were then ratified and refined to reflect the most significant constructs included in the recommendations, with special attention to the ones that could bring changes to a clinician/manager's practice. Among them, key indicators were singled out by the Scientific Committee through a voting process. For each section of the CPG, members were invited to identify and rank the top 5 indicators, either by selecting them through the existing ones, or by suggesting new ones. A mean rank analysis was ultimately performed and allowed for the identification of the most important indicators to present with each section's guideline text.

The key translation occurred once the recommendations were finalized in English. All 266 recommendations were translated and reviewed by the French speaking members of the expert panel. Difficulties in translating terms were highlighted and adjusted in accordance with not altering the intention of the recommendation. The French speaking panel members then reviewed the recommendations to verify that the intention and key aspects were maintained. In some cases the translation revealed wording in English that was overly complicated and unclear, resulting in additional clarifications. Once the supplementary text for each section (rationale, system implications, key indicators, tools and resources as well summary of the evidence) was verified, this too was translated into French.

A draft of the guideline was circulated to recognized international experts in the field and stakeholders who did not participate in the development process. One of the international experts was French speaking and reviewed the translated version of the recommendations. The external reviewers were requested to provide input about the validity and relevance of the guideline. This feedback was incorporated into the final draft.

6. Implementation of the CPG in clinical settings

ONF and INESSS each formed an Implementation Advisory Committee to guide and advise on key system issues and considerations for implementation of guideline recommendations within their province. A consultation was conducted in clinical settings to assess the differences between the current practices and the recommendations proposed in the CPG and to determine which recommendations are priorities for implementation, the level of feasibility and the main issues to be considered. A detailed strategy will be developed in the fall of 2016 in collaboration with our stakeholders.