Can-SCIP Coach App
  1. Can-SCIP
  2. Development Process

Development Process

Can-SCIP Guideline Development Process

The Can-SCIP Guideline was developed using the Guidelines Adaptation Cycle (ADAPTE) (www.adapte.org) originally derived and modified from a process developed by Graham & Harrison (2005). The steps involved in the process are outlined in detail in Patsakos et al. (2021)

Below is a summary of the Guideline development process.

(1) Establish an expert panel:

Clinicians, individuals with lived experience, program directors, knowledge translation experts, researchers, and administrators and other relevant stakeholders were invited to join the Can-SCIP Expert Panel in May 2019. Panel members were asked to declare any conflicts of interest, involvement in research and provide their role and discipline.

(2) Search and retrieval of previously published guidelines:

A systematic scoping review was undertaken for CPGs focused on treatment and evidence-based recommendations in the field of SCI.

The Can-SCIP steering committee consulted with the Health Sciences Librarian at University of British Columbia to assist with construction of the search. The following databases were searched:

  • PubMed,
  • Medline,
  • Embase,
  • CINAHL, and
  • PsycINFO

In addition, indexes and databases that specifically archive clinical guidelines and medical evidence were also included in the search:

  • NCCIH Clearinghouse,
  • Clinical Key,
  • Trip Medical Database,
  • DynaMed Plus,
  • Scottish Intercollegiate Guidelines Network,
  • CADTH Grey Matters tool,
  • Guidelines International Network, and
  • Physiotherapy Evidence Database Ratings).

The key search terms included: 'spinal cord injury', 'spinal cord dysfunction', 'tetraplegia', 'quadriplegia', 'paraplegia', 'spinal cord impaired', 'spinal cord lesion' (including truncations of these SCI terms) and 'clinical practice guidelines'.

CPGs published between 2011 and 2018 in English or French, written by four or more authors, applicable to the Canadian health care setting, including evidence-based recommendations for adults over 18 years of age were considered for inclusion. Systematic reviews were excluded, and shorter evidence-based documents were excluded, but their reference lists were hand-searched to find any additional clinical practice guidelines for inclusion. The Can-SCIP steering committee reached out to stakeholder organizations to identify CPGs that were currently in development.

(3) Guideline quality appraisal process using AGREE instrument:

Each eligible CPG was then evaluated individually by two to four appraisers from the expert panel and/or Can-SCIP steering committee, using the Appraisal of Guidelines for Research and Evaluation II instrument (AGREE II; https://www.agreetrust.org/wp-content/uploads/2017/12/AGREE-II-Users-Manual-and-23-item-Instrument-2009-Update-2017.pdf).

The AGREE II instrument evaluates the guideline development process and quality of the guideline across six domains including: (1) scope and purpose, (2) stakeholder involvement, (3) rigour of development, (4) clarity of presentation, (5) applicability, and (6) editorial independence.

Training sessions were held to ensure all Expert Panel members were familiar with using the AGREE instrument. Each CPG was given a standardized score ranging from 1-100 (100 representing a strong score) by the reviewing appraiser from the expert panel.

As the AGREE User Manual does not specify a minimum score that is considered 'low-quality,' the Can-SCIP steering committee set a benchmark of 40% for inclusion; whereby scores higher than 40% represent higher quality, and scores below 40% represent poorer quality. CPGs with an AGREE score below 40% were excluded from the recommendation review process.

(4) Create a recommendation matrix for selection of CPG recommendations:

A recommendations matrix was created to compare similar or overlapping recommendations across all included CPGs. The CPG recommendations and evidence statements obtained from SCIRE were divided into twenty-four domains relevant to SCI care and treatment within the matrix.

As various evidence grading systems were used across the different selected CPGs, the Can-SCIP steering committee used a standardized grading system.

(5) Literature search by SCIRE team on prioritized topics:

The search processes were enhanced by systematic searches of the SCI literature conducted by the Spinal Cord Injury Research Evidence (SCIRE) project team (https://scireproject.com/) to ensure the incorporated recommendations are based on the most current evidence.

Evidence statements formulated by the SCIRE project team were added to the synthesized materials prior to convening the entire expert panel to facilitate the formulation of de novo recommendations when there were not existing recommendations, or where the existing guidelines were outdated, insufficient or not relevant to the Canadian context.

(6) Consensus Meeting to select and/or adapt Can-SCIP recommendations from published recommendations or develop new ones:

A 2-day consensus conference was held in Niagara Falls, Ontario, Canada on October 15 to 16, 2019. The expert panel was convened in a two-day meeting prior to the Canadian Spinal Cord Injury Rehabilitation Association (CSCI-RA) 8th National Conference in Niagara Falls, Ontario, Canada, in October 2019. All expert panel members completed declarations of conflicts of interests.

The expert panel members were invited to review domains in which they had established expertise or a unique perspective to contribute. Fifty expert panel members reviewed the recommendations matrix independently prior to the consensus meeting and in small working groups during the meeting. At least two individuals with lived experience with SCI were included within each working group to ensure their views were considered.

The key activities undertaken by the expert panel members at the consensus conference were to:

  • Review the quality assessment of previously published best practice guidelines in the SCI field.
  • Consider the evidence tables derived from the SCIRE systematic review.
  • Draft or refine recommendations: Each working group selected recommendations from existing CPGs for inclusion, modification or refinement of existing recommendations based on current evidence (i.e. rewording with Canadian terminology, separated some lengthy recommendations into two separate recommendations), or developed new recommendations based on the most current evidence provided by the SCIRE Project. New recommendations with consensus support were also articulated by the experts in each working group. In summary, the recommendations were either adopted with the original wording or revised/reworded based on current evidence/settings.

  • Assign a level of evidence: The experts reviewed the recommendations and supporting evidence and assigned one of 3 levels of evidence.

    • (i) the section of the care continuum the recommendations applies to (pre-hospital, acute & surgical, tertiary rehabilitation, community),
    • (ii) the neurological level of injury and AIS to which the recommendation applies,
    • (iii) whether the recommendation applies to an individual with a specific cord syndrome – central cord syndrome, anterior cord syndrome, posterior cord syndrome, Brown-Séquard syndrome or cauda equina syndrome;
    • (iv) whether the recommendation applies to a person with an upper motor neuron neurogenic bowel or bladder; and,
    • (v) and whether there were any groups for whom the recommendation does not apply.
  • Identify potential implementation toolkits/resources to assist with implementation. Experts were asked to provide lists of websites, publications, decision rules and other implementation tools that could be used to facilitate recommendation uptake.

(7) Expert consensus vote and final modification suggestions:

The expert panel voted on all the recommendations using the online survey tool, Survey Monkey®.

For each recommendation, the expert panel selected whether the recommendation should be included within the Can-SCIP Guideline, whether the recommendation should not be included in the Can-SCIP Guideline, or whether the recommendation should be included in the Can-SCIP Guideline but that further modifications were necessary. Recommendations with less than 80% agreement were excluded from the Can-SCIP Guideline.

(8) Finalize recommendation grading and wording:

The Can-SCIP steering committee adapted and refined each draft recommendation based on the feedback received from the expert panel and the established format for the recommendations based on the weight of the evidence underpinning the recommendation.

(9) External review:

As part of the validation process, the Can-SCIP Guideline was externally reviewed by recognized international experts in SCI who did not participate in the Can-SCIP Guideline development process. The purpose of conducting the external review was to gather information on both the reviewer's overall impression of the Guideline and specific comments addressing the following issues: validity, relevance, awareness of new information, evidence or concerns, scope and purpose, stakeholder involvement, rigor of the methods and clarity of presentation according to some questions from the AGREE II instrument. The steering committee considered revisions to the Can-SCIP Guideline based on the suggestions and comments from the external reviewers, feedback from the expert panel and the CPG aims and structure. 1 Graham ID, Harrison MB. Evaluation and adaptation of clinical practice guidelines. Evidence-based Nursing. 2005;8:68-72.