OUP user menu

Quality of Reporting of Regional Anesthesia Outcomes in the Literature

Alexander Stojadinovic MD, Sean M. Shockey MD, Scott M. Croll MD, Chester C. Buckenmaier III MD
DOI: http://dx.doi.org/10.1111/j.1526-4637.2009.00683.x 1123-1131 First published online: 1 September 2009

ABSTRACT

Objective. Consistent and reliable standards for reporting of regional anesthetic adverse events are lacking. The quality of reporting of regional anesthetic morbidity has not been assessed critically.

Aim. To evaluate quality of regional anesthesia outcomes reporting.

Methods. Published retrospective or prospective observational cohort or randomized controlled trials in peer-reviewed journals were reviewed, and judged according to seven criteria related to quality of reporting of regional anesthesia complications: method of accrual, duration of data collection, definition of complication, morbidity and mortality rates, grade of complication severity, exclusion criteria, and study follow up. Differences in reporting outcomes according to study design, sample size and time period were compared.

Results. Ninety-one articles published from 1996–2006 involving 8,833 patients were analyzed. The majority of studies (75%) met ≤4 reporting criteria. Recently published, prospective studies with >200 patients were associated with significantly higher-quality reporting (P < 0.05). Fewer than 50% of studies reported at least one recognized, accepted complication with defined criteria or indicated duration of follow up. Reporting compliance was worse (29%) for reporting of actual morbidity rates, and complications leading to death. Complication severity grading related to regional anesthesia was reported in 2% of studies.

Conclusion. Consistent and comparative regional anesthesia outcome data are lacking in peer-reviewed journals. A graded regional anesthetic morbidity and mortality system according to the intensity of therapy required for the treatment of the defined complication is proposed, along with a structured format for the reporting of regional anesthesia complications according to defined reporting standards.

  • Regional Anesthesia
  • Peripheral Nerve Block
  • Continuous Peripheral Nerve Block
  • Outcomes
  • Morbidity

Introduction

The reporting of procedure-related complications in medicine is common worldwide; however, measuring morbidity remains complex. This is particularly true for regional anesthetic block and catheter-related procedures. Part of the difficulty stems from the lack of a comprehensive and reliable standard for the reporting of adverse events related to regional anesthesia, which is accepted uniformly and applied broadly by the anesthesia community. The quality of reporting of regional anesthetic morbidity in the literature has not been assessed critically.

Regional anesthetic outcome reporting relies on procedure-specific morbidity. Such outcome measures form the basis of treatment modification of existing patient management paradigms. The anesthesia outcome literature generally fails to include such critical elements as complications that affect subsequent patient care, particularly sequela that persist for some time following the procedure. Most peer-reviewed regional anesthesia outcome reports lack specific complication definitions or stratified regional anesthetic procedure-specific complications according to severity, or adequate outcome predictors (risk factor adjustments) using standardized analytical methods.

The present study was conducted to evaluate the quality of reporting of regional anesthesia outcomes in the literature. Following a comprehensive search, retrieved articles were reviewed and judged according to seven criteria related to quality of reporting of regional anesthetic complications.

Methods

A standard MEDLINE search strategy was employed to identify retrospective and prospective clinical studies for regional anesthesia. The strategy consisted of the principal medical subject heading term, “anesthesia” being searched, limited to the subheadings: “/adverse effects”, “/complications,” “/mortality,” or “/secondary.” This search was combined with publication type, “Clinical Trial” (including all types and phases of clinical trials, such as randomized controlled trials and meta-analyses) for the years 1996 to the present. This search was combined with terms derived from the Unified Medical Language System: “regional anesthesia,” “regional anesthetic block,” “regional anesthetic catheter,” “continuous regional,” “continuous regional anesthesia,” “peripheral nerve block,” “peripheral catheter,” “continuous nerve block,” “continuous peripheral nerve block.” The results of the combined searches were limited to “Human” studies and “English” language.

Inclusion criteria were defined as 1) published articles (no abstracts); 2) peer-reviewed journal; and 3) retrospective or prospective observational cohort or randomized controlled trial in anesthesia (excluding case reports and volunteer studies). The search was not restricted by study population size. The list of publications, including abstracts, produced by the search was reviewed in order to select articles that met the inclusion criteria. Articles found to be ineligible for inclusion according to the previously listed criteria were discarded. Ninety-one publications with study population size ranging from 10 to 1,422 patients met the inclusion criteria.

The selected publications were divided amongst four senior members of the Army Regional Anesthesia and Pain Management Initiative team (SMS, AS, SMC, and CCB) read, reviewed, and judged according to seven criteria related to quality of reporting of complications related to regional anesthesia including reported method of study accrual, duration of data collection, definition of complication, morbidity and mortality rates, grade of complication severity, exclusion criteria and study follow up (Table 1). Each article was catalogued by study design (retrospective vs prospective) and population size, and publication year.

View this table:
Table 1

Seven criteria utilized to evaluate published articles

CriteriaRequirement
1. Study accrual method definedArticle indicates whether data was collected prospectively or retrospectively.
2. Duration of data collectionArticle indicates how long the patient was followed and evaluated for complications.
3. Definition of complicationThere is at least one recognized, accepted complication with defined criteria.
4. Morbidity and mortality ratesThe number of patients with any complication or death and total complications are reported.
5. Grade of complication severityA grading system was utilized to establish the severity of complication from no morbidity to most severe complication.
6. Exclusion criteriaArticle defines why patients were not included in the study.
7. Follow upArticle specifies that follow up was part of the study design.

Each selected reference was entered into a customized database, and each referee entered determined which of the seven criteria for article evaluation were met and the total number of criteria met for each publication reviewed. This data was entered into the study database. The database also contains information regarding a critical review by the authors. The critical reviews included an appraisal of the strengths and weaknesses of the study design, the statistical merits of the trial, and overall quality of reporting of regional anesthesia outcomes. The mean number of criteria met varied according to reviewer: AS = 4.0 ± 1.0; SS = 2.3 ± 1.1; SC = 3.6 ± 1.7; CB = 4.3 ± 1.0. Reviewer SS's lower overall score was reflective of having reviewed principally small (N < 100) trials published in the early period (1996–2001) studies.

Differences in reporting outcomes between groups according to study design, sample size, and time period were compared using the t-test or Wilcoxon rank sum test. The analysis of categorical data was conducted using Fisher's exact test (two-tailed). Significance was determined at the P ≤ 0.05 level. Statistical analyses were carried out using SPSS statistical software (ver.13, SPSS, Inc., Chicago, IL).

Results

Ninety-one regional anesthesia articles published between 1996 and 2006 involving 8,833 patients (mean = 97; median = 44; range 10–1,422 per study) were analyzed [1–91]. The majority of studies were prospective (N = 81 [89%]) in nature, involved <100 study subjects (N = 72 [79%]), and were published in the last 5 years (N = 66 [73%]). Only one article met all seven quality-reporting [51] criteria. The majority of studies (N = 68 [75%]) met four or fewer reporting criteria (mean = 3.6 ± 1.5 [SD]; median = 4). Specific reporting criteria met and compliance rates according to study design, sample size, and time period are shown in Table 2.

View this table:
Table 2

Criteria met according to study design, sample size, and time period

# Criteria metStudy design: retrospective vs prospectiveStudy population size: <100 vs 100–200 vs 200+Time period study published
RetroPro<100100–200200+1996–20012002–2006
1 to 2 521242 01313
3 to 4 438326 4 834
5 to 6 121161 5 418
7 0 1 00 1 0 1
Total1081729102566

Study design influenced quality outcome reporting (retrospective: mean criteria = 2.7 ± 1.5 [SD]; prospective: mean criteria = 3.7 ± 1.4 [SD]; P = 0.04). Study population size significantly influenced the likelihood of higher-quality reporting (N < 100: mean criteria = 3.4 ± 1.4 [SD]; N = 100–200: mean criteria = 3.3 ± 1.4 [SD]; N > 200: mean criteria = 5.0 ± 1.2 [SD]; P = 0.009). The quality of reporting improved during the latest study period (1996–2001: mean criteria = 3.0 ± 1.4 [SD]; 2002–2006: mean criteria = 3.8 ± 1.4 [SD]; P = 0.03).

Specific reporting criteria and reporting compliance according to study design, sample size, and time period are tabulated below (Table 3). Study accrual method defining retrospective or prospective design was almost uniformly met. Fewer than 50% of studies reported at least one recognized, accepted complication with defined criteria or indicated duration of follow up for assessment of complications. Reporting compliance was notably worse (29%) for the reporting of actual morbidity and mortality rates, and complications leading to death within the study population. Severity grading of complications related to regional anesthesia was reported in two of 91 studies (2%), limited to generalized major or minor categories of morbidity.

View this table:
Table 3

Specific criteria and reporting compliance according to study design, sample size, and time period

CriteriaStudy designPopulation size:Time periodTotal
Retro N = 10Pro N = 81<100 N = 72100–200 N = 9200+ N = 101996–2001 N = 252002–2006 N = 66N = 91
1. Accrual method88170910246589 (98%)
2. Duration556485 8144761 (67%)
3. Definition5363056 73441 (45%)
4. M&M rates2241538 62026 (29%)
5. Grade/severity0 2 101 0 2 2 (2%)
6. Exclusion3635457155166 (73%)
7. Follow up5362741093241 (45%)

Discussion

This is the first systematic analysis of the regional anesthesia peer-reviewed literature for outcomes reporting in peripheral nerve block studies. We find that consistent and comparative regional anesthesia outcome data are generally lacking given the absence of working definitions of complications, taxonomy, and reproducible conventions for objective characterization of the severity of complications. The paucity and inconsistency in outcomes reporting makes comparisons of published studies for patient safety-related issues particularly difficult.

Severity grading of complications related to regional anesthesia was reported in only 2% of articles reviewed. Frequently, authors of the reviewed manuscripts suggested that “no complications” occurred during the study period. As a clear definition of what would be considered a complication was provided in less than half of the manuscripts evaluated, this statement must be viewed with some incredulity. Furthermore, only 45% of the manuscripts reviewed provided for some outpatient follow up to identify potential late complications of the regional block. The reporting of morbidity rates occurred in less then a third of the manuscripts. This review suggests an unmet need for a structured format for the reporting of regional anesthetic complications.

In 1992, Clavien, Sanabria, and Strasberg recognized that the absence of a standardized reporting system for surgical complications made comparison of outcomes and quality of care between medical centers and meta-analysis with the aim of identifying independent risk factors for operative morbidity impossible [92]. They developed a classification system for complications related to cholecystectomy, stratified according to morbidity: minor (Grade I), potentially life threatening (Grade II) without residual disability, potentially life threatening (Grade III) with residual disability, including organ resection or persistence of life-threatening condition, or death (Grade IV). Over a decade later, Clavien and colleagues proposed revisions to the original system taking into account life-threatening complications warranting care in a monitored care setting, the lethality of central nervous system complications, and the prolonged disability that specific complications entail [93]. Clavien's revised classification of surgical complications is an extraordinary step toward an internationally accepted system to measure operative morbidity; however, the added value of supplemental subclassifications such as grade IIIa, Ivb, and suffix “d” is uncertain. Widespread acceptance and utilization of any classification system of procedure-related morbidity will rely heavily on ease of interpretation and application, clarity of communication and consistency that in the end provides information critical to clinical problem-solving [94].

Similarly, the lack of standardized nomenclature for regional anesthetic complications reflects the challenges of comparing procedure-specific outcomes between centers and points toward the need for anesthesia societies to develop consensus complication definitions. We propose a six-grade regional anesthetic morbidity and mortality system (Regional Anesthesia Outcomes Reporting System), a modification of Clavien's [92] original classification system, according to the intensity of therapy required for the treatment of the defined complication (Table 4).

View this table:
Table 4

Regional anesthesia adverse events defined by the Regional Anesthesia Outcomes Reporting system

GradeExamples
1. Technical difficulties, equipment failure, added equipment, or errors that DO NOT increase morbidity in the patient.Pump failure, broken catheter, difficult needle insertion, stopped infusion (any reason), Horner's syndrome, localized myalgia, and addition of ultrasound to a stimulation block.
2. Complications that require minimal intervention (no added medications) and DO NOT increase morbidity in the patient.Blood vessel puncture, sedation requiring airway support (chin lift), recognized intravascular catheter placement, failed block, local anesthetic toxicity symptoms not requiring therapy, superficial catheter site infection.
3. Complication that results in self-limited morbidity to the patient (added medication).Seizure, medication errors, wrong sided block, sedation requiring airway intervention (mask ventilation or medications), unintended epidural or intrathecal injection, pneumothorax requiring observation only, drug reactions.
4. Complications that change care and result in morbidity lasting <90 days.Nerve injury, pneumothorax requiring a chest tube, hematoma requiring intervention, cancelled surgery, catheter infection requiring antibiotics or surgery, respiratory or cardiac arrest.
5. Complication resulting in permanent, >90 days, morbidity.Any prior category event that lasts >90 days.
6. Death related to the regional anesthetic.Local anesthetic induced cardiac arrest resulting in death.

Having identified marked variations in the methods of regional anesthesia outcome reporting and lack of clear-cut reporting standards, we also suggest a structured format for the reporting of regional anesthesia complications according to the reporting standards in this report and utilize a modified format of “Explanation and Elaboration” established in the revised CONSORT Statement (Table 5; ref. [95]). Key elements are provided in an effort to add clarity to regional anesthesia morbidity and mortality reporting. These reporting standards should serve as a basis for an ongoing working document that journal editors and referees may rely on to judge the completeness and adequacy of regional anesthesia reports and the reliability of study findings and interpretation of its results.

View this table:
Table 5

Regional anesthesia morbidity and mortality reporting standards

StandardRequirement
1. Study design, data accrual, study setting, eligibility criteria, and study periodRetrospective or prospective observational or randomized study; Method and source used to obtain data; Location and setting where data is collected; Inclusion and exclusion criteria for study participants and study time period.
2. Duration of data collectionPeriod of time of post-procedure accrual of complication data such as 90 days is specified
3. Definitions of complicationsAll procedure-specific complications are clearly defined.
4. Morbidity ratesNumber of patients with any complication and the total number of complications are indicated.
5. Mortality ratesAny procedure-related death is reported along with precise details of the cause of death. The total number of patients that died in the post-procedure period is indicated. Median time to death is reported.
6. Grade of procedure-specific severityProcedure-specific complications are listed for any grade morbidity (Grade I through VI) and complication rate for each procedure-specific complication amongst all patients. Grade distribution (I through VI) of the complication and percent of each specific complication with that grade is presented. Morbidity and mortality rates by severity grade (Grade I through VI) of maximum complication per patient are tabulated.
7. Risk factorsStatistical methods used to identify independent patient-specific risk factors for major (Grade III-VI) procedure-specific morbidity and risk-adjusted regional anesthetic outcome are reported.

Footnotes

  • The opinions or assertions contained herein are the private views of the authors and are not to be construed as official or reflecting the views of the Department of the Army or the Department of Defense.

  • Dr Stojadinovic and Dr Shockey share first authorship.

  • Financial support: 1. John P. Murtha Neuroscience & Pain Institute; 2. Departmental.

References

View Abstract