OUP user menu

Development of a Comprehensive E-Learning Resource in Pain Management

Leanne M. Yanni MD, John W. Priestley MA, Jeanne B. Schlesinger MEd, Jessica M. Ketchum PhD, Betty A. Johnson MD, PhD, Sarah E. Harrington MD
DOI: http://dx.doi.org/10.1111/j.1526-4637.2008.00511.x 95-105 First published online: 1 January 2009


Objective. The prevalence of chronic nonmalignant pain (CNMP), the lack of confidence and reward among trainees and providers caring for patients with CNMP, and the lack of a comprehensive curriculum in pain management prompted the creation of the Virginia Commonwealth University (VCU) Chronic Nonmalignant Pain Management curriculum, an innovative e-learning resource. This article describes the development of the curriculum and presents initial evaluation data.

Design. The curriculum is organized into six modules that cover 20 specific Accreditation Council of Graduate Medical Education competency-based objectives. Broad content and effective instructional design elements promote its utility among a range of learner levels in a variety of medical disciplines.

Results. Twenty-four physician reviewers and over 430 trainees (medical students and graduate medical residents) have evaluated the curriculum. Of the respondents to course evaluation questions, 85.7% (366/427) stated that they would access the practice resources again, 86.3% (366/424) agreed that the treatment of CNMP was more important to them after completing the curriculum, 73.9% (312/422) stated that they would make changes in their behavior or practice, and 92.3% (386/418) stated that they would recommend the curriculum to their colleagues. Qualitative data are uniformly positive. Results of pretest and posttest scores and item analyses have been used to make content changes.

Conclusions. The VCU Chronic Nonmalignant Pain Management curriculum is an e-learning resource that has the potential to fill a significant training void. Design and content changes have been made as a result of initial evaluation data. Data from ongoing evauation will allow curricular refinement.

  • Accreditation Council for Graduate Medical Education (ACGME)
  • Chronic Pain
  • Curriculum Development
  • Medical Education
  • Pain Management
  • Pain Training Programs


National Perspective

Due to well-described management challenges, residents and practicing physicians indicate a lack of confidence and reward in managing patients with chronic nonmalignant pain (CNMP) [1–3]. Inadequate training has been cited as one reason for providers feeling ill-prepared to manage this problem [4].

Both the Joint Commission and the U.S. Congress, which declared 2001–2011 as the “Decade of Pain Control and Research,” stress the importance of effective pain management [5,6]. There has also been a call for establishing widespread training requirements in pain management [7–10]. Importantly, providers and trainees themselves have indicated a need for relevant training [10–12].

Curricular Efforts by Others

Data on several pain curricula have been published. Each of these curricula has limitations: some target only a specific audience or specialty; some address a selected aspect of chronic pain management, such as prescribing opioid analgesics; and some require significant delivery resources, such as standardized patients [13–19].

E-learning programs are growing in number, content, and educational efficacy [20–22]. However, there are no published descriptions of e-learning programs in chronic pain management. Several pain organizations offer online programs funded by the pharmaceutical industry, but evaluation and efficacy data are lacking. No extant online programs feature delineation of objectives by standard Accreditation Council of Graduate Medical Education (ACGME) competencies or offer tracking of user completion, both particularly relevant for clerkship and program directors.

Institutional Perspective

Consistent with national training deficiencies, our institution, Virginia Commonwealth University (VCU) Medical Center, lacked a comprehensive approach to education about chronic pain management at both the undergraduate and graduate training levels. Lack of knowledge contributed to practice deficiencies, lack of preparation undermined trainee confidence, and lack of awareness and access resulted in underutilization of practice resources. Curricular aims were to address these three broad areas: knowledge, confidence, and access to resources.

In recommended format [23], this article describes the process of curriculum development, the content and elements of instructional design, initial evaluation data, and revisions based on these evaluation data.

Development Process

Needs Assessment

We conducted a needs assessment at VCU Medical Center, which included an internal medicine residents' questionnaire on CNMP, as well as a retrospective chart review in the internal medicine resident continuity clinic [24]. The results of the questionnaire revealed a self-perceived lack of preparation in managing chronic pain, a lack of personal reward when treating patients with chronic pain, and a significant negative impact on residents' view of primary care as a career. The retrospective chart review identified significant practice deficiencies in risk assessment and prescription drug misuse monitoring in patients treated with opioids for CNMP.

Decisions about Scope, Selection of Topics, and Design

The topic of chronic pain management spans disciplines. Several studies have documented improved patient outcomes with a multidisciplinary approach to treatment [25,26]. The content writers for our curriculum included general internists, a palliative care specialist, an addiction specialist with additional training in psychiatry, a pharmacist, and a toxicologist. While not primary content writers, experts from other disciplines including psychiatry, rheumatology, and neurology were consulted as curriculum objectives and content were developed. The curriculum was written over a span of 4 months in 2005 and included the following topics in systematic progression: assessing chronic pain, treating chronic pain with a multidisciplinary approach, reviewing the management of specific pain syndromes, confronting challenges including prescription drug misuse, and reviewing laws and regulations that govern controlled substance prescribing.

Several key references in chronic pain management were germane to the choice of module headings and development of measurable objectives including published resources for developing pain curricula in medical schools, comprehensive pain management text books, and federal documents from the Drug Enforcement Administration and the Federation of State Medical Boards [27–32]. Over 6 months, the objectives, questions, and content were transformed into an e-learning resource through a partnership with VCU School of Medicine's Office of Faculty and Instructional Development.

We chose e-learning for several reasons. First, trainees have indicated satisfaction with and preference for e-learning [33]. Additionally, the curriculum is accessible at any time, regardless of clinical schedules or duty hours, and from anywhere, considering that computers are now ubiquitous. Ease of accessibility in a busy practice setting is especially beneficial with the inclusion of practice resources including pharmacology tables, summary tables, tools, references, and Web links allowing rapid access to information. Online delivery also encourages widespread use within the academic medical center and among affiliated community-based providers. Finally, online delivery allows a potential training resource to be shared with other institutions [34].

Design decisions focused on optimizing the curriculum for use among a range of learner levels in a variety of medical disciplines. The process followed “Ten Steps to Effective Web-Based Learning” and included “Characteristics of Effectively Designed Webpages,” as described by Cook and Dupras [35].

Curriculum Description


The VCU Chronic Nonmalignant Pain Management curriculum was created to fill a void in training with aims to improve knowledge, improve confidence by better preparing students and residents to manage a challenging problem, and to provide access to practice resources.

Curricular Topics

The curriculum is composed of six modules: Overview and Assessment of Chronic Nonmalignant Pain; Treatment of Chronic Nonmalignant Pain; Common Pain Syndromes: Fibromyalgia; Common Pain Syndromes: Neuropathic Pain; Identifying and Meeting Challenges; and Legal and Regulatory Aspects of Prescribing Controlled Substances. Three to four objectives are housed within each module. Each of the 20 objectives is tied to one of the six standard ACGME competencies, based on consensus from several expert clinician educators. These objectives and associated competencies can be viewed in Table 1.

View this table:
Table 1

Module headings and specific objectives

Associated ACGME Competency
Module 1: Overview and assessment of chronic nonmalignant pain
Recognize the responsibility of the health care provider in caring for patients with chronic nonmalignant painProfessionalism
Identify elements of the chronic pain assessment that are essential for developing a patient-centered treatment regimenPatient care
Describe methods of assessing chronic pain in the elderly, specifically in those with cognitive and/or functional impairmentInterpersonal and communication skills
Describe features and examples of non-nociceptive or maladaptive painMedical knowledge
Module 2: Treatment of chronic nonmalignant pain
Apply a multidisciplinary approach to the management of patients with chronic nonmalignant painPatient care
Identify effective nonpharmacologic therapies for chronic nonmalignant painMedical knowledge
Describe and apply principles of non-opioid pharmacologic therapy for the treatment of chronic nonmalignant painMedical knowledge
Describe and apply principles of opioid use for the treatment of chronic nonmalignant painMedical knowledge
Module 3: Common pain syndromes: fibromyalgia
Recognize the clinical syndrome of fibromyalgiaPatient care
Apply a stepwise approach when managing patients with fibromyalgiaPatient care
Choose specific pharmacologic treatment options for patients with fibromyalgiaMedical knowledge
Module 4: Common pain syndromes: neuropathic pain
Recognize the major diagnostic features of neuropathic pain as well as common neuropathic pain syndromesPatient care
Advanced content: review concepts in neuropathic pain pathophysiologyMedical knowledge
Name the evidence-based pharmacologic treatment options for neuropathic painMedical knowledge
Recall the management of specific neuropathic pain syndromes including: diabetic polyneuropathy; trigeminal neuralgia; complex regional pain syndromeMedical knowledge
Module 5: Identifying and meeting challenges
With an understanding of chronic pain terms, recognize drug-seeking behaviors consistent with either: pseudoaddiction or addictionPatient care
Recall specific office practice tools that help avert patient misbehaviors and achieve consistency in treatment of chronic nonmalignant painSystems-based practice
Identify the use for and limitations of urine drug screeningSystems-based practice
Module 6: Legal and regulatory aspects of prescribing controlled substances
Distinguish the differences between classifications of medications in the U.S. Controlled Substance scheduling systemMedical knowledge
Define the role of law enforcement authorities regarding the prescribing of controlled substancesSystems-based practice
Recognize and apply the requirements for proper and safe controlled substance prescribingProfessionalism
  • ACGME = Accreditation Council of Graduate Medical Education.

Learning Strategies (See Figures 1–3)

Figure 1

(1) Module titles are clickable navigation tools. (2) Module pretest is completed before proceeding to module content. (2.1) Case-based pretest questions encourage application and problem solving. (2.2, 2.3) Pretest question feedback emphasizes self-assessment. (3) Learners enter a 20-question posttest that applies content from module objectives.

Figure 2

(1, 1.1) Learning objectives are used as navigation tools and are paired with (2, 2.1) Accreditation Council of Graduate Medical Education (ACGME) competencies. (3) Pharm tables resource tab provides access to original tables for use in practice. (4) Tools resource tab provides access to practice tools, as illustrated by this original patient education template.

Figure 3

(1) Case-based self-assessment questions are interwoven into content. (1.1.) Feedback button provides the learner with immediate feedback. (2–6) Resource tabs allow immediate accessibility of practice resources from anywhere in the curriculum. (5,7,8) Reference links and citations provide evidence-based support for validity of content. (6) Key point summaries are presented at the end of each module and are also accessible from this tab.

Effective Website design enhances principles of active learning. For our curriculum, learners progress through the modules systematically, starting with a case-based module pretest with answer feedback. Learning objectives, paired with standard ACGME competencies, are used as navigation tools, promoting clear delineation of content purpose and emphasis of key points. Case-based self-assessment questions are woven into the text to enhance application of didactic content. Immediate feedback appears in pop-up windows to reinforce learner comprehension [33]. Tabbed resource pages collect and organize practice resources including pharmacology tables, summary tables, and tools, encouraging utilization beyond curriculum completion. Extensive references and Web links to more in-depth information support the authority of the program content and model effective evidence-based practice. These also help to keep text streamlined, letting learners decide when and if to pursue further information. To support varying learner levels from undergraduate students to practicing providers, the curriculum incorporates a tiered approach with optional advanced content. Additionally, the printable practice resources serve as didactic content while also supporting a provider's need for quick clinical references and documentation tools. Key point summaries, available after module completion, may also be viewed under tab headings. Finally, unique posttest questions reflect content from each of the 20 objectives, thus encouraging review and durable synthesis of information. A printable certificate reflects the number of correct posttest responses within each ACGME competency.



Because our needs assessment revealed that internal medicine residents were not exposed to coursework in CNMP during their medical school experience, we decided to integrate the curriculum into the ambulatory rotation of the third-year VCU School of Medicine Internal Medicine clerkship. As part of the evaluation phase, the curriculum was also piloted in 13 different graduate medical education (GME) programs in several institutions from December 2006 to June 2007. As of June 30, 2007, 439 trainees (161 medical students and 278 graduate medial residents) had completed the curriculum.

Curriculum Evaluation Strategies

During the 2006–2007 academic year, evaluation of the VCU Chronic Nonmalignant Pain Management curriculum focused on the examination of 1) ease of use of the e-learning design; 2) completeness and accuracy of online course content; 3) initial curriculum impact through qualitative and quantitative assessments; and 4) impact based on quantitative analysis of pretest and posttest scores and item analyses. The underlying assumption was that the initial evaluation data would allow focused curricular revision to positively impact learners' clinical approach to patients with CNMP through enhancement of knowledge, confidence, and utilization of practice resources.

Evaluation data were collected from a reviewer response form that prompted feedback and suggestions on the curriculum's content and design elements, a voluntary Web-based course evaluation form available after the completion of the posttest, pretest and posttest scores, and item analyses of individual pretest and posttest question responses.

Twenty-four knowledgeable physicians from 18 academic institutions were recruited as reviewers through listservs and personal contacts in pain management and medical education. Twelve physician reviewers completed the reviewer response form and 14 completed the course evaluation form. Additionally, 161 third-year medical students and 278 medical residents from 13 different GME specialities and 3 institutions completed the online curriculum (see Table 2).

View this table:
Table 2

Graduate medical resident completion

Graduate Medical ProgramNumberPercentage (%)
Internal medicine116 41.7
Family medicine 57 20.5
Psychiatry 28 10.1
Orthopedics 24  8.6
Physical medicine and rehabilitation 17  6.1
Medicine–pediatrics 13  4.7
Neurology  9  3.2
Anesthesia  7  2.5
Geriatrics  2  0.7
Rheumatology  2  0.7
Hematology/oncology  1  0.4
Infectious disease  1  0.4
Neurosurgery  1  0.4

To evaluate ease of use and completeness and accuracy of course content, physician reviewers and trainees (medical students and graduate medical residents) responded to the following two questions using a 5-point Likert scale: “Rate the ease of use of this curriculum” (1 = very difficult to 5 = very easy) and for each of the six modules “Rate the extent to which the objectives of the curriculum were met” (1 = strongly disagree to 5 = strongly agree). A repeated-measures mixed model assuming an unstructured correlation structure was used to model the mean score across the seven measures separately for each group as the response to the measures given by each person was highly correlated (r ≥ 0.4076). Comments from the reviewer response form were collected and are summarized under the Summarized Evaluation Data section below.

Additionally, comments on ease and use and completeness and accuracy of course content from the reviewer response form were collected and summarized (see “Summarized Evaluation Data”). Physician reviewer, student, and resident responses to four yes/no questions on the course evaluation form were used to provide quantitative data on initial curriculum impact. Fisher exact tests were used for each question to determine if the proportion of “yes” responses was different across the three groups.

  1. “Having completed the curriculum, will you access any of the resources?”

  2. “Is the assessment and treatment of patients with CNMP more important to you as a result of using this curriculum?”

  3. “As a result of the curriculum, will you make any changes in your behavior or practice?”

  4. “Would you recommmend this curriculum to your colleagues?”

We then analyzed student and resident pretest and posttest scores. For the student and resident responses, a repeated-measures analysis assuming an unstructured correlation structure (r = 0.13) was used to determine the following:

  1. Are the pretest to posttest score changes significant for students and residents?

  2. Are the pretest scores different between the two groups?

  3. Are the posttest scores different between the two groups?

  4. Are the pretest to posttest score changes different between the two groups?

Because the physician reviewers, most of whom were likely to be practicing pain management at a high level, were recruited primarily to provide feedback about the curriculum and to assess the accuracy of course content, pretest and posttest score analyses were performed only as pilot testing on this group. Thus, a paired t-test was used to test the changes in the pretest to posttest scores for the physician reviewers.

Item analyses of individual pretest and posttest question responses were analyzed. Difficulty and discrimination were assessed based on all the responses for each pretest and posttest item. Difficulty is defined as the percentage of correct responders (irrespective of total score). Discrimination (D) is defined as the difference in the proportion of correct responders in the upper group (upper 27% based on total score) and the proportion of correct responders in the lower group (lower 27% based on total score) [36]. For our initial item analyses, we sought a difficulty of between 45% and 75% and a discrimination of 0.3 or greater. We categorized questions as “best,” “acceptable,” “review for editing,” or “edit or eliminate” based on the description in Table 3.

View this table:
Table 3

Categorization of questions by difficulty and discrimination

CategoryDifficulty*Discrimination*Pretest Questions (20)Posttest Questions (20)All Questions (40)
Best45–75%≥0.35 (25%)6 (30%)11 (27.5%)
Acceptable45–75% 0.15–0.292 (10%)1 (5%)11 (27.5%)
76–91%≥0.32 (10%)
25–44%≥0.32 (10%)4 (20%)
Review for editing45–75%<0.158 (20%)
76–91%<0.292 (10%)6 (30%)
Edit or eliminate>24 or <91%Any7 (35%)3 (15%)10 (25%)
  • * Difficulty is defined as the percentage of correct responders, and discrimination is defined as the difference in the proportions of correct responders in the upper and lower 27th percentiles.

  • Each pretest and posttest question was a unique question.

  • To fall into any category, a question had to meet criteria of both difficulty and discrimination.

Summarized Evaluation Data

Responses from physician reviewers were overwhelmingly positive on open-ended prompts regarding curriculum design and structure. Physician reviewers found the program easy to use––mean (X) = 4.64, standard deviation (SD) = 0.63 on a 5-point Likert scale: 1 = very difficult to 5 = very easy. Student and resident mean scores were lower and similar to each other (X = 3.60, SD = 0.91; X = 3.64, SD = 0.91). Based on the results from the mixed model, the trends in mean rating across the six measures were not significantly different between the groups (P value = 0.1412). However, the physician reviewers rated the accomplishment of each of the curricular objectives much higher than did either the students or the residents (P value = 0.0001).

From the reviewer response form, physician reviewers found the six module topics to be appropriate for training. Five reviewers suggested the addition of a specific musculoskeletal module. All reviewers agreed that the online material adequately covered the stated objectives. Ninety-two percent (11/12) felt the module objectives matched the designated ACGME competency well. One respondent commented that several objectives may correlate with more than one ACGME competency. All respondents affirmed that the curriculum is evidence-based and reported consulting listed references.

The distribution by group of the “yes” responses to four yes/no “course evaluation questions” is summarized in Table 4. Students, residents, and physician reviewers gave similar percentages of “yes” responses to questions 1, 3, and 4 (P values > 0.08). There was a statistically significant difference among the groups (P value = 0.0027) in the percentage of “yes” responses to question 2: “Is the assessment and treatment of patients with CNMP more important to you as a result of using this curriculum?” The percentage reporting “yes” was greatest among students (92.4%), followed by residents (83.2%), and then physician reviewers (61.5%).

View this table:
Table 4

Responses to course evaluation questions

QuestionStudents (N = 161)*Residents (N = 278)*Reviewers (N = 14)*Fisher P Value
Percent “Yes” (Ratio)
1. Having completed the curriculum, will you access any of the resources?89.1% (131/147)83.2% (222/267)100.0% (13/13)0.0873
2. Is the assessment and treatment of patients with CNMP more important to you as a result of using this curriculum?92.4% (133/144)84.3% (225/267) 61.5% (8/13)0.0027
3. As a result of the curriculum, will you make any changes in your behavior or practice?74.7% (106/142)73.8% (197/267) 69.2% (9/13)0.8677
4. Would you recommend this curriculum to your colleagues?95.7% (134/140)90.2% (239/265)100.0% (13/13)0.1020
  • * Sample sizes for specific questions may vary due to missing data.

  • Indicates statistical significance at α = 0.05.

  • CNMP = chronic nonmalignant pain.

From the reviewer response form, physician reviewers found the practice resources valuable. Table 4 reveals that the majority of students, residents, and physician reviewers would access the practice resources again. Physician reviewers in particular commented positively on the pharmacology and summary tables as well as tools (pain assessment tools, urine drug screening tables, sample policy documents and consent forms). The patient education tool featuring a multidimensional approach to therapy was also well received. Students, residents, and physician reviewers agreed that as a result of the curriculum, they would make changes in their clinical practice. Selected comments are presented in Table 5.

View this table:
Table 5

Selected student, resident, and reviewer comments

Student Comments
I will be less hasty in making assumptions about patients and their relationship to their pain. I will be more open to treating and understanding their pain as it relates to them, rather than judging them on their past medical history.
I will be more vigilant about pain management than I likely would have without the program.
I will feel more comfortable treating patients with chronic pain and identifying those patients who misuse scheduled medications. I also feel less bias.
I will follow the guidelines outlined in this curriculum for diagnosing and treating chronic pain.
It was very helpful in understanding chronic pain. It makes working with patients less frustrating and makes me more understanding of what they are dealing with. I am not sure where I would have gotten this content in my regular curriculum and it is really important.
All of the links and tables were excellent. The pretest—the whole thing. Excellent.
Resident Comments
Now I am more aware of CNMP; I can treat my patients better without hesitating to give them proper medication.
I will take chronic pain more seriously and make every attempt to ease patient discomfort.
I will treat fibromyalgia more as a “real” condition and use a multimodal approach.
I feel more prepared to address this pathology. Helpful to have new ideas and other avenues with which to treat chronic pain.
Thank you so much for your involvement in the modules. They are extremely helpful and the embedded documents/tables I am finding to be very valuable.
Reviewer Comments
Thanks very much for sharing your work with us. The Website you've developed is impressive for its comprehensive coverage of chronic pain issues and for the resources you've gathered together in a user-friendly site.
I just finished reviewing the course and thought it was great. I liked the content, the format, the fact that you could link directly to so many things, the actual items you need to conduct a proper pain management visit were there. Also, the pretest and posttest were excellent. I would certainly endorse this for our residency program.
I reviewed the curriculum, think it is a great educational resource and look forward to getting my residency program involved. I also think the curriculum does a good job incorporating aspects of the chronic care model.
We deal with chronic nonmalignant pain all the time and are challenged. I commend you for taking on such a challenge; our residents and fellows will benefit.
  • CNMP = chronic nonmalignant pain.

Student comments suggest increased awareness of all aspects of chronic pain management and self-reported improved attitudes including being less judgmental, less skeptical, and more empathetic toward patients with chronic pain syndromes. Resident comments suggest improved knowledge of chronic pain assessment and treatment, enhanced appreciation for the complexity of chronic pain syndromes, willingness to expand treatment plans beyond a single therapeutic approach, and increased awareness of monitoring and documentation tools for patients on controlled substance medications. Examples of stated changes from physician reviewers included increased use of non-opioid pain medications for pain syndromes and increased utilization of both medication agreements and urine drug screening.

From the reviewer evaluation form, physician reviewers identified multiple user groups at all levels of training and practice as appropriate target audiences. They also identified multiple specialties and/or disciplines, including non-physician providers who may benefit from its use. All respondents agreed that if they were residency program directors, they would utilize the program for their trainees.

For the reviewers, the mean of the pretest scores was 67.14 (N = 14, SD = 9.35, range 55–85) and 70.00 (N = 13, SD = 11.37, range 50–85) for the mean posttest score, resulting in a nominal but insignificant increase of 2.69 (N = 13, 95% confidence interval = −6.08, SD = 11.47, P value = 0.5165).

On the average, students (N = 161) scored 62.3% (SD = 11.32, range = 10–85) on the pretest and 64.0% (SD = 12.83, range = 25–100) on the posttest. Residents (N = 278) scored 63.8% (SD = 11.73, range 15–95) on the pretest and 61.7% (SD = 12.74, range 25–100) on the posttest. Based on the results from the mixed model, the mean pretest score was not significantly different between the students and residents (P value = 0.1926) nor was the mean posttest score (P value = 0.0724).

While there was a nominal improvement of 1.71 (standard error [SE] = 1.27) pretest to posttest for the students, this increase was not statistically significant (P = 0.1783). The residents, however, showed a significant decrease in scores of 2.07 (SE = 0.97; P = 0.0325).

The difficulty and discrimination for each pretest and posttest item was assessed based on all question responses. These measures are summarized in Table 3: categorization of questions by difficulty and discrimination. Of the 20 pretest questions, 11 questions (55%) were classified as “best” or “acceptable,” 2 questions (10%) were classified as “review for editing,” and 7 questions (35%) were classified as “edit or eliminate.” Of the 20 posttest questions, 11 questions (55%) were classified as “best” or “acceptable,” 6 questions (30%) were classified as “review for editing,” and 3 questions (15%) were classified as “edit or eliminate.”


The development of the VCU Chronic Nonmalignant Pain Management curriculum closely followed published recommendations for developing e-learning curricula including the incorporation of changes based on evaluation data [35]. Initial evaluation data have resulted in revisions to support our curricular aims of knowledge acquisition, improved confidence in management, and utilization of practice resources.

The ease of use and course content were highly rated. Of note is the significantly greater physician reviewer rating for each of the curriculum modules. Possible explanations for this difference include the fact that the statistical power was less for this group due to comparatively low numbers, recognition of the importance of the modules based on physician reviewer experience, self-selection of reviewers through willingness to participate in the review, and the desire to provide positive feedback for a new training resource with the potential to fill a training void.

Quantitative data on curriculum impact was uniformly positive and significant. Interestingly, physician reviewers were less likely than students or residents to indicate an increase in importance in the topic of CNMP as a result of completing the curriculum. The most likely explanation for this is that these reviewers were already clinically experienced in pain management and well aware of its importance.

Initial evaluation data did not corroborate an increase in knowledge through pretest to posttest scores, as qualitative information suggested. There are several important reasons why posttest scores did not increase. First, the question writers were not question-writing experts. Item analyses have been helpful in identifying questions needing revision. Second, we consciously chose to create different questions for the pretest and posttest to illustrate numerous case-based scenarios and thus, in theory, maximize application of content. Third, the structure of the curriculum is unique; while the pretests are imbedded in each module, the posttest is completed in entirety after all six modules are finished. While data are not available on time delays, by report, many learners completed the curriculum over a course of several weeks, even as long as 2 months, which may have negatively impacted the posttest scores. Finally, the pretests and posttest are not “high-stakes” tests, meaning that the questions were written to illustrate important practice points rather than to reflect knowledge gains. The writers felt strongly that these principles would challenge the learner and emphasize application and problem solving (representative of higher-level learning) over the memorization of content [37].

Revisions Based on Evaluation Data

Changes in the curriculum have already been made as a result of the initial evaluation phase. These include technical revisions, design changes that incorporated suggestions for improving ease of use, content updates, and question revisions. From the item analyses, approximately 55% of all questions were “best” or “acceptable.” While the original questions met several criteria for effective multiple-choice questions and Web cases, they also lacked some important elements [37,38]. These were identified and questions have since been appropriately modified. Item content was modified or changed for several questions based on physician reviewer comments to remove inaccuracies and/or test its associated objective; question stems were modified to more closely conform to recommendations (complete statement, shorter length, avoidance of absolute or imprecise terms) and question options were modified to approximate the same length, remove implausible distracters, and to follow a logical or consistent order (alphabetical, for example). Because completing the 20-question posttest at the end of the curriculum may have been cumbersome for some learners (thus affecting posttest scores), we have now separated the curriculum posttest (20 items) into discrete module posttests (three to four items).

The second version of the curriculum (demonstration version accessible: http://www.paineducation.vcu.edu) is housed within an original VCU-designed learning management system, allowing program administrator access, the ability to choose specific modules based on the needs of various learner groups (students, postgraduate trainees, practicing physicians, nurses, and/or other ancillary health providers), and the ability to generate reports based on user group. Additionally, a new state-specific module supported by the Virginia Department of Health Professions has been developed (accessible: http://www.dhp.virginia.gov; click “Prescription Monitoring Program”).

Other changes may include the addition of several objectives and/or modules, notably the diagnosis and management of psychiatric comorbidities in chronic pain syndromes, an inpatient pain management module, and a pediatric pain module. Technology suggestions have included the incorporation of a search feature and the potential to download resources to a personal digital assistant.

Further evaluation is in process with a pre- and postcurriculum survey of attitudes in third-year medical students and a pre- and postcurriculum survey of attitudes and self-assessment of clinical skills in graduate medical residents. To assess the effect of the curriculum on knowledge and practice, potential evaluation strategies for the second version include delayed (3–6 months) postcurriculum knowledge assessments, postcurriculum retrospective chart review for documentation of pain assessment and pain management strategies as taught in the curriculum, and qualtitative and quantitative assessment of practice resource usage (tables, tools, references).


Developing e-learning resources should be considered for medical education topics that span disciplines, apply to various learner levels, and can be enriched by the availability of practice resources. Though this curriculum was created without any grant or pharmaceutical funding, an enormous amount of time was involved in writing content, creating resources, designing technical features, analyzing evaluation data, and integrating improvements. Thus, we hope to share this resource with other institutions to ameliorate the burden of recreating such a comprehensive curriculum.

Results of the initial evaluation of the VCU Chronic Nonmalignant Pain Management curriculum are positive and support curricular aims of improving knowledge, confidence, and access to practice resources. Additional outcome data are being collected and will help refine the curriculum by learner level and discipline.


The authors wish to acknowledge the following individuals involved in the writing, design, and evaluation of the VCU Chronic Nonmalignant Pain Management curriculum: Laura A. Morgan, PharmD; Mike F. Weaver, MD; Carl E. Wolf, PhD; Chris L. Stephens, MS; Sonya Ranson, PhD; and Semhar B. Ogbagaber, MA. The authors also wish to acknowledge the physician reviewers for the VCU Chronic Nonmalignant Pain Management curriculum.


View Abstract