Nielsolson.us

Educational Epidemiology
Applying Population-Based Design and Analytic Approaches
to Study Medical Education
Conducting educational research in medical schools is challenging partly be-
cause interventional controlled research designs are difficult to apply. In ad-
dition, strict accreditation requirements and student/faculty concerns about
educational inequality reduce the flexibility needed to plan and execute edu-
cational experiments. Consequently, there is a paucity of rigorous and gen-
eralizable educational research to provide an evidence-guided foundation
to support educational effectiveness. “Educational epidemiology,” ie, the
application across the physician education continuum of observational de-
providing the best patient carepossible and, over time, the signs (eg, cross-sectional, longitudinal, cohort, and case-control studies) and
randomized experimental designs (eg, randomized controlled trials, ran-
domized crossover designs), could revolutionize the conduct of research in
clinical research in evidence-based medi- medical education. Furthermore, the creation of a comprehensive national
network of educational epidemiologists could enhance collaboration and the
ies often generate testable hypotheses or development of a strong educational research foundation.
in randomized controlled trials (RCTs).
An example is the number of epidemio-logic studies on the risks and benefits as- diovascular disease and breast cancer).
pers.4 Furthermore, difficulties exist that quality educational research, such as lack educational research are limited to their Author Affiliations: Departments of Community &
cardiovascular disease or invasive breast Family Medicine (Drs Carney, Pipas, and Stukel), Medi-cine (Drs Nierenberg and Brooks), and Pharmacology & Toxicology (Dr Nierenberg), Dartmouth Medical School, Hanover and Lebanon, NH; Department ofHealth Policy, Management and Evaluation, Faculty of Medicine, University of Toronto, Toronto, Ontario (Dr Stukel); Institute for Clinical Evaluative Sciences, Toronto(Dr Stukel); Office of the Vice President and Trea- surer, Dartmouth College, Hanover, NH (Mr Keller).
risk of osteoporosis), other results con- Corresponding Author: Patricia A. Carney, PhD, Dart-
mouth Medical School, 1 Medical Center Dr HB 7925,Lebanon, NH 03756 (patricia.a.carney@dartmouth 1044 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted)
2004 American Medical Association. All rights reserved.
skills will begin in July 2004,8 with the studies are often restricted by the costs tient satisfaction, quality of care, likeli- previously possible. Identification and re- ates in any such analysis. Every US medi- tencies are the first step in a long-term cal school also collects and classifies spe- knowledge, clinical skills for patient care, THE CONCEPT OF
scheduled for accreditation in 20057: (1) EDUCATIONAL
EPIDEMIOLOGY
evaluation across all alternative instruc- end-of-course, clerkship, or rotation as- dents and residents constitute “popula- long-term outcomes can be identified.
EDUCATIONAL
EPIDEMIOLOGIC RESEARCH
DESIGNS AND EXAMPLES
2004 American Medical Association. All rights reserved.
(Reprinted) JAMA, September 1, 2004—Vol 292, No. 9 1045
Cohort Studies. Cohort studies in-
with an outcome in 2 or more groups.
stratified by age. The hypothesis was not there are 2 types of cohort studies: pro- ample illustrates how cross-sectional re- Longitudinal Studies. Longitudi-
potheses must satisfy thresholds for sci- Observational Designs
Cross-Sectional Studies. Cross-
els and risk of cardiovascular disease.
medicine is a study that assessed differ- acted in 1996)16 or to adjuvant therapy.
cal students’ clinical experiences in their a finding with clinical relevance for psy- dents’ clinical experiences during clerk- ships were not related to success in their and use for clinical and educational pur- 1046 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted)
2004 American Medical Association. All rights reserved.
first controlled trials in clinical medi- cine but also because the first 2 parts of C a s e - C o n t r o l S t u d i e s . Case-
so they might be referred for early coun- control studies are used for studying rare events or diseases. They involve partici- result in an effective “treatment.” This tween use of diethylstilbesterol (DES) for Experimental Designs
ciency in early pregnancy20,21 and the in- gram and residents’ teaching skills.24 This ing skills than residents not involved in Randomized Crossover Designs.
Randomized Controlled Trials. Ran-
great potential in educational studies.
care. A particularly interesting histori- cine is the study by James Lind, A Trea- tise of the Scurvy in Three Parts,23 which the “control group” after crossover. In scurvy. In this study, 12 sailors with the as students’ receiving failing scores on cider per day; 25 “gutts” of elixir vit- in this study, providing its rationale for based on a ratio of 1 case to 2 controls.
tar; or no specific treatment. By the end ously unidentified learning disability or oped in controls compared with cases.
design has great potential for use in edu- 2004 American Medical Association. All rights reserved.
(Reprinted) JAMA, September 1, 2004—Vol 292, No. 9 1047
costs should generally be available.
orous assessment with adequate power.
sense of balance and fairness in a cross- over design that is especially attractive when studying new learning strategies.
justify the interventions to be tested.
dents, and faculty. Also, educational ex- approach. The implications of such a trial the top of the research hierarchy is com- Challenges of
Experimental Designs
of study is especially difficult to apply yielded a relative risk of 0.49 (95% con- studies may allow for an institutional re- case-control/cohort studies. Thus, if well consulted whenever this is a concern.
affecting power and generalizability.
BUILDING AN EDUCATIONAL
EPIDEMIOLOGY RESEARCH
Existing Infrastructure
cation of less than 0.5,29 not dissimilar ably detect an effect size of 0.5 with 80% power using an ␣ level of .05 would re- 1048 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted)
2004 American Medical Association. All rights reserved.
ing for a critical review of existing weak- serve as a central statistical coordinat- Institutional Motivation
of existing infrastructure and that would clinical faculty to generate patient care discussed in this article are in place, but either not assessed or not visible or are Acknowledgment: We gratefully acknowledge M.
Scottie Eliassen, MS, for her important editorial con- tions to the educational literature. Fac- tributions to the final draft of this article. This work was supported by the mission of the Offices of Medi-cal Education and Community-Based Education and demic portfolios as well as raising the bar REFERENCES
1. Boissel JP, Cucherat M, Amsallem E, et al. Getting
evidence to prescribers and patients or how to makeEBM a reality. Stud Health Technol Inform. 2003;95: 2. Writing Group for the Women’s Health Initiative
(WHI) Study. Risks and benefits of estrogen plus pro-
gestin in healthy postmenopausal women: principal results from the Women’s Health Initiative random-ized controlled trial. JAMA. 2002;288:321-333.
3. Women’s Health Initiative Steering Committee. Ef-
National Commitment
fects of conjugated equine estrogen in postmeno- pausal women with hysterectomy: the Women’s HealthInitiative randomized controlled trial. JAMA. 2004; 2004 American Medical Association. All rights reserved.
(Reprinted) JAMA, September 1, 2004—Vol 292, No. 9 1049
4. Lurie SJ. Raising the passing grade for studies of
physician preceptors: does age matter? Acad Med.
pen LD, Irby DM. Teaching the One-Minute Precep- medical education. JAMA. 2003;290:1210-1212.
tor: a randomized controlled trial. J Gen Intern Med.
5. Elnicki DM, Cykert S, Linger B, Ogden P, Hannis
15. Quinn MJ. Cancer trends in the United States—a
M. Effectiveness of a curriculum in telephone medi- view from Europe. J Natl Cancer Inst. 2003;95:1258- 25. Smorenburg CH, Sparrenboom A, Bontenbal M,
cine. Teach Learn Med. 1998;10:223-227.
Stoter G, Nooter K, Verweij J. Randomized cross- 6. Jekel JF, Elmore JG, Katz DL. Epidemiology, Bio-
16. Houn F, Elliot M, McCrohan JL. The Mammog-
over evaluation of body-surface area-based dosing ver- statistics and Preventive Medicine. Philadelphia, Pa: raphy Quality Standards Act of 1992: history and phi- sus flat-fixed dosing of paclitaxel. J Clin Oncol. 2003; losophy. Radiol Clin North Am. 1995;33:1059- 7. Liaison Committee on Medical Education Web site.
26. Maclure M, Mittleman MA. Should we use a case-
Available at: http://www.lcme.org/. Accessed April 29, 17. Reed VA, Jernstedt GC, Boudreau D, et al.
crossover design? Annu Rev Public Health. 2000;21: Meeting the challenge of MSOP: comprehensive 8. United States Medical Licensing Examination Web
measurement of profiles of student characteristics 27. Linzer M, Brown JT, Fraizer LM, DeLong ER, Sie-
Site. Clinical Skills Examination. Available at: http: across medical schools. Acad Med. 1999;74(suppl 10): gel WC. Impact of a medical journal club on house- //www.usmle.org/news/cse. Accessed November 11, staff reading habits, knowledge, and critical ap- 18. Walsh CR, Larson MG, Leip EP, Vasan RS, Levy
praisal skills: a randomized control trial. JAMA. 1988; 9. Association of American Medical Colleges. Medi-
D. Serum potassium and risk of cardiovascular dis- cal Schools Objectives Project. Available at: http: ease: the Framingham Heart Study. Arch Intern Med.
28. Smits PBA, Verbeek JHAM, deBuisonje CD. Prob-
//www.aamc.org/meded/msop. Accessed Decem- lem-based learning in continuing medical education: 19. McManus IC, Richards P, Winder BC, Sproston
a review of controlled evaluation studies. BMJ. 2002; 10. Accreditation Council for Graduate Medical Edu-
KA. Clinical experience, performance in final exami- cation Outcome Project. General Competencies: mini- nations, and learning style in medical students: pro- 29. Albanese M. Problem-based learning: why cur-
mum program requirements language. Available at: spective study. BMJ. 1998;316:345-350.
ricula are likely to show little effect on knowledge and http://www.acgme.org/outcome/comp/compMin 20. Engehausen DG, Schrott KM. PROGINS poly-
clinical skills. Med Educ. 2000;34:729-738.
morphism of progesterone receptor is increased in fe- 30. Torgerson CJ. Educational research and random-
11. Goodwin R. The prevalence of panic attacks in
male offspring with maternal exposure to diethylstil- ized trials. Med Educ. 2002;36:1002-1003.
the United States: 1980-1995. J Clin Epidemiol. 2003; besterol. Anticancer Res. 2000;20:5145-5149.
31. Torgerson CJ. Researching outcomes of educa-
21. Swan SH. Intrauterine exposure to diethylstilbes-
tional interventions: initial formation of unbiased groups 12. Eaton WW, Holzer CE, Von Korff M, et al. The
trol: long-term effects in humans. APMIS. 2000;108: is most important issue in randomised controlled trials.
design of the Epidemiologic Catchment Area sur- veys: the control and measurement of error. Arch Gen 22. Preventive Services Task Force. Guide to Clini-
32. Concato J, Shah N, Horwitz RI. Randomized, con-
cal Preventive Services: Report of the US Preventive trolled trials, observational studies and the hierarchy 13. Wanf PS, Berglund P, Kessler RC. Recent care of
Services Task Force. 2nd ed. Baltimore, Md: Williams of research designs. N Engl J Med. 2000;342:1887- common mental disorders in the United States: preva- lence and conformance with evidence-based recom- 23. Lind J. A treatise of the scurvy. In: Stewart CP,
33. Cohen JJ. Compliance of the Graduation Ques-
mendations. J Gen Intern Med. 2000;15:284-292.
Guthrie D, eds. Lind’s Treatise on Scurvy: A Bicente- tionnaire With Federal Human Subjects Require- 14. Carney PA, Poor DA, Schifferdecker KE, Gephart
nary Volume. Edinburgh, Scotland: The University ments. Washington, DC: Association of American DS, Brooks WB, Nierenberg DW. Computer connec- Medical Colleges; February 9, 2004. Memorandum tivity and use among community-based primary care 24. Furney SL, Orsini AN, Orsetti KE, Stern DT, Grup-
What we have to do is to be forever curiously testingnew opinions and courting new impressions.
1050 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted)
2004 American Medical Association. All rights reserved.

Source: http://nielsolson.us/STLHE020409/Educational%20Epidemiology_JAMA2004.pdf