Educational Epidemiology Applying Population-Based Design and Analytic Approaches to Study Medical Education Conducting educational research in medical schools is challenging partly be- cause interventional controlled research designs are difficult to apply. In ad- dition, strict accreditation requirements and student/faculty concerns about educational inequality reduce the flexibility needed to plan and execute edu- cational experiments. Consequently, there is a paucity of rigorous and gen- eralizable educational research to provide an evidence-guided foundation to support educational effectiveness. “Educational epidemiology,” ie, the application across the physician education continuum of observational de-
providing the best patient carepossible and, over time, the
signs (eg, cross-sectional, longitudinal, cohort, and case-control studies) and randomized experimental designs (eg, randomized controlled trials, ran- domized crossover designs), could revolutionize the conduct of research in
clinical research in evidence-based medi-
medical education. Furthermore, the creation of a comprehensive national network of educational epidemiologists could enhance collaboration and the
ies often generate testable hypotheses or
development of a strong educational research foundation.
in randomized controlled trials (RCTs). An example is the number of epidemio-logic studies on the risks and benefits as-
diovascular disease and breast cancer).
pers.4 Furthermore, difficulties exist that
quality educational research, such as lack
educational research are limited to their
Author Affiliations: Departments of Community &
cardiovascular disease or invasive breast
Family Medicine (Drs Carney, Pipas, and Stukel), Medi-cine (Drs Nierenberg and Brooks), and Pharmacology
& Toxicology (Dr Nierenberg), Dartmouth Medical
School, Hanover and Lebanon, NH; Department ofHealth Policy, Management and Evaluation, Faculty of
Medicine, University of Toronto, Toronto, Ontario (Dr
Stukel); Institute for Clinical Evaluative Sciences, Toronto(Dr Stukel); Office of the Vice President and Trea-
surer, Dartmouth College, Hanover, NH (Mr Keller).
risk of osteoporosis), other results con-
Corresponding Author: Patricia A. Carney, PhD, Dart-
mouth Medical School, 1 Medical Center Dr HB 7925,Lebanon, NH 03756 (patricia.a.carney@dartmouth
1044 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted) 2004 American Medical Association. All rights reserved.
skills will begin in July 2004,8 with the
studies are often restricted by the costs
tient satisfaction, quality of care, likeli-
previously possible. Identification and re-
ates in any such analysis. Every US medi-
tencies are the first step in a long-term
cal school also collects and classifies spe-
knowledge, clinical skills for patient care,
THE CONCEPT OF
scheduled for accreditation in 20057: (1)
EDUCATIONAL EPIDEMIOLOGY
evaluation across all alternative instruc-
end-of-course, clerkship, or rotation as-
dents and residents constitute “popula-
long-term outcomes can be identified. EDUCATIONAL EPIDEMIOLOGIC RESEARCH DESIGNS AND EXAMPLES 2004 American Medical Association. All rights reserved.
(Reprinted) JAMA, September 1, 2004—Vol 292, No. 9 1045 Cohort Studies. Cohort studies in-
with an outcome in 2 or more groups.
stratified by age. The hypothesis was not
there are 2 types of cohort studies: pro-
ample illustrates how cross-sectional re-
Longitudinal Studies. Longitudi-
potheses must satisfy thresholds for sci-
Observational Designs Cross-Sectional Studies. Cross-
els and risk of cardiovascular disease.
medicine is a study that assessed differ-
acted in 1996)16 or to adjuvant therapy.
cal students’ clinical experiences in their
a finding with clinical relevance for psy-
dents’ clinical experiences during clerk-
ships were not related to success in their
and use for clinical and educational pur-
1046 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted) 2004 American Medical Association. All rights reserved.
first controlled trials in clinical medi-
cine but also because the first 2 parts of
C a s e - C o n t r o l S t u d i e s . Case-
so they might be referred for early coun-
control studies are used for studying rare
events or diseases. They involve partici-
result in an effective “treatment.” This
tween use of diethylstilbesterol (DES) for
Experimental Designs
ciency in early pregnancy20,21 and the in-
gram and residents’ teaching skills.24 This
ing skills than residents not involved in
Randomized Crossover Designs. Randomized Controlled Trials. Ran-
great potential in educational studies.
care. A particularly interesting histori-
cine is the study by James Lind, A Trea-tise of the Scurvy in Three Parts,23 which
the “control group” after crossover. In
scurvy. In this study, 12 sailors with the
as students’ receiving failing scores on
cider per day; 25 “gutts” of elixir vit-
in this study, providing its rationale for
based on a ratio of 1 case to 2 controls.
tar; or no specific treatment. By the end
ously unidentified learning disability or
oped in controls compared with cases.
design has great potential for use in edu-
2004 American Medical Association. All rights reserved.
(Reprinted) JAMA, September 1, 2004—Vol 292, No. 9 1047
costs should generally be available.
orous assessment with adequate power.
sense of balance and fairness in a cross-
over design that is especially attractive
when studying new learning strategies.
justify the interventions to be tested.
dents, and faculty. Also, educational ex-
approach. The implications of such a trial
the top of the research hierarchy is com-
Challenges of Experimental Designs
of study is especially difficult to apply
yielded a relative risk of 0.49 (95% con-
studies may allow for an institutional re-
case-control/cohort studies. Thus, if well
consulted whenever this is a concern.
affecting power and generalizability. BUILDING AN EDUCATIONAL EPIDEMIOLOGY RESEARCH Existing Infrastructure
cation of less than 0.5,29 not dissimilar
ably detect an effect size of 0.5 with 80%
power using an ␣ level of .05 would re-
1048 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted) 2004 American Medical Association. All rights reserved.
ing for a critical review of existing weak-
serve as a central statistical coordinat-
Institutional Motivation
of existing infrastructure and that would
clinical faculty to generate patient care
discussed in this article are in place, but
either not assessed or not visible or are
Acknowledgment: We gratefully acknowledge M.
Scottie Eliassen, MS, for her important editorial con-
tions to the educational literature. Fac-
tributions to the final draft of this article. This work
was supported by the mission of the Offices of Medi-cal Education and Community-Based Education and
demic portfolios as well as raising the bar
REFERENCES 1. Boissel JP, Cucherat M, Amsallem E, et al. Getting
evidence to prescribers and patients or how to makeEBM a reality. Stud Health Technol Inform. 2003;95:
2. Writing Group for the Women’s Health Initiative (WHI) Study. Risks and benefits of estrogen plus pro-
gestin in healthy postmenopausal women: principal
results from the Women’s Health Initiative random-ized controlled trial. JAMA. 2002;288:321-333. 3. Women’s Health Initiative Steering Committee. Ef- National Commitment
fects of conjugated equine estrogen in postmeno-
pausal women with hysterectomy: the Women’s HealthInitiative randomized controlled trial. JAMA. 2004;
2004 American Medical Association. All rights reserved.
(Reprinted) JAMA, September 1, 2004—Vol 292, No. 9 1049 4. Lurie SJ. Raising the passing grade for studies of
physician preceptors: does age matter? Acad Med.
pen LD, Irby DM. Teaching the One-Minute Precep-
medical education. JAMA. 2003;290:1210-1212.
tor: a randomized controlled trial. J Gen Intern Med. 5. Elnicki DM, Cykert S, Linger B, Ogden P, Hannis 15. Quinn MJ. Cancer trends in the United States—a
M. Effectiveness of a curriculum in telephone medi-
view from Europe. J Natl Cancer Inst. 2003;95:1258-
25. Smorenburg CH, Sparrenboom A, Bontenbal M,
cine. Teach Learn Med. 1998;10:223-227.
Stoter G, Nooter K, Verweij J. Randomized cross-
6. Jekel JF, Elmore JG, Katz DL. Epidemiology, Bio- 16. Houn F, Elliot M, McCrohan JL. The Mammog-
over evaluation of body-surface area-based dosing ver-
statistics and Preventive Medicine. Philadelphia, Pa:
raphy Quality Standards Act of 1992: history and phi-
sus flat-fixed dosing of paclitaxel. J Clin Oncol. 2003;
losophy. Radiol Clin North Am. 1995;33:1059-
7. Liaison Committee on Medical Education Web site. 26. Maclure M, Mittleman MA. Should we use a case-
Available at: http://www.lcme.org/. Accessed April 29,
17. Reed VA, Jernstedt GC, Boudreau D, et al.
crossover design? Annu Rev Public Health. 2000;21:
Meeting the challenge of MSOP: comprehensive
8. United States Medical Licensing Examination Web
measurement of profiles of student characteristics
27. Linzer M, Brown JT, Fraizer LM, DeLong ER, Sie-
Site. Clinical Skills Examination. Available at: http:
across medical schools. Acad Med. 1999;74(suppl 10):
gel WC. Impact of a medical journal club on house-
//www.usmle.org/news/cse. Accessed November 11,
staff reading habits, knowledge, and critical ap-
18. Walsh CR, Larson MG, Leip EP, Vasan RS, Levy
praisal skills: a randomized control trial. JAMA. 1988;
9. Association of American Medical Colleges. Medi-
D. Serum potassium and risk of cardiovascular dis-
cal Schools Objectives Project. Available at: http:
ease: the Framingham Heart Study. Arch Intern Med. 28. Smits PBA, Verbeek JHAM, deBuisonje CD. Prob-
//www.aamc.org/meded/msop. Accessed Decem-
lem-based learning in continuing medical education:
19. McManus IC, Richards P, Winder BC, Sproston
a review of controlled evaluation studies. BMJ. 2002;
10. Accreditation Council for Graduate Medical Edu-
KA. Clinical experience, performance in final exami-
cation Outcome Project. General Competencies: mini-
nations, and learning style in medical students: pro-
29. Albanese M. Problem-based learning: why cur-
mum program requirements language. Available at:
spective study. BMJ. 1998;316:345-350.
ricula are likely to show little effect on knowledge and
http://www.acgme.org/outcome/comp/compMin
20. Engehausen DG, Schrott KM. PROGINS poly-
clinical skills. Med Educ. 2000;34:729-738.
morphism of progesterone receptor is increased in fe-
30. Torgerson CJ. Educational research and random- 11. Goodwin R. The prevalence of panic attacks in
male offspring with maternal exposure to diethylstil-
ized trials. Med Educ. 2002;36:1002-1003.
the United States: 1980-1995. J Clin Epidemiol. 2003;
besterol. Anticancer Res. 2000;20:5145-5149. 31. Torgerson CJ. Researching outcomes of educa- 21. Swan SH. Intrauterine exposure to diethylstilbes-
tional interventions: initial formation of unbiased groups
12. Eaton WW, Holzer CE, Von Korff M, et al. The
trol: long-term effects in humans. APMIS. 2000;108:
is most important issue in randomised controlled trials.
design of the Epidemiologic Catchment Area sur-
veys: the control and measurement of error. Arch Gen22. Preventive Services Task Force. Guide to Clini- 32. Concato J, Shah N, Horwitz RI. Randomized, con- cal Preventive Services: Report of the US Preventive
trolled trials, observational studies and the hierarchy
13. Wanf PS, Berglund P, Kessler RC. Recent care of Services Task Force. 2nd ed. Baltimore, Md: Williams
of research designs. N Engl J Med. 2000;342:1887-
common mental disorders in the United States: preva-
lence and conformance with evidence-based recom-
23. Lind J. A treatise of the scurvy. In: Stewart CP, 33. Cohen JJ. Compliance of the Graduation Ques-
mendations. J Gen Intern Med. 2000;15:284-292.
Guthrie D, eds. Lind’s Treatise on Scurvy: A Bicente-
tionnaire With Federal Human Subjects Require-
14. Carney PA, Poor DA, Schifferdecker KE, Gephart nary Volume. Edinburgh, Scotland: The University
ments. Washington, DC: Association of American
DS, Brooks WB, Nierenberg DW. Computer connec-
Medical Colleges; February 9, 2004. Memorandum
tivity and use among community-based primary care
24. Furney SL, Orsini AN, Orsetti KE, Stern DT, Grup-
What we have to do is to be forever curiously testingnew opinions and courting new impressions. 1050 JAMA, September 1, 2004—Vol 292, No. 9 (Reprinted) 2004 American Medical Association. All rights reserved.
The pharmacologic treatment of schizophrenia remains a for a dopamine hypothesis of antipsychotic drug activity critical component in the short- and long-term manage- and, in turn, a dopamine hypothesis of schizophrenia. ment of this disease. Considerable progress has been At the same time, further developments in establishing made in delineating different domains of this illness, the val
T h e n e w e ng l a n d j o u r na l o f m e dic i n eH e a l t h L aw, E t h i c s , a n d H u m a n R i g h t s Ethical Considerations in Studying Drug Safety — The Institute of Medicine Report Michelle M. Mello, J.D., Ph.D., Steven N. Goodman, M.D., M.H.S., Ph.D., The tumult arising from revelations of serious ment began, some argued that the evidence of the safety risks assoc