What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask
(1)
,
(2)
,
(1)
,
(1)
,
(3, 4)
,
(5)
,
(1)
,
(6)
,
(7)
,
(8)
,
(1)
,
(9)
,
(1)
,
(10)
,
(11)
,
(1)
,
(1, 12)
,
(13)
,
(14)
,
(15)
,
(15)
,
(11)
,
(1, 12)
,
(16, 17, 18)
,
(13)
,
(19)
,
(1)
,
(20)
,
(8)
,
(15)
,
(21)
,
(1, 22)
,
(23, 11)
,
(24)
,
(6)
,
(24)
,
(1, 12)
,
(1)
,
(1)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Isaac Kohane
- Function : Author
- PersonId : 792802
- ORCID : 0000-0003-2192-5160
- IdRef : 070058687
Bruce Aronow
- Function : Author
- PersonId : 801155
- ORCID : 0000-0001-5109-6514
Paul Avillach
- Function : Author
- PersonId : 803243
- ORCID : 0000-0002-0235-7543
Brett Beaulieu-Jones
- Function : Author
- PersonId : 818062
- ORCID : 0000-0002-6700-1468
Riccardo Bellazzi
- Function : Author
- PersonId : 763117
- ORCID : 0000-0002-6974-9808
- IdRef : 204354269
Robert Bradford
- Function : Author
- PersonId : 802112
- ORCID : 0000-0003-0908-1428
Gabriel Brat
- Function : Author
- PersonId : 818068
- ORCID : 0000-0003-3928-5931
Mario Cannataro
- Function : Author
- PersonId : 773116
- ORCID : 0000-0003-1502-2387
James Cimino
- Function : Author
- PersonId : 802106
- ORCID : 0000-0003-4101-1622
- IdRef : 106980068
Noelia García-Barrio
- Function : Author
- PersonId : 818071
- ORCID : 0000-0002-2789-8426
Nils Gehlenborg
- Function : Author
- PersonId : 802104
- ORCID : 0000-0003-0327-8297
Marzyeh Ghassemi
- Function : Author
- PersonId : 818106
- ORCID : 0000-0001-6349-7251
Alba Gutiérrez-Sacristán
- Function : Author
- PersonId : 818061
- ORCID : 0000-0002-1245-198X
David Hanauer
- Function : Author
- PersonId : 818072
- ORCID : 0000-0001-6931-3791
John Holmes
- Function : Author
- PersonId : 794711
- ORCID : 0000-0003-2167-3602
Chuan Hong
- Function : Author
- PersonId : 792801
- ORCID : 0000-0001-7056-9559
Jeffrey Klann
- Function : Author
- PersonId : 818075
- ORCID : 0000-0003-2043-1601
Ne Hooi Will Loh
- Function : Author
- PersonId : 817815
- ORCID : 0000-0002-4114-1286
Yuan Luo
- Function : Author
- PersonId : 795668
- ORCID : 0000-0003-0195-7456
Kenneth Mandl
- Function : Author
- PersonId : 818079
- ORCID : 0000-0002-9781-0477
Mohamad Daniar
- Function : Author
- PersonId : 802113
- ORCID : 0000-0001-9031-0835
Jason Moore
- Function : Author
- PersonId : 807955
- ORCID : 0000-0002-5015-1099
- IdRef : 148457932
Shawn Murphy
- Function : Author
- PersonId : 792910
- ORCID : 0000-0002-1905-8806
Antoine Neuraz
- Function : Author
- PersonId : 1122536
- IdHAL : antoine-neuraz
Kee Yuan Ngiam
- Function : Author
- PersonId : 802111
- ORCID : 0000-0001-5676-2520
Gilbert Omenn
- Function : Author
- PersonId : 794562
- ORCID : 0000-0002-8976-6074
- IdRef : 077657802
Nathan Palmer
- Function : Author
- PersonId : 818081
- ORCID : 0000-0002-4361-207X
Lav Patel
- Function : Author
- PersonId : 818063
- ORCID : 0000-0002-8626-137X
Miguel Pedrera-Jiménez
- Function : Author
- PersonId : 818097
- ORCID : 0000-0003-0187-3826
Piotr Sliz
- Function : Author
- PersonId : 777645
- ORCID : 0000-0002-6522-0835
Andrew South
- Function : Author
- PersonId : 802114
- ORCID : 0000-0002-3204-4142
Amelia Li Min Tan
- Function : Author
- PersonId : 801553
- ORCID : 0000-0003-0623-6623
Deanne Taylor
- Function : Author
- PersonId : 809771
- ORCID : 0000-0002-3302-4610
Bradley Taylor
- Function : Author
- PersonId : 818107
- ORCID : 0000-0002-6414-4172
Carlo Torti
- Function : Author
- PersonId : 784550
- ORCID : 0000-0001-7631-5453
Andrew Vallejos
- Function : Author
- PersonId : 818084
- ORCID : 0000-0002-6543-5430
Kavishwar Wagholikar
- Function : Author
- PersonId : 818085
- ORCID : 0000-0002-6219-861X
Griffin Weber
- Function : Author
- PersonId : 818086
- ORCID : 0000-0002-2597-881X
Tianxi Cai
- Function : Author
- PersonId : 785101
- ORCID : 0000-0002-5379-2502
Abstract
Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.