Early View
PERSPECTIVES IN HOSPITAL MEDICINE
Open Access

Unlocking inpatient workload insights with electronic health record event logs

Marisha Burden MD, MBA

Corresponding Author

Marisha Burden MD, MBA

Division of Hospital Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA

Correspondence Marisha Burden, MD, MBA, Division of Hospital Medicine, University of Colorado School of Medicine, 12401 E. 17th Ave, Aurora, CO 80045, USA.

Email: [email protected]; Twitter: @marishaburden

Search for more papers by this author
Angela Keniston PhD, MSPH

Angela Keniston PhD, MSPH

Division of Hospital Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA

Search for more papers by this author
Jonathan Pell MD

Jonathan Pell MD

Division of Hospital Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA

Search for more papers by this author
Amy Yu MD

Amy Yu MD

Division of Hospital Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA

Search for more papers by this author
Liselotte Dyrbye MD

Liselotte Dyrbye MD

Division of General Internal Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA

Search for more papers by this author
Thomas Kannampallil PhD

Thomas Kannampallil PhD

Department of Anesthesiology, Washington University School of Medicine, St Louis, Missouri, USA

Institute for Informatics, Data Science and Biostatistics, Washington University School of Medicine, St Louis, Missouri, USA

Search for more papers by this author
First published: 05 May 2024

INTRODUCTION

High workloads in inpatient settings are associated with clinician burnout and have the potential to negatively impact patient care and the overall performance of organizations.1, 2 However, the workload thresholds at which these effects occur are unclear, with a notable lack of evidence-based guidance for optimizing work design in inpatient settings. Conventional measures of workload (i.e., work relative value units [wRVUs] and volume of patient encounters) are typically used to capture measures of patient-related workloads but inadequately capture the full work effort or the impact of workloads on key outcomes.3

To address the research to practice re-design gap, an emerging field offers promise: the use of electronic health record (EHR) event logs to understand how work design—which includes strategies around deriving optimal workloads and team structures—influences work patterns and, subsequently, patient outcomes.4, 5 EHR event logs represent a wealth of data, recording various aspects of clinical work activities and flow of events based on actions performed within the EHR.6 Event log data are unique as they are user-based (as opposed to clinical data that are patient-based) and capture the sequence of clinician care workflows that are typically not part of traditional medical record data elements.7 Major EHR vendors have now developed within-EHR platforms (e.g., Epic's Signal) that aggregate event log data into operational measures of clinician work (e.g., time spent on notes) making the data accessible to organizational leaders.

To date, considerable research has used event log data to quantify workload and work patterns in the outpatient setting.8-10 National organizations, such as the American Medical Association, are encouraging the use of EHR-based measures in practice transformation efforts to improve clinician well-being and reduce burnout.11 Although there has been much excitement in the outpatient space, the use of event logs in inpatient settings has been limited to a handful of studies.12-15 In this perspective, we identify EHR event log measures relevant to inpatient clinician work, describe challenges with its use, and propose innovative use cases for future work.

DEFINING THE MEASURES

In outpatient settings, Sinsky et al. identified seven core measures for assessing practice efficiency through EHR event logs10; however, inpatient clinician workflows are considerably different from outpatient settings. Recognizing the need for tailored measures to capture the unique demands and dynamics in inpatient settings, our authorship team, consisting of clinician experts in the inpatient setting and outpatient setting and experts in clinical informatics, derived potential measures for use in the field of hospital medicine (i.e., generalist, nonsurgical) given the limited prior applications of such measures in inpatient settings. Our selection process involved considering the work conducted by Sinsky et al.10 while also considering the distinct differences and challenges faced by inpatient clinicians, such as shift structures, patient encounters that often last for days/weeks (i.e., inpatient stays), communication patterns, structures of teams, interruptions, and the dynamic nature of the work environment. The proposed measures were also reviewed by a group of national experts on clinician workload. Table 1 shows the proposed measures, descriptions, nuances, challenges, and limitations for each measure. This work serves as a starting point for future research and discussion around the most salient measures using event log data and to bring awareness regarding the utility of this data that is available in all modern EHRs. Future efforts should focus on refining, standardizing, and prioritizing these measures.

Table 1. Proposed core event log measures in the inpatient setting.
Core measurea Description Nuance Challenges and limitations
Measures of time
Total EHR time Total time per 24-h day; Total time per shift (and per type of shift) by time from first sign-in to last sign-out. Ideally paired with clinician schedule data for shift length and type of shift worked. Vendor-derived measures use different time-outs for pauses in mouse/keystroke data. Non-EHR activities such as time at bedside and team meetings are not captured.
Time spent on various clinical tasks or combined tasks Minutes/hours spent in various activities per patient/per shift (i.e., documentation, data review, orders, communications, and transitions such as discharging a patient). Use to identify inefficiencies or increasing complexity of patients (i.e., increases in time required for care). Gives an understanding for complex workflows that require multiple grouped tasks. Provides a partial view of workday. There are multitask workflows like discharge that require coordinated use of data review, orders such as discharge medication reconciliation, and multidisciplinary communications, all of which must be documented. Tasks may need to be bundled as workflows.
WoW Time on EHR outside of scheduled clinical hours. Would include work on days off (i.e., nonclinical days). Requires an understanding of what hours are outside of work for a given clinician shift type (e.g., will vary for night shifts and for day shifts). Ideally pair with scheduling data and where the clinician is accessing the EHR from. In certain contexts, the attending physician may always be responsible. This measure is primarily defined for outpatient settings. Some clinicians may prefer to finish some tasks later in the day and may appreciate flexibility in their workday; thus, WoW may not always be perceived as negative.
Measures of workload
Number of notes Number of notes per day as primary author or cosigning author per shift by note type. This measure is indicative of the number of daily patient encounters and types of encounters (e.g., history and physicals, progress notes, discharge summaries). Consider attribution by cosigned, attested, primary author. Notes shared with other team members such as house staff or advanced practice providers may contribute to note count but challenging to attribute specific work duties to different team members.
Number of patient charts reviewed Number of unique patient charts accessed during a shift. Number of times a patient's chart was accessed. Could account for cross cover work where documentation or billing may not be performed but contributes to cognitive load and work during the shift. It may also indicate inefficiencies or increasing clinical care duties. Clinical leaders may access charts for quality improvement or other operational reasons and thus cross-linking with clinical schedule will be important to understand when work is clinical versus for other types of roles. Less helpful in understanding overall work but could be helpful for any work that does not require documentation or orders such as triage work. Would need to be cautious about judging this measure as an unintended consequence is clinicians feeling punished (i.e., labeled for being inefficient) for providing more care. While linking to schedules will be helpful to understand when accessing EHR is for clinical purposes, clinical work can carry over to nonclinical days.
Count of electronic communications Number of messages or notifications sent/received per shift, how they were sent/received (mobile or desktop). Includes time spent on voice phone calls when Voice over Internet Protocol (VoIP) data is captured. Includes both secure EHR text-based messaging (e.g., Secure Chat), pushed notifications to mobile devices, and clinical in-box communication. Should normalize to per shift, per patient, or per patient per day. Duplicate messages may be in the count. For example, a team consisting of a physician and advanced practice provider may be on the same messages, thus duplicating a team's number of messages received.
Responsiveness and attention measures
Response times for electronic communications Lag between message received and message accessed/replied to per shift This measure links different tasks that may have dependent delays in response times (e.g., high workloads could lead to delays in response times). Include unanswered VoIP calls if data available. One challenge with this measure is that response times can also be too fast (i.e., not tending to other duties or perhaps constantly being distracted/interrupted). Faster may not equal better.
Attention Continuous minutes while in EHR without interruption. Gives insights into the potential for focused work and links closely to interruptions (i.e., the more interruptions, the less attention). Event logs only show electronic interruptions and thus only a partial view of attention. Finding a balance between inversely related measures will be important (e.g., as response time improves, attention may decrease).
Measures for teamwork
Teamwork for all activities including orders, documentation, communications Percentage of orders, notes, and communications performed by/shared with care team members (such as APPs, residents) per shift. Understanding how teams work and time contributions is helpful for understanding optimal work design. Measures could also expand to include team familiarity (i.e., previous experience caring for patients together) and interactions. Could assess interdependent team performance through sequence analysis to understand collaboration, networks, task allocation, and time to task completion to name a few. There may be some necessary duplication of work (i.e., on the same messages). Large quantities of data may make this type of analysis more challenging and may require higher-level analytics capabilities. Defining a team will be challenging as there are many different team members that participate in the care of patients. Future work should be conducted to develop how team should be defined and in what contexts.
Care efficiency
Resource utilization Number of tests ordered (radiology, daily labs, consults) per patient. This measure assesses the utilization of healthcare resources in the context of clinician workload and work patterns. This measure will be closely linked to patient complexity and patient type; thus, cross-comparisons may be challenging.
Clinical decision support Use of order sets and clinical care pathways, best practice alerts. Increasingly, pathways and order sets are being utilized to drive evidence-based care. This measure determines the frequency and utilization of order sets and clinical pathways in practice. In some cases, following a pathway may not be the best approach. Thus, understanding when deviation from a protocol is necessary will be key. It may be challenging for organizations to keep up with guidelines and, if so, may limit the utilization of such order sets and pathways.
Use of assisted/augmented documentation Percent of documentation carried forward/copied forward; use of templates, smart phrases, dictation, ambient listening device use (if available). Copy forward/carry forward may be appropriate or may indicate an overworked workforce with less time to update notes. Could also indicate less attentiveness. The challenge will be knowing when it is appropriate to carry/copy forward and when it is not.
Documentation timeliness Timing of documentation completion. Need to pair with scheduling data. Balancing measure could be amount of notes carried/copied forward.
Modifiers
Patient complexity Measures of patient complexity such as patient turnover, observed to expected ratios, and case mix index. Patient complexity impacts each of the core measures and has implications on workload and resource utilization. Understanding when a patient requires more direct care by a clinician, whether it be medical or social aspects of care, will be important to understand. Care demands may vary whether a patient is a new patient, a follow-up visit, or discharge. Turnover of patients can impact work demands. There are potentially many modifiers. Future work needs to entail which additional modifiers to include and how to weigh different patient modifiers based on impact on workload or resource utilization. Modifiers will need additional validation and likely iterative improvement.
Location of work Work may be conducted on fixed or mobile devices on wired or wireless/Bluetooth connection allowing for tracking. Only able to capture location of work when it is performed on fixed workstations or on wireless/Bluetooth but not cellular connections. Locations of work completion may not be captured when using mobile or fixed devices on cellular connection.
  • Abbreviations: APPs, advanced practice providers; EHR, electronic health record; VoIP, voice over internet protocol; WoW, work outside of work.
  • a Adapted from Sinsky et al.10

OPPORTUNITIES FOR USE IN THE INPATIENT SETTING

With potential measures defined, the use cases for event log data in the inpatient setting are immense and include practice management, educational, and quality improvement purposes in addition to research; however, efforts in these areas outside of research settings are sparse. Within the realm of practice management, event log data have the potential to support evidence-based work design practices. As clinicians in the inpatient setting continue to face pressures to increase the volume of patient encounters,2, 16 event log-based measures paired with critical workforce, patient, and organizational outcomes may offer pragmatic practice insights. These insights could help organizational leaders understand when work design and workloads may contribute to harm. For example, if certain workload thresholds, as measured by event logs—such as number of patient encounters, secure messages sent or received, or EHR-based interruptions (e.g., alerts, messages)—are found to be associated with burnout or patient harm, then these thresholds could be monitored and proactively addressed. Early research has indicated that conducting ecological assessments through carefully timed clinician surveys can provide valuable insights into how workloads and work environments are perceived. Analyzing these data in conjunction with event log measures may help leaders to understand when workloads and work environments contribute to adverse outcomes or lead to improved outcomes.17 Additionally, insights into practice patterns such as resource utilization and team member interactions15 could be incorporated into event log measures.

In education, event log measures can provide valuable insights into trainee work patterns,18 potentially reducing the need for surveys for auditing work activities and measuring workload. Measures such as total EHR time, time from first sign-in to last log-off, and location of the sign-in could signal shifts in work patterns or identify when trainees exceed work-hour thresholds. Additionally, EHR event logs can be utilized for phenotyping and identifying clinicians who may be struggling with certain activities of the work day such as time spent completing documentation or orders19 and assessing clinical performance outcomes.20 Understanding the measures that are indicative of struggling learners can also be important. Sebok-Syer et al. have highlighted the importance of understanding how EHR-based measures incorporate the impact of team dynamics on individual performance, as well as the reciprocal influence of the individual on the performance of a team.21 Additionally, given that organizations are also increasingly utilizing secure electronic messaging platforms (e.g., Epic Secure Chat), large language models in the future could be utilized to analyze potential knowledge gaps or lapses in professionalism.

In the process improvement domain, event log-based measures may be able to provide valuable insights into care coordination processes, particularly during critical transitions of care. For example, within the context of unexpected transfers to an intensive care unit, clinician leaders could utilize event logs to evaluate care workflows and patterns before the transfer to evaluate chart access, communication patterns, and interactions to assess opportunities to optimize care coordination. There is also an opportunity to explore how workload and work patterns drive cognitive14 and diagnostic errors.5 Innovations in this space include utilizing patterns of orders such as the retract-and-reorder (an indicator of wrong-patient ordering errors)22, which could serve as a measure of cognitive error. Pairing these types of orders with event log measures such as workloads or measures of attention could help operational leaders to understand when work design may be contributing to these types of errors.14

CHALLENGES

Although there are many opportunities with event log measures, there are also several challenges. As measures are developed, rigorous validation techniques must be conducted to ensure accuracy and generalizability. As organizational leaders gain access to this data, considerable work will also need to be conducted to build theories and frameworks on how to interpret the data and associated measures. For example, a measure such as work outside of work (WoW) requires contextual interpretation. A high WoW may mean work inefficiencies and/or work overload; in contrast, it could also mean competing work-life demands (e.g., an inpatient clinician may choose to do charting at home to pick children up in a timely fashion, therefore choosing to do WoW) or it could be an adaptive response to fight burnout. Furthermore, understanding the associations with outcomes will be key. Merely defining and assessing these measures will be insufficient and additional work will be needed to understand the significance and the contextual implications of using these measures. If an issue is identified, it will be imperative to collaborate with individuals or groups where the problem is presumed to exist, as there is a risk that this data can be misinterpreted. Additionally, attribution of outcomes to individual clinicians can also be challenging with some work to define best practices already conducted.23, 24

Although all hospitals with EHRs have access to event logs, only 53% of hospitals reported using EHR data to track clinician time.19 Insights gained (as well as access to event log data) may not trickle down to practice management leaders. Granular event log data also exists (i.e., unaggregated data) and requires considerable data processing and analytics capabilities, which may be challenging for many clinical practices. Although EHR vendors have developed platforms with higher-level reports with summaries of the data, vendors' data aggregation practices have not been standardized, rendering generalizations across organizations with different EHR vendors challenging. Different types of clinical work will influence clinician's EHR use and, thus, the patterns seen in event log data. We have primarily focused on hospitalists' work; however, additional considerations should be pursued to understand how proceduralist or higher acuity services may impact patterns in the data. Thus, measure validation will be an important next step.

Next, inpatient work is also challenging from both a scheduling and attribution perspective, complicating the process of linking event log measures to outcomes. Event logs, in theory, are highly attributable to individual clinicians; however, the data are also impacted by patient complexity, the multidisciplinary nature of inpatient work, and systems-related factors. Next, event log measures present several considerations within the frameworks of Campbell's (i.e., once a measure is used for decision-making, the more it will be subject to corruption pressures, and the processes it is intended to monitor may be undermined)25 and Goodhart's Laws (i.e., once a measure becomes a target it is no longer a good measure).26 Similarly, healthcare leaders with pressures for both financial performance and productivity targets may use these measures to attempt to boost productivity without considering the context for the data (i.e., increasing workloads when electronic measures of workload are perceived to be low). Finally, the concept of surveillance, while commonplace in many occupations (e.g., trucking industry27), is now increasingly prevalent in healthcare workplace settings and must be conducted thoughtfully.

THE FUTURE: PROACTIVE AND PREDICTIVE

Looking ahead, leveraging EHR event log measures offers the potential for optimizing work design to improve outcomes for clinicians, patients, and organizations. Organizations must develop prerequisites to integrate EHR measures into practice. This involves developing best practices around how event log measures will be used, ensuring data privacy, and emphasizing that interpretation of the data requires some caution and knowledge. Furthermore, fostering a culture of psychological safety will be paramount to ensure individuals feel comfortable sharing insights and concerns arising from this data. Collaboration across disciplines will be needed, particularly when making decisions informed by this data. In summary, EHR event log measures represent a significant opportunity to utilize data collected during routine clinical work to begin to understand how work design impacts clinicians, patient care, and organizational outcomes.

ACKNOWLEDGMENTS

Dr.'s Burden and Keniston report funding from the Centers for Disease Control and Prevention (CDC), the National Institute for Occupational Safety and Health (NIOSH), the Center for Health, Work, and Environment (CHWE), a NIOSH Center of Excellence for Total Worker Health, and the Agency for Healthcare Quality and Research. Dr.'s Burden, Keniston, and Dyrbye report funding from the American Medical Association. Dr.'s Burden and Keniston have a trademark for a workforce application, GrittyWork. Dr. Dyrbye reports funding from the National Science Foundation (NSF) award under grant award number 2041339 and the National Institute of Nursing Research (NINR) grant award number R01NR020362-01. Dr. Dyrbye reported receiving royalties from CWS, Inc., for the Well-being Index licensed by the Mayo Clinic outside the submitted work. Dr. Kannampallil reports funding from the Agency for Healthcare Research and Quality (AHRQ), the National Library of Medicine (NLM), and the National Institute on Aging (NIA). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the AHRQ, NLM, NIA, CWS, NSF, NINR, or NIOSH. The authors utilized the ChatGPT language model (version 3.5 and 4.0) developed by OpenAI for editing of original author content to improve readability. All information and materials in the manuscript are original. This work was funded in part by grant (1R01HS29020) from the Agency for Healthcare Research and Quality (AHRQ) as well as by an EHR Use Research Grant from the American Medical Association. The views expressed in this paper do not reflect those of the funding agencies. They had no role in writing or reviewing the work.

    CONFLICT OF INTEREST STATEMENT

    The authors declare no conflict of interest.