Clinical Decision Support for Emergency Department Nursing Discharge Pain Reassessment


Gold, D., Hicks, J., Macheska, J., Mason, P., & McLaughlin, P. (November, 2018). Clinical decision support for emergency nursing discharge pain reassessment. Online Journal of Nursing Informatics (OJNI), 22 (3), Available at


A well-designed electronic medical record (EMR) system incorporates functionality resulting in many benefits for patients, healthcare providers, and healthcare organizations. These benefits include improved efficiency, reliability, and data accessibility, ultimately contributing to greater patient safety and reduced healthcare delivery costs. The application of clinical decision support (CDS) best practices improves documentation compliance and patient care and prevents common pitfalls related to less-effective CDS functionality, such as poor user experience and alert fatigue resulting in ignored messages. This article presents a literature review to explore CDS implementation best practices, details the development of CDS functionality to improve emergency department pain reassessment documentation, and presents quantifiable evidence to show documentation compliance improvement after implementation without incorporating “hard stop” mandatory completion functionality. Incorporating evidence- based design principles into current and future CDS projects may result in improved patient care quality, outcomes, and compliance with treatment and documentation-based initiatives for clinicians of all levels, including nurses.

Themes: Literature review and performance improvement project related to clinical decision support improving nursing documentation of pain reassessment prior to discharge from the emergency department without the use of a “hard stop” workflow


A well-designed electronic medical record (EMR) system incorporates functionality to promote improved efficiency, reliability, and data accessibility. Most systems also include clinical decision support (CDS) engineered to assist with the clinical decision- making process by analyzing characteristics of individual patients and generating patient-specific recommendations, which are presented to the end-user in the form of an alert, reminder, order set, drug-dose calculation, or care summary dashboard to display patient care quality compared to established quality indicators (Mahabee-Gittens, Dexheimer, & Gordon, 2016).  Furthermore, the federal meaningful use incentive program rewards compliance with progressive system functionality, including criteria related to CDS. The reality is, however, that 49-96% of provider “pop-up” alerts are overridden; the recipient of a potentially important patient care-related alert ignores the message by selecting an option to dismiss the alert without acting to mitigate the issue that triggered the alert (Anker, Edwards, Nosal, Hauser, Mauer, & Kaushal, 2017). These overrides, while occasionally justified, could be associated with medication errors and other adverse events, potentially resulting in patient mortality. One common solution is the creation of a documentation “hard stop,” which prevents users from proceeding unless a specified element is immediately and appropriately addressed; the consequence of this method is often poor end- user experience. The purpose of this article is to present a literature review of CDS best practices and to demonstrate how to create a system that does not involve alerts or “hard stops” and that results in improved compliance with patient pain reassessment documentation initiatives in emergency departments (ED).  

Literature Review

Anker et al. (2017) identified phenomenon related to repeated, inappropriate bypassing of decision support alerts by the clinician as alert fatigue, and suggested the root cause could be related to cognitive overload, desensitization, or a combination of these two factors. Cognitive overload occurs when a clinician receives a large quantity of information, and he/she is unable to process this information due to time limitations or because the end user does not have related clinical information to determine relevance of the alert content. In the case of cognitive overload, the end users are presented with information; however, they lack the ability to process that information and convert it to an actionable or relevant clinical decision.
Desensitization, a nother potential cause of alert fatigue, results from repeated exposure to alert messages or CDS information that contribute to decreased end user responsiveness. The desensitization model suggests that an alert is most effective when first received and becomes less effective as it is regenerated over time (Simpao, Ahumada, Desai, Bonafide, Gálvez, Rehman, & Shelov, 2015). Therefore, even clinically relevant alerts may contribute to alert fatigue; an end user receiving multiple alerts during each interaction with the EMR could eventually become fatigued. Research shows that alert override rates decrease after potentially clinically irrelevant alerts are identified and removed (Simpao et al.,2015). Also, providers report that their ability to recall alerts diminished when a large number of alerts were communicated (Baseman, Revere, Painter, Mariko, Thiede, & Duchin, 2013).

Clinical decision support must provide the right information to the right person, for the right patient, in the right place, at the right time. While achieving this goal may seem easy at first , many variables must be considered when designing and implementing a CDS system. Design functionality must be integrated within the environment of the healthcare setting in a way that does not disrupt clinician workflow (Marcy, Kaplan, Connolly, Michel, Shiffman, & Flynn, 2008). There often exists an opinion disparity between the developer of the EMR decision- support system and the end user (Kaipio, Lääveri, Hyppönen, Vainiomäki, Reponen, Kushniruk, & Vänskä, 2017). Physicians and nurses have high expectations regarding many aspects of system usability: system response speed to input data, system stability or up-time, and perceived impact on patient care. Functionality must not only work properly but must be designed to optimize workflow. Furthermore, the clinical impact or purpose must be realized by the clinical staff to fully engage end users and avoid alert fatigue. The lower the perceived clinical relevance of the alert, the more likely alert fatigue will manifest within the end- user group (deWit, Mestres Gonzalvo, Cardenas, Derijks, Janknegt, van der Kuy, & Schols, 2015).

When designing or evaluating a CDS system, the speed at which aggregated data is queried and presented should be considered. A well- designed system should operate quickly; the database query and associated alert or feedback generation must not impede the ability of a provider to quickly navigate the chart (Wright, & Robicsek, 2015). This is accomplished by using r elational d atabase programming best practices to optimize database query rates (Sharma & Nelson, 2017).
A CDS project should ideally avoid the use of hard stop alerts, which prevent end users from proceeding unless a specific action is taken. A hard stop alert prevents, or is perceived to prevent, clinical staff members from doing their jobs by not allowing users to proceed within the electronic medical record until they comply with the alert instructions. T he alerts should offer alternatives within the CDS workflow rather than causing a complete productivity barrier (Mahabee-Gittens , Dexheimer & Gordon, 2016).

Information display output format is also important, as alerts are typically less effective if the information is presented to the end user in an illogical or unusable manner. The information should include clinically relevant data when possible to prevent the need to search throughout the EMR for supporting data (Slager, Weir, Kim, Mostafa, & Del Fiol, 2017). Usability testing can determine if users of a system agree that CDS works well to assist in patient care rather than impeding the efforts of clinicians . Mahabee-Gittens et al. (2016) used an observational, think-aloud protocol that allowed clinicians to describe exactly what they were doing and why while performing typical activities within a system. A written survey was also used to gather data from a large sample population.

To improve compliance with CDS systems, a procedure should be developed to measure and share success with staff. Providing feedback to end users provides reinforcement to high performers and informs those who have a high level of alert overrides to consider adjusting their practice and system utilization workflow (Mahabee-Gittens et al., 2016).  Furthermore, feedback builds trust within the system and promotes end user engagement. Poor performers are often not aware that their actions are not commensurate with expectations or with the majority of their colleagues (Kim, Chiu, & Bregant, 2015). Kim et al. (2015) refer to this phenomenon as the unskilled and unaware effect; it is thought to relate to either the poor performer’s lack of metacognitive ability to realize sub-par performance, or perhaps to the poor performer’s motivation to ignore the performance to avoid feeling inferior.

Project Background

A best practice for ED nurses is the provision of pain reassessment and documenting these findings prior to patient discharge to establish that adequate pain management had occurred during the visit. To perform this reassessment, the ED nurse must select an appropriate numeric pain assessment tool according to the patient’s age and ability to easily answer questions regarding pain level. A system query was performed (prior to any documentation system changes) that showed that less than half of discharged patients’ charts included a documented pain reassessment during ED visits. Educational efforts were provided at daily staff huddles to encourage compliance; however, no significant pain reassessment documentation improvements were realized. Several EMR modifications were then considered, including a “pop-up” alert if the nurse attempted to discharge the patient and a pain reassessment was not documented.  Concerns were raised that this alert would be ignored by clinical staff and therefore might not have the desired effect to improve documentation compliance. 

Applying CDS best practices, the ED team applied modifications to the ED workflow form, which is completed for every patient who is discharged from the department. New fields were placed on the form to allow the discharge nurse to document a pain assessment at the time of discharge. A dynamic text area was also placed on the form (directly above the new documentation fields), causing an automatically query of the system when the discharge process is initiated to show the most recent pain assessment, the time of that assessment, and any interventions that were performed for the patient during an ED visit. P roviding this information to the clinician at the time the documentation is expected, supports the user to provide both the information to help make the assessment and a location to document findings. The previous-state workflow involved closing the depart form and then navigating to the pain documentation area of the patient’s chart to document pain reassessment; once pain was documented, the user then was required to re-open the form and proceed with the patient discharge.

Implementation and Data Analysis

Modifications were made to the Emergency Department Depart Form in December 2017, including new fields to document pain reassessment and the dynamic text area to display pertinent clinical assessment and treatment information. None of these fields are set for mandatory completion; the end user may proceed without completing this documentation:

Figure 1: ED Discharge Pain Assessment screen

If the clinician selects “Yes actual or suspected pain” in the “Is patient having pain at discharge? ” field, another window opens to allow documentation of the actual pain level using one of several applicable pain rating scales. The box titled “Latest Pain Assessment ” contains the dynamic text produced by the system according to the most recent pain assessment and pain interventions performed during the ED visit. Design resource requirements were minimal; one informatics clinical analyst was assigned to this project and worked with ED management to perfect the content and appearance intermittently over a period of three weeks prior to go-live.

Education was provided at daily huddles for one week to ensure that all staff members were aware of the new form section. Automated retrospective chart review was performed using a report query to aggregate data for a period of six months before the implementation, and then again for five months after the go-live. An immediate improvement in discharge pain reassessment compliance was noted after implementation, as seen in the ED pain assessment/reassessment report results table (see Table 1 and Figure 2 ).

Table 1: Emergency Department Pain Reassessment Chart Abstraction Data
Figure 2 : Emergency Department Pain Reassessment Chart Abstraction Trends

Data review revealed that compliance with the pain reassessment documentation initiative improved from 42.1% before the implementation of the CDS tool to 68.6% after implementation.  Furthermore, documentation compliance was sustained, reaching 80.3% five months after implementation. It is significant to note that this improvement did not require utilization of a hard stop to prevent further tasks until the task at hand was completed; these results were achieved by using CDS to provide the clinical staff with the required information, in a usable format, presented at the optimal time with a convenient mechanism to immediately document findings without breaking workflow. 

Another interesting data observation is that compliance continues to steadily increase from the date of implementation. This improvement could be attributed to clinicians becoming accustomed to the workflow change, or possibly due to positive feedback they received regarding overall staff compliance with the initiative and the importance of proper documentation of pain reassessment. Further data trend analysis would be helpful to determine if a plateau is ultimately reached, or if compliance diminishes after a period of time, especially if positive reinforcement is not provided to staff .


Many examples of current-state CDS systems are designed to assist physicians and provide pharmacy-related alerts to ensure compliance with regulatory and reimbursement standards.  There are many opportunities for CDS functionality that could benefit nurses and other healthcare providers. By incorporating evidence- based design principles into current and future CDS design projects, improved quality, outcomes and compliance with treatment and documentation initiatives are likely. The project presented uses a dynamic text field to present critical information to the nurse . The text field also provides a place to document pain reassessment findings without disrupting the established workflow. Data show a significant improvement in documentation practices for pain reassessment, despite the absence of a hard stop, which forces documentation by preventing other activities if documentation is not immediately completed.  Clinicians typically have the choice whether or not to follow best practices, and when provided with a system that supports rather than impedes their workflow, they are more likely to comply.


Anker, J.S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., Kaushal, R. (2017). Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making, 17, 36. http:// doi: 10.1186/s12911-017-0430-8
Baseman, J. G., Revere, D., Painter, I., Mariko, T., Thiede, H., & Duchin, J. (2013). Public health communications and alert fatigue. BMC Health Services Research, 13(1), 1. doi:10.1186/1472-6963-13-295
deWit, H. A., Mestres Gonzalvo, C., Cardenas, J., Derijks, H. J., Janknegt, R., van der Kuy, P. M., & Schols, J. M. (2015). Evaluation of clinical rules in a standalone pharmacy based clinical decision support system for hospitalized and nursing home patients. International Journal o f Medical Informatics, 84(6), 396-405 . doi:10.1016/j.ijmedinf.2015.02.004
Kaipio, J., Lääveri, T., Hyppönen, H., Vainiomäki, S., Reponen, J., Kushniruk, A., & ... Vänskä, J. (2017). Usability problems do not heal by themselves: National survey on physicians’ experiences with EHRs in Finland. International Journal o f Medical Informatics, 97, 266-281. doi:10.1016/j.ijmedinf.2016.10.010
Kim, Y., Chiu, C., & Bregant, J. (2015). Unskilled and don't want to be aware of it: the effect of self-relevance on the unskilled and unaware phenomenon. Plos ONE, 10(6), 1-9. doi:10.1371/journal.pone.0130309
Mahabee-Gittens, E. M., Dexheimer, J. W., & Gordon, J. S. (2016). Development of a tobacco cessation clinical decision support system for pediatric emergency nurses. Computers, Informatics, Nursing: CIN, 34(12), 560-569.
Marcy, T. W., Kaplan, B., Connolly, S. W., Michel, G., Shiffman, R. N., & Flynn, B. S. (2008). Developing a decision support system for tobacco use counselling using primary care physicians. Informatics i n Primary Care, 16(2), 101-109.
Sharma, H. K., & Nelson, S. C. (2017). Performance enhancement using SQL statement tuning approach. Database Systems Journal, 8(1), 12.
Simpao, A. F., Ahumada, L. M., Desai, B. R., Bonafide, C. P., Gálvez, J. A., Rehman, M. A., & Shelov, E. D. (2015). Optimization of drug-drug interaction alert rules in a pediatric hospital's electronic health record system using a visual analytics dashboard. Journal o f t he American Medical Informatics Association, 22(2), 361-369. doi:10.1136/amiajnl-2013-002538
Slager, S. L., Weir, C. R., Kim, H., Mostafa, J., & Del Fiol, G. (2017). Physicians’ perception of alternative displays of clinical research evidence for clinical decision support – a study with case vignettes. Journal o f Biomedical Informatics, 71(Supplement), S53-S59. doi:10.1016/j.jbi.2017.01.007
Wright, M., & Robicsek, A. (2015). Special article: c linical decision support systems and infection prevention: t o know is not enough. AJIC: American Journal o f Infection Control, 43(6), 554-8 . doi:10.1016/j.ajic.2015.02.004

Author Bios:

David Gold, MSN, RN, CEN, CCRN, is currently the director of emergency management and nurse manager for quality and performance improvement in the Emergency Department at Newark Beth Israel Medical Center, Newark, New Jersey; current responsibilities include monitoring and improving Emergency Department patient care by optimizing operational workflow. David graduated from the Middlesex County College (CEG) School of Nursing in 2000, received his B achelor of science in nursing from Excelsior College School of Nursing in 2015, and graduated with his M aster of science in nursing from Excelsior College School of Nursing in 2018.

Jamie Hicks, MSN, RN, RN-BC, CEN, works as the Emergency Department e ducator at Newark Beth Israel Medical Center in Newark, New Jersey. In this role for seven years, he has taught courses in basic cardiac dysrhythmia , IVs , general and ED- specific competencies, and verbal/physical de-escalation. He is a certified emergency nurse and board certified as a nursing professional development specialist.  Jamie received his B achelor of science in nursing from Georgia State University in 2003 and his M aster of science in nursing education from Rutgers University in 2015.

Jennifer Macheska, BSN, RN, graduated from Mount Saint Mary College in 2009 with a B achelor of science in nursing degree. After graduation, she accepted a job as a staff nurse at Newark Beth Israel Medical Center in the Emergency Department. Jennifer currently works as the nurse manager in the Newark Beth Israel Medical Center Emergency Department, supervising operations for the adult and pediatric ED with a combined annual volume of 90,000 patients.

Patricia Mason, MBA, BSN, RN, is currently the assistant vice president of emergency services at Newark Bath Israel Medical Center in Newark, New Jersey. In this role, she oversees the clinical, operations, and administrative management of the Emergency Department. Patricia received her B achelor of science in nursing in 2014, and M aster of business administration degree from Thomas Edison State University in New Jersey in 2018.

Patricia McLaughlin, MSN, MPA, RN, RNC-OB, RN-BC, currently works as a clinical analyst within the Robert Wood Johnson Barnabas Health (New Jersey) Corporate Informatics D epartment, maintaining and developing clinical information systems. She graduated from Monmouth University with her B achelor of science in nursing in 1986, M aster of public administration from Fairleigh Dickinson University in 1999, and M aster of science in nursing from the University of Phoenix in 2004.