By Jaana K. Junttila, Aija Koivu, Kaisa Haatainen, and Pirkko Nykänen
CITATION: Junttila, J.K., Koivu, A., Haatainen, K. & Haatainen, P. (Fall, 2019). Nurse-Assessed Optimality of Workload - A Valid Measure for the Adequacy of Nursing Resources? Online Journal of Nursing Informatics (OJNI), 23(3). Available at http://www.himss.org/ojni
Aim: To examine whether a valid measure for the adequacy of nursing resources can be based on nurse-assessed optimality of the workload on nursing intensity and staffing systems.
Background: In health care, nurse leaders need valid information regarding the adequacy of nursing resources. Earlier research suggested that nursing resources predict patient outcomes.
Method: In a cross-sectional retrospective observational study consisting of 34 inpatient units from two acute care hospitals in 2012 and 2013 (n = 732 monthly reports), we investigated whether the optimality level of the workload determined by the Nursing Intensity and Staffing System RAFAELA® predicted the number of safety incidents.
Results: A clear increasing trend in the number of safety reports with an increasing nursing workload level was found only in units where the patient-to-nurse ratio was low and patient dependency was high.
Conclusions: In this study, additional validity of the RAFAELA® system was partially confirmed using patient safety incident data. However, underreporting of patient safety incidents may have influenced the results.
Implications for Nursing Management: Engaging nurses in analysing daily variations in the adequacy of nursing resources enables nurse leaders to recognize unit-specific vulnerabilities, to identify ways to prevent work overload, and to safeguard quality care in times of inevitable high workload.
It has been argued that the future sustainability of health systems will depend on the use of smart information techniques to optimize the deployment of human resources (Tursunbayeva, Bunduchi, Franco, & Pagliari, 2016). However, evidence-based information management still faces many challenges in nursing. Both nurses and nurse leaders often doubt the validity and reliability of statistics based on the data collected routinely on their units. Nurses may ask, “Do indicators of the nursing workload correctly reflect the actual burden of nursing work?” “Does documentation completed in a hurry meet the criteria for reliable and accurate measurements?”
This study addresses these justifiable questions by investigating the validity of the nursing intensity and staffing system RAFAELA® used in the daily documentation of nursing work in Finland. This study also renders additional validity to a previous study (Junttila, Koivu, Fagerström, Haatainen, & Nykänen, 2016), which confirmed that hospital mortality, which is an exact patient outcome, can be predicted by RAFAELA®. In this study, we used nurse-reported, voluntary-based patient safety reports.
At the hospital level, the most common measure for the nursing workload, the patient-to (registered)-nurse ratio, has been shown to predict both patient and nurse outcomes (Aiken et al., 2014). At the hospital unit level, however, the patient-to-nurse ratio does not definitively reflect the nursing workload (Needleman, 2015; Twigg, Gelder, & Myers, 2015). Scientific evidence for patient, nurse and organizational factors that affect nursing workload provides further indicators for unit staffing levels, e.g., average length of stay, average number of medication doses administered per day, percentage of patients 70 years or older, percentage of patients with a body mass index ≥ 25, top three diagnostic categories on the unit, average daily census, and daily patient turnover (Fasoli, Fincke, & Haddock, 2011). However, as MacPhee, Dahinten, & Havaei (2017) have argued, these quantitative measures exclude complex, invisible aspects of nurses’ work that can only be gauged by the nurses themselves. There is ample evidence that nurses’ perceptions of the adequacy of resources and the quality of care predict patient and nurse outcomes.
Patient classification systems (PCS) define the need for nursing resources through the identification and quantification of an individual patient’s care needs that relate to the acuity/severity of illness, dependency of the patient, or required intensity of nursing activities. The non-commercial PCS used in Finnish hospitals since the late 1990s, the RAFAELA Nursing Intensity and Staffing System®is built upon nurses’ professional judgement in both assessing patient care needs and setting the standard for the optimal nursing workload of the unit (Rauhala, 2008).
It is important to involve nurses in designing and maintaining the information system that measures their workload (Fasoli et al., 2011). However, whether a valid and reliable PCS can be based on nurses’ subjective assessments is questionable. Some authors have drawn attention to difficulties in making PCSs acceptable because they tend to increase nurses’ workload and have advocated simple and objective indicators of the nursing workload (Fasoli & Haddock, 2010). Indeed, to maintain the validity and reliability of PCS instruments, several procedures must be regularly implemented that aim to guarantee uniform practice and the quality of patient classifications. The feasibility, internal validity and reliability of the RAFAELA®have been studied intensively (Andersen, Lønning, & Fagerström, 2014; Flo, Landmark, Hatlevik, & Fagerström, 2018; Liljamo, Kinnunen, & Saranto, 2018; Rauta, Salanterä, Vahlberg, & Junttila, 2017; van Oostveen, Mathijssen, & Vermeulen, 2015). However, there is still limited scientific evidence on the predictive validity of the RAFAELA®, e.g., its relationship to estimates of quality of care (Fagerström, Kinnunen, & Saarela, 2018). Junttila et al., (2016) found a link between mortality and nursing workload (NWL) based on binomial regression analysis. Compared to the incidence rate of death in the months of overstaffing when average daily nursing workload was below the optimal level, the incidence rate was nearly fivefold when average daily nursing workload was at the optimal level (IRR 4.79, 95% CI 1.57-14.67, p=0.006) and 13-fold in the months of understaffing when average daily nursing workload was above the optimal level (IRR 12.97, 95% CI 2.86-58.88, p=0.001).
A basic characteristic of high-quality care is patient safety (Wolf & Hughes, 2008). Improving patient safety means avoiding and reducing incidents that cause patient harm. Many studies have shown that greater nursing workloads are associated with negative patient outcomes (Aiken et al., 2018; S. Cho, Mark, Knafl, Chang, & Yoon, 2017; Junttila et al., 2016). The quality and safety of patient care may be compromised if nurses have workloads that are too heavy and essential tasks are left undone (MacPhee et al., 2017). An increased workload has been linked to falls (Carlesi, Padilha, Toffoletto, Henriquez-Roldn, & Juan, 2017; Cho et al., 2015), medication errors, urinary tract infections and pressure ulcers (MacPhee et al., 2017), as well as perceived adverse patient outcomes and complaints from patients and families (Al-Kandari & Thomas, 2009).
In scientific research, patient harm has been studied by collecting data with questionnaires and asking patient outcome-related questions (Cho et al., 2015), from patient records (Carlesi et al., 2017) or by direct observation (Wolf & Hughes, 2008). In most hospitals, there are mandatory reporting systems with a primary purpose of holding providers accountable for errors associated with serious injuries or death. In addition, an increasing number of voluntary reporting systems focus on safety improvements by also reporting errors that result in no or minimal harm (referred to as ‘near misses’) (Doupi, 2009). In Finland, the law obliges health professionals to report safety incidents and “near misses” anonymously in the nationwide comprehensive and standardized patient harm reporting system called The Reporting System for Safety Incidents in Health Care Organizations (HaiPro). The HaiPro database meets World Health Organization criteria for a good reporting system (Holmström, 2015).
The aim of this study was to examine whether the nurse-assessed optimality of the nursing workload is a valid measure of the adequacy of nursing resources. In this study, the predictive validity of the RAFAELA® was tested by investigating whether the number of patient safety incidents could be predicted by the optimality data retrieved from the RAFAELA®. It was assumed that the number of patient safety reports increases with increasing levels of nursing workload.
This was a cross-sectional retrospective observational study using administrative data from two Finnish hospitals. Hospital A is a tertiary acute care hospital that serves as the specialist medical care centre for the 248,000 citizens in its area. As one of the five university hospitals in Finland, it also serves nearly one million inhabitants who require specialist medical care for especially demanding cases. Hospital B is a secondary acute care hospital that arranges specialized health care for a population of approximately 167,000. The hospital offers services in 16 specialized fields.
Data collection and sample
In 2012 and 2013, monthly hospital safety incident report statistics held by a web-based HaiPro® reporting system and monthly summary reports of daily registrations from the RAFAELA® were collected from 34 inpatient units of the study hospitals. Monthly unit reports were included in the analysis if two criteria were met: first, the RAFAELA® was in use; second, the optimal level of nursing workload for the unit had been determined. Intensive care units (ICUs) were not included in this study because the RAFAELA® was not used in ICUs. In hospital A, 35 monthly reports were rejected because of incomplete data, and 48 monthly reports were not available because of organizational rearrangements. In hospital B, one monthly report was rejected because of incomplete data. The final number of complete months of unit data (732 in total) included 469 monthly reports from 23 out of 30 inpatient units of hospital A (n = 242 in 2012 and n = 227 in 2013) and 263 monthly reports from all 11 inpatient units of hospital B (n = 132 in 2012 and n = 131 in 2013).
Four estimates of nursing workload were used in this study.
Patient safety incident reports
In this study, monthly reports of patient safety incidents were retrieved via the HaiPro® system. Incidents were classified into 14 categories, but there were two main categories: (1) ‘near misses’, which might have caused harm to the patient but were prevented by chance or by timely preventive actions; and (2) harm incidents, which were negative events that caused harm to the patient. In this study, the measure used was the monthly number of all safety incident reports on the unit (i.e., reports of both ‘near misses’ and harm incidents).
The data were analysed using SPSS 22 for Windows (IBM, n.d.). The data were described as means and standard deviations (continuous data) or as medians and ranges (categorical and count data). Mostly non-parametric statistical tests were applied (Spearman’s rho, Mann–Whitney U test or Kruskal–Wallis test). A p-value <0.05 was considered to indicate statistical significance. To investigate the association of the number of patient safety incident reports with the chosen predictors (patient-to-nurse ratio, nursing intensity points per patient and optimality level of the workload), univariate and multivariate negative binomial regression analyses were conducted using generalized estimation equations (GEE) with the hospital unit as a subject variable. Based on the univariate and bivariate analyses, three predictors were chosen for the final model: patient-to-nurse ratio as a dichotomous variable (split into two by median: low/high), nursing intensity points per patient as a dichotomous variable (split into two by median: low/high) and optimality level of the workload (low/optimal/high).
The average nursing workload was at the optimal level in 68.6%, above the optimal level in 21.6% and below the optimal level in 9.8% of the monthly reports of the study units (n = 732). High workload was more common in hospital B than hospital A (prevailing in 30.4% vs. 16.6% of the monthly reports, p < 0.001, Chi square test). Within both hospitals, there were significant differences in the optimality of workload between the different units. The number of monthly safety incident reports varied between zero and 23, with the overall median being two reports per month and unit. The oncology units exceeded all other specialties in the number of reports (median 9). Overall, fewer incidents were reported in hospital A (mean 2.54, STD 3.07) than in hospital B (mean 3.56, STD 3.77, p=0.001, Mann-Whitney U test). The difference between the hospitals related to the preponderance of “near miss” reports of hospital B (mean 1.98, STD 2.43) compared with hospital A (mean 0.86, STD 1.52, p < 0.001, Mann-Whitney U test). The number of reports associated with patient harm was equal in both hospitals. The most common patient harm reports involved medication errors and accidents (Table 1).
Table 1: Optimality of the nursing workload and safety incident reports per unit and month according to the specialty and the hospital.
The number of incident reports was lowest in the months when the nursing workload was below the optimal level (mean 1.65, STD 2.06, n = 72), significantly higher when the nursing workload was at the optimal level (mean 2.91, STD 3.28, n = 502), and highest when the nursing workload was above the optimal level (mean 3.46, STD 3.97, n = 158). Although overall differences in the number of incident reports between the levels of nursing workload were statistically significant (p=0.002, Kruskal-Wallis test), the difference between optimal and above-optimal levels did not reach statistical significance (Table 2).
Table 2: Average number of patients and nurses, patient-to-nurses ratio, nursing intensity (NI) points per patient, per nurse and patient safety incident reports according to the level of the nursing workload (NWL).
The number of safety reports did not correlate with the nursing intensity (NI) points per nurse but correlated mildly positively with the optimality level of the nursing workload (NWL) (Spearman rho = 0.109, p = 0.003). In addition, the number of safety reports correlated positively with the NI points per patient (Spearman rho = 0.193, p < 0.000) and negatively with the patient-to-nurse ratio (Spearman rho = - 0.131, p < 0.000). A high negative correlation (Spearman rho = - 0.611, p < 0.000) was found between the NI points per patient and the patient-to-nurse ratio.
In the negative binomial regression analysis with the three predictors: the patient-to-nurse ratio (as a dichotomous variable, low/high), patient dependency (as a dichotomous variable, low/high) and optimality level of the nursing workload (low, optimal, high), only the last one had a main effect on the number of reported patient safety incidents (Wald Chi square 14,671, df 2, p = 0.001). However, all pairs of predictors showed statistically significant interaction effects that complicated the interpretation of the results. Table 3 shows the number of patient safety reports estimated by this negative binomial model. Only with a low patient-to-nurse ratio and a high nursing intensity per patient was there a rather clear trend of increased patient safety reports with the growing level of the nursing workload. Under other conditions, the estimated number of patient safety reports was approximately equivalent at both optimal and high nursing workload levels (Table 3).
Table 3: Number of patient safety incident reports estimated by a negative binomial model (GEE) including the predictors: patient-to-nurse ratio, patient dependency, and optimality level of the nursing workload
The present analysis revealed a negative correlation between the patient-to-nurse ratio and the number of reported patient safety incidents. The fewer patients per nurse, the greater the number of patient safety incidents on a unit. This result seems to contradict the well-known findings of large-scale, hospital-level studies (Duffield et al., 2011; Needleman et al., 2011; Twigg, Duffield, Bremner, Rapley, & Finn, 2012). Fagerström et al. (2018) suggested that with fewer patients per nurse, there is more time to report incidents. We propose another explanation, hypothesizing that there are fewer patients per nurse on hospital units when caring for more seriously ill patients and that these patients have a greater likelihood of negative patient outcomes. In this study, this hypothesis was confirmed. The inconsistency of research findings is probably influenced by different study designs. While comparing hospitals, better overall nursing resources seemed to be associated with fewer negative patient outcomes. However, while comparing hospital units, characteristics other than the patient-to-nurse ratio may also determine negative patient outcomes. Thus, the results of this study further emphasize the importance of considering the hospital unit as a vantage point while considering the impact of nursing resources on patient and nurse outcomes, as suggested by many authors (MacPhee et al., 2017).
According to the results of this study, nursing intensity as such was not associated with the number of reported safety incidents. This result was expected. The basic premise of the RAFAELA® is that the same amount of nursing work may signify a low, optimal or high workload depending on unit-specific characteristics such as the actual caseload (acuteness, severity and dependency of the patients) and the quality of the nursing work environment (facilities, equipment, working arrangements, leadership, team work and interprofessional cooperation). Unit-specific information regarding the optimality level of the nursing workload is needed for a reliable estimation of the adequacy of nursing resources. Indeed, the number of reported safety incidents in this study increased with the growing nursing workload level; however, a clear increasing trend in the number of safety reports with the increasing nursing workload level was found only when the patient-to-nurse ratio was relatively low, and the patient dependency was relatively high. In all other circumstances, the number of safety reports was almost the same at the optimal level compared with the high level of nursing workload.
If our assumption that the number of patient safety reports should increase with increasing levels of nursing workload is correct, our results only partially confirm the validity of the RAFAELA® as a measure for the adequacy of nursing resources. However, this finding should be considered with caution. First, as suggested by some results of this study, patient safety may be lacking sensitivity as an outcome of nursing care at the hospital-unit level because other factors, such as the severity of patients’ illness and comorbidities, may be a major determinant of the outcome. Second, it would be logical to assume that underreporting is more common when the nursing workload is high. The differences between the two hospitals and, in particular, between hospital units within each hospital in reporting safety incidents raise questions regarding the extent these differences might be related to differences in patient safety culture and reporting practices.
There were some limitations in this study. It was a rather small-scale study. Elaborate statistical analyses with multiple risk adjustments were not possible because of limited data. Thus, confounding is a major concern. In supplement to our previous study, these results indicated that while exploring the impact of nursing workload at a hospital-unit level, other factors in addition to nursing resources are needed to explain the occurrence of safety incidents.
Despite limitations, this study rendered some additional confirmation on the validity of the RAFAELA Nursing Intensity and Staffing System®. High workload may constitute a higher risk for patient safety incidents more in some hospital units than in others. The findings of this study demonstrate the importance of considering actual differences between hospital units, particularly in relation to the capacity of this particular nursing staff to meet the care needs of these kinds of patients in this kind of nursing work environment. These results complement the findings of the previous study performed by researchers with other variables and similar methods (Junttila et al., 2016).
Implications for nursing informatics
The role of the nursing intensity and staffing system is indispensable in promoting evidence-based decision-making in nursing. Utilization of all data collected on the unit should be considered an integral part of nurses’ and nurse leaders’ determination to improve patient care and redefine professional nursing. Engaging nurses in analysing daily variations in the adequacy of nursing resources enables nurse leaders to recognize unit-specific vulnerabilities and to identify new innovative ways to prevent work overload or to safeguard quality care in times of an inevitable high workload.
In this study, all research data were in the form of anonymous monthly statistics. No data concerning individual patients or nurses and no personal sensitive health-related data were available to the researchers. In Finland, only permission for the use of the statistical administrative data by each organization concerned is required from the health care organization for this kind of study. Each organization included provided permission for this study.
Aiken, L. H., Sloane, D. M., Ball, J., Bruyneel, L., Rafferty, A. M., & Griffiths, P. (2018). Patient satisfaction with hospital care and nurses in England: An observational study. BMJ Open, 8(1), e019189.
Aiken, L. H., Sloane, D. M., Bruyneel, L., Van den Heede, K., Griffiths, P., Busse, R., . . . Lesaffre, E. (2014). Nurse staffing and education and hospital mortality in nine European countries: A retrospective observational study. The Lancet, 383(9931), 1824-1830.
Al-Kandari, F., & Thomas, D. (2009). Perceived adverse patient outcomes correlated to nurses' workload in medical and surgical wards of selected hospitals in Kuwait. Journal of Clinical Nursing, 18(4), 581-590. doi:10.1111/j.1365-2702.2008.02369.x
Andersen, M. H., Lønning, K., & Fagerström, L. (2014). Testing reliability and validity of the Oulu patient classification instrument—the first step in evaluating the RAFAELA system in Norway. Open Journal of Nursing, 4(04), 303.
Carlesi, K. C., Padilha, K. G., Toffoletto, M. C., Henriquez-Roldn, C., & Juan, M. A. C. (2017). Patient safety incidents and nursing workload. Revista Latino-Americana De Enfermagem, 25
Cho, E., Sloane, D. M., Kim, E., Kim, S., Choi, M., Yoo, I. Y., . . . Aiken, L. H. (2015). Effects of nurse staffing, work environments, and education on patient mortality: An observational study. International Journal of Nursing Studies, 52(2), 535-542.
Cho, S., Mark, B. A., Knafl, G., Chang, H. E., & Yoon, H. (2017). Relationships between nurse staffing and patients’ experiences, and the mediating effects of missed nursing care. Journal of Nursing Scholarship, 49(3), 347-355.
Doupi, P. (2009). National reporting systems for patient safety incidents: A review of the situation in europe. Raportti/Terveyden Ja Hyvinvoinnin Laitos (THL)= Report/National Institute for Health and Welfare: 13/2009.
Duffield, C., Diers, D., O'Brien-Pallas, L., Aisbett, C., Roche, M., King, M., & Aisbett, K. (2011). Nursing staffing, nursing workload, the work environment and patient outcomes. Applied Nursing Research, 24(4), 244-255.
Fagerström, L., Kinnunen, M., & Saarela, J. (2018). Nursing workload, patient safety incidents and mortality: An observational study from Finland. BMJ Open, 8(4), e016367.
Fasoli, D. R., Fincke, B. G., & Haddock, K. S. (2011). Going beyond patient classification systems to create an evidence-based staffing methodology. Journal of Nursing Administration, 41(10), 434-439.
Fasoli, D. R., & Haddock, K. S. (2010). Results of an integrative review of patient classification systems. Annual Review of Nursing Research, 28(1), 295-316.
Flo, J., Landmark, B., Hatlevik, O. E., & Fagerström, L. (2018). Using a new interrater reliability method to test the modified Oulu patient classification instrument in home health care. Nursing Open, 5(2), 167-175.
Holmström, A. (2015). Learning from medication errors in healthcare how to make medication error reporting systems work? Health Policy, 119(8):1046-54
IBM. (n.d.). IBM SPSS software. Chicago, IL: IBM. Retrieved from https://www.ibm.com/analytics/spss-statistics-software
Junttila, J. K., Koivu, A., Fagerström, L., Haatainen, K., & Nykänen, P. (2016). Hospital mortality and optimality of nursing workload: A study on the predictive validity of the RAFAELA nursing intensity and staffing system. International Journal of Nursing Studies, 60, 46-53. doi:10.1016/j.ijnurstu.2016.03.008
Liljamo, P., Kinnunen, U., & Saranto, K. (2018). Assessing the relation of the coded nursing care and nursing intensity data: Towards the exploitation of clinical data for administrative use and the design of nursing workload. Health Informatics Journal, Dec. 5. 1460458218813613.
MacPhee, M., Dahinten, V. S., & Havaei, F. (2017). The impact of heavy perceived nurse workloads on patient and nurse outcomes. Administrative Sciences, 7(1), 7.
Needleman, J. (2015). Nurse staffing: The knowns and unknowns. Nursing Economic$, 33(1), 5-7.
Needleman, J., Buerhaus, P., Pankratz, V. S., Leibson, C. L., Stevens, S. R., & Harris, M. (2011). Nurse staffing and inpatient hospital mortality. New England Journal of Medicine, 364(11), 1037-1045.
Rauhala, A. (2008). The validity and feasibility of measurement tools for human resources management in nursing case of the RAFAELA system.Kuopion Yliopiston Julkaisuja E.Yhteiskuntatieteet, 159.
Rauhala, A., & Fagerström, L. (2004). Determining optimal nursing intensity: The RAFAELA method.Journal of Advanced Nursing, 45(4), 351-359.
Rauta, S., Salanterä, S., Vahlberg, T., & Junttila, K. (2017). The criterion validity, reliability, and feasibility of an instrument for assessing the nursing intensity in perioperative settings. Nursing Research and Practice, Epub July 2017.
Tursunbayeva, A., Bunduchi, R., Franco, M., & Pagliari, C. (2016). Human resource information systems in health care: A systematic evidence review. Journal of the American Medical Informatics Association, 24(3), 633-654.
Twigg, D., Duffield, C., Bremner, A., Rapley, P., & Finn, J. (2012). Impact of skill mix variations on patient outcomes following implementation of nursing hours per patient day staffing: A retrospective study. Journal of Advanced Nursing, 68(12), 2710-2718.
Twigg, D. E., Gelder, L., & Myers, H. (2015). The impact of understaffed shifts on nurse‐sensitive outcomes. Journal of Advanced Nursing, 71(7), 1564-1572.
van Oostveen, C. J., Mathijssen, E., & Vermeulen, H. (2015). Nurse staffing issues are just the tip of the iceberg: A qualitative study about nurses’ perceptions of nurse staffing. International Journal of Nursing Studies, 52(8), 1300-1309.
Wolf, Z. R., & Hughes, R. G. (2008). Error reporting and disclosure. Chapter 35. In: Hughes RG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville (MD): Agency for Healthcare Research and Quality (US).
Jaana K. Junttila, RN, MSc, PhD student, Kuopio University Hospital, Kuopio, Finland, Faculty of Information Technology and Communication Sciences (ITC), Tampere University, Tampere, Finland
Pirkko Nykänen, Professor Emerita, Faculty of Information Technology and Communication Sciences (ITC), Tampere University, Tampere, Finland
Kaisa Haatainen, PhD, Docent, Patient Safety Manager, Kuopio University Hospital, Kuopio, Finland, University of Eastern Finland, Department of Nursing Science, Kuopio, Finland
Aija Koivu, PhD, psychologist, statistics, Kuopio University Hospital, Kuopio, Finland