Development and Testing of an Instrument to Measure Informatics Knowledge, Skills, and Attitudes among Undergraduate Nursing Students

Citation

Bryant, L., Whitehead, D., & Kleier, J. (July, 2016). Development and testing of an instrument to measure informatics knowledge, skills, and attitudes among entry-level nursing students. Online Journal of Nursing Informatics (OJNI), 20(2), Available at http://www.himss.org/ojni

Abstract

Informatics competencies in nursing education have long been and continue to be a concern. This article reports on the development and psychometric testing of the Knowledge, Skills, and Attitudes towards Nursing Informatics (KSANI) Scale to measure these constructs among entry-level nursing students. A measurement instrument was developed based on the Quality and Safety Education for Nurses (QSEN) Institute informatics competencies for pre-licensure students (Cronenwett et al., 2007). Survey data were collected from a convenience sample of 300 undergraduate nursing students attending the 2014 Florida Student Nurses Association’s annual convention. The data were subjected to Cronbach’s test to estimate the level of reliability as internal consistency. At 0.90, the alpha for the overall scale exceeded the 0.70 benchmark for acceptability. The scale items were clustered into the intended three factors – knowledge, skills and attitudes – as well as into the added factor of opportunities. The instrument was found to be sound and appropriate for the target population. 

Nursing informatics combine the disciplines of nursing science, information science, and computer science (McGonigle & Mastrian, 2015). Ever since the time of Florence Nightingale, one of the critical roles of the registered nurse (RN) has been to collect and interpret data to provide safe and effective patient care. Since the early 1980s, informatics competencies in nursing education have been discussed in nursing literature (Staggers, Gassert, & Curran, 2001). In 1992, the American Nurses Association (ANA, 2015) recognized the importance of technology to nursing practice, identifying nursing informatics as a specialty practice. The 1999 Institute of Medicine (IOM) report calling for a safer health care system identified the use information technology (IT) as a key factor toward meeting this goal. In 2010, the IOM published The Future of Nursing, which recommended making technology an essential component of nursing education. Both the American Association of Colleges of Nursing (AACN, 2008) and the National League for Nursing (NLN, 2008) emphasized that  knowledge and skills in information management and patient care technology are critical components in nursing education and accreditation. 

Skiba, Connors, and Jeffries (2008) identified a lack of informatics competencies in nursing education prior to 2008. Since that time, the American Association of Colleges of Nursing (AACN) and the Robert Wood Johnson Foundation (RWJF) have partnered to support the Quality and Safety Education for Nurses (QSEN) Initiative (AACN, 2016). One of the components of the QSEN Initiative was the development of competencies in various areas including informatics.

This research contributes to the development of a reliable and valid instrument based on the QSEN competencies to test the informatics knowledge, skills, and attitudes of current RN students in Florida.

Problem Statement

No succinct research instrument that addresses the informatics knowledge, skills and attitudes of undergraduate nursing students was found. Several studies have sought to validate instruments addressing informatics skills level in health professions students, but studies have not addressed students’ perceptions of integration of informatics into their program of study and opportunities to use a variety of technologies as opportunities for learning (Choi & Bakken, 2013; Sun & Falan, 2013; Yoon, Yen, & Bakken, 2009).   

Purpose

The purpose of this study was to develop a new instrument, the Knowledge, Skills, and Attitudes towards Nursing Informatics (KSANI) Scale, and test it to assure it met the psychometric standards for use among this population. The following research questions were addressed:

  • Do the items on the KSANI measure have content validity in that they are an adequate measure of the common underlying dimensions of knowledge, skills and attitude toward informatics among undergraduate nursing students?  
  • Are the items on the KSANI internally consistent and reliable in that they correlate with other items on the scale that purport to measure the same construct but do not correlate with items on the scale that do not measure the same construct?

Significance

Although healthcare professionals seem to express positive attitudes toward computer information systems (CIS) and their use in evidence-based practice, recent research continues to identify a poor correlation between these positive attitudes and the use of CIS in practice (Melas, Zampetakis, Dimopoulou, & Moustakis, 2014). Lavin, Harper, and Barr (2015) cited a current lack of basic informatics education, coupled with the lack of expectation for graduates to exhibit informatics competencies, as a continuing and critical issue impacting patient outcomes. Hern, Key, Goss, and Owens (2015) identified that readiness to become proficient in informatics competencies varies with the age of the provider. Although nursing programs have worked to incorporate informatics into their curricula, there is no current research on the knowledge, skills, and attitudes of undergraduate nursing students.

Literature Review

The 1999 IOM recommendation to establish a safer health care system identified IT as one of the ways to meet this goal (National Research Council, 2012). The Health Information Technology for Economic and Clinical Health (HITECH) Act, enacted as part of the American Recovery and Reinvestment Act of 2009, was signed into law on February 17, 2009. This law promotes the adoption and meaningful use of health information technology and provides the Department of Health and Human Services (HealthIT.gov, 2015) the authority to establish programs to improve health care quality, safety, and efficiency through the promotion of health information technology.

In 2007, the Technology Informatics Guiding Education Reform (TIGER, 2014) initiative developed informatics competency recommendations for nursing education, research and practice. The AACN (2008) and the Accreditation Commission for Education in Nursing (ACEN, 2013) have added informatics competencies to their standards. Since that time, informatics outcomes have been added to the curricula of many entry-level nursing programs.

Over the past decade, research has been conducted on incorporating a variety of informatics competencies into nursing curricula (Gallaghar-Lepak, Scheibel, & Gibson, 2009; Gardner & Jones, 2012; Hern et al., 2015; Schwirian, 2013; Tellez, 2012). Choi (2012) revealed that traditional pre-licensure students differed significantly from accelerated baccalaureate in science of nursing (BSN) and RN-to-BSN students in their overall informatics competency.  Choi and De Martinis (2013) indicated that undergraduate students continued to perceive themselves as lacking competence in applied computer skills and the clinical informatics role.  To date, no assessment of undergraduate students’ perception of their knowledge, skills, and attitudes has been published.

Instrument Development

The QSEN Institute developed recommendations for informatics to be included in the education of students. An instrument was developed based on the QSEN competencies (Cronenwett et al., 2007). The scale was intended to measure three factors: perception of informatics knowledge, perception of skills in using informatics, and attitudes toward the use of informatics. In addition, questions were included to elicit if the participants were offered opportunities in their nursing programs to apply informatics knowledge.

The 24-item instrument contained three sections, each with varying numbers of questions. The estimated time to complete the survey was 10 minutes. Section 1, which contained 12 items dealing with participant abilities related to specific competencies and the stem, I feel confident in my ability to…, was included with the competency statements in this section. Section 2, which contained four questions, addressed participant thoughts about various competencies, and the stem, It is important to me that…, was included. Section 3, which contained eight questions, was included to determine opportunities to apply informatics during participant educational programs. The stem, In my nursing program I had opportunities to…, was included with the competency statements.
Some QSEN competencies were divided into two different statements when there were two action verbs included. For example, the competency, document and plan patient care in an electronic health record, was included in the instrument as, I feel confident in my ability to document patient care in an electronic health record, and a second item, I feel confident in my ability to plan patient care in an electronic health record. Other items were adapted to in the attitude about informatics section. The competency, appreciate the necessity for all health professionals to seek lifelong, continuous learning of information technology skills, was also adapted into two statements to elicit the participants’ personal attitudes about lifelong learning for self and also for others. The resulting items were, It is important to me that I seek lifelong, continuous learning of information technology skills, and, It is important to me that all health professionals seek lifelong, continuous learning of information technology skills.

A four-point Likert scale was applied in each section. In Section 1, in which the statements related to the participants’ confidence in their ability to, the scale ranged from 1 (not confident) to 4 (extremely confident). Section 2 related to the participants’ attitudes, and the scale ranged from 1 (not important) to 4 (extremely important). In Section 3, participants rated their opportunities to apply informatics in their nursing programs. The scale in this section ranged from 1 (no opportunity) to 4 (frequent opportunity). 

A demographic survey was included to obtain information about the participants’ program types, their progress in the programs, use of electronic heath records (EHRs), and education about informatics. The survey asked participants to indicate the type of program and the semester in which they were currently enrolled. Questions about the use of EHRs asked about use in a current or previous place of employment. Information about the inclusion of an informatics course in a program was requested, as well as whether the course was required and/or if the participants had taken the course. Participants were also asked to indicate if they had taken an informatics course outside of their nursing program.

Instrument Testing

The Florida Nursing Students Association (FNSA, 2015), under the auspices of the Florida Nurses Association, has chapters in 50 undergraduate nursing programs throughout the state. Each year, the FNSA presents a convention that is popular among its constituents and draws attendees from across the state. In 2014, the convention was held in Daytona, FL; about 1,200 associate degree and undergraduate BSN nursing students attended. With permission from the convention organizers and the Institutional Review Board (IRB), anonymous data were collected from a convenience sample of the attendees. A priori sample size was determined based on the 24 items on the instrument. Using the rule-of-thumb of at least 10 participants per item (Nunnally & Bernstein, 1994), the sample size was set at a minimum of 240 participants providing complete data forms. 

Two researchers attended the FNSA convention. One of the researchers circulated in the exhibit area and distributed flyers announcing the opportunity to participate to the attendees. Those who were interested were directed to the researchers’ exhibit table and were given an information letter and the survey. To protect their confidentiality, the participants were encouraged to move away from the table to complete the survey.  Once the survey was completed, the participant placed it in a sealed box and received a numbered ticket for a drawing for a $10 gift card from a beverage company. At the conclusion of the convention, 20 tickets were drawn by a third party and gift cards were distributed. The sealed box opened when the researchers returned to the university campus and the data were entered in SPSS for statistical analysis.

Results

A total of 308 individuals submitted completed surveys. Of these, eight were eliminated due to incomplete information relating to the type of nursing program in which they were enrolled.  Data analysis was carried out using the remaining 300 participants. Information regarding the type of nursing educational program attended and the percentage of the program completed is presented in Table 1. 

The majority (n = 186, 62.0%) reported using an EHR in the current place of employment and an even larger number (n = 200, 66.7%) had used an EHR at a previous place of employment at some time. However, most participants (n = 237, 79.0%) reported that there was no informatics course offered in their nursing programs, and of these, 196 (65.3%) had not completed an informatics course outside of their nursing programs. Of the 61 (20.3%) participants that reported that there was an informatics course offered, 41 (67.2%) indicated that the course was required and 34 (55.7%) said they had completed the course. However, these 34 represented only 11.3% of the total sample.

To determine if the scale items were clustering into the intended three factors, a principal component analysis (PCA) was conducted on the 24 items with orthogonal rotation (varimax).  The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis, KMO = 0.89 and all KMO values for individual items were > 0.75, which is well above the limit of 0.5.  Bartlett’s test of sphericity, c2 (276) = 3332.35, p < 0.000, indicated that correlations between items were sufficiently large for PCA. An initial analysis was also run to obtain eigenvalues for each component in the data. 

Four components had eigenvalues over Kaiser’s criterion of 1 and, in combination, explained 57.96% of the variance. The scree plot reached a stable plateau at four factors. Given the large sample size (N = 300) and the convergence of the scree plot and Kaiser’s criterion on four components, this is the number of components that were retained in the final analysis.  Table 2 shows the factor loadings after rotation. The items that cluster on the same components suggest that component 1 represents educational opportunity to apply informatics, component 2 represents knowledge of informatics, component 3 represents informatics skills confidence, and component 4 represents attitude toward informatics



In this component matrix, each of the variables has a substantial loading on only one factor except for I feel confident in my ability to use information management tools to monitor outcomes of care, which loaded nearly equally on the factor labeled knowledge of informatics (0.474) and on the factor labeled informatics skills confidence (0.468). As this variable has a substantial loading on two factors, it is considered as complex. After careful consideration, the team chose to retain this item and attach it to the factor informatics skills confidence, for conceptual reasons. 
The data were subjected to Cronbach’s test to estimate the level of reliability in internal consistency. At 0.90, the alpha for the overall scale exceeded the 0.70 benchmark for acceptability. Corrected item-total correlation found that no item correlated below 0.30. These values suggest that the items on the scale are measuring the same thing. Alpha values for the individual factors are provided in Table 2. 

Discussion

The instrument contains four factors, educational opportunity to apply informatics, knowledge of informatics, informatics skills confidence, and attitude toward informatics, all of which have a high internal consistency and reliability. One item loaded equally into both the knowledge of informatics factor and the informatics skills confidence factor. In the knowledge, skills and attitudes of informatics competencies, this statement, use information management tools to monitor outcomes of care, is considered a skill competency (Cronenwett et al., 2007). Therefore, it was decided to consider the item an informatics skills confidence statement. The instrument appeared to be appropriate for students from both associate degree and baccalaureate nursing programs. As a result of the psychometric testing, no changes were made in the individual items. This instrument could be used to ascertain the knowledge, skills and attitudes about informatics. 

Strengths and Limitations

The items included on the instrument were directly derived from QSEN recommendations (Cronenwett et al., 2007). The data were collected at a convention attended by nursing students from different programs within Florida; thus the sample consisted of students enrolled in many programs within the state, increasing the generalizability of the findings.  However, there is no assurance that attendees represented all nursing programs in the state. Students attending a conference may be different from those students that did not attend the conference. Participants were asked to self-report and may have over-estimated their knowledge, skills or attitudes toward informatics. 

Conclusions and Recommendations

Several studies have sought to validate instruments addressing informatics skills level in health professions students, but studies have not addressed students’ perceptions of integration of informatics into their programs of study and opportunities to use a variety of technologies as opportunities for learning (Choi & Bakken, 2013; Sun & Falan, 2013; Yoon, Yen, & Bakken, 2009). A research instrument was developed using the QSEN attitudes, knowledge, and skills recommendations for inclusion in the education of pre-licensure students (Cronenwett et al., 2007). The data were subjected to tests for psychometric estimates. Validity was supported by feedback from an expert panel and factor analysis. Reliability as internal consistency was supported by Cronbach’s alpha meeting benchmark. The instrument was found to be sound and appropriate for the target population. 

The researchers recommend that the KSANI instrument be used to explore the current informatics knowledge, skills, and attitudes of undergraduate students in other nursing programs throughout the United States to gain a more thorough picture of the current state of informatics education in nursing programs.  

References

Accreditation Commission for Education in Nursing. (2013). Accreditation manual. Retrieved from: http://www.acenursing.org/accreditation-manual/

American Association of Colleges of Nursing. (2008). The essentials of baccalaureate education for professional nursing practice. Washington, DC: Author. Retrieved from http://www.aacn.nche.edu/publications/order-form/baccalaureate-essentials

American Association of Colleges of Nursing (2016). Quality & Safety Education for Nurses (QSEN). Retrieved from http://www.aacn.nche.edu/qsen/home

American Nurses Association. (2015). Nursing informatics: Scope and standards of practice (2nd ed.). Silver Springs: MD: Nursingbooks.org.

Choi, J. (2012). Comparative assessment of informatics competencies in three undergraduate programs. Online Journal of Nursing Informatics, 16(2). Retrieved from http://ojni.org/issues/?p=1700

Choi, J., & Bakken, S. (2013). Validation of the self-assessment of nursing informatics competencies scale among undergraduate and graduate nursing students. Journal of Nursing Education, 52(5), 275-282. doi: 10.3928/01484834-20130412-0

Choi, J., & De Martinis, J. (2013). Nursing informatics competencies: Assessment of undergraduate and graduate nursing students. Journal of Clinical Nursing, 22(13-14), 1970-1976. doi: 10.1111/jocn.12188

Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., Mitchell, P.,…Warren, J. (2007). Quality and safety education for nurses. Nursing Outlook, 55(3), 122-131. doi: http://dx.doi.org/10.1016/j.outlook.2007.02.006

Florida Nursing Students Association. (2015). FNSA regions. Retrieved from http://www.fnsa.net/fnsa-regions.html

Gallagher-Lepack, S., Scheibel, P., & Gibson, C. (2009). Integrating telehealth in nursing education: Can you hear me now? Online Journal of Nursing Informatics, 13(2), 1-16. Retrieved from http://ojni.org/13_2/GallagherLepak.pdf  

Gardner, C. L., & Jones, S. J. (2012). Utilization of academic electronic medical records in undergraduate nursing education. Online Journal of Nursing Informatics, 16(2).  Retrieved from http://ojni.org/issues/?p=1702

HealthIT.gov. (2015). Health IT legislation and regulations. Retrieved from http://healthit.gov/policy-researchers-implementers/health-it-legislation

Hern, M. J., Key, M., Goss, L. K., & Owens, H. (2015). Facilitating adoption of informatics and meaningful use of electronic health records with nursing faculty. Journal of Nursing Education and Practice, 5(3), 118-126. doi: 10.5430/jnep.v5n3p118

Institute of Medicine. (1999). To err is human: Building a safer health system. Washington, DC: National Academy Press. Retrieved from https://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf

Institute of Medicine. (2010). The future of nursing: Leading change, advancing health. Washington, DC: National Academies Press.

Lavin, M. A., Harper, E., & Barr, N. (2015). Health information technology, patient safety, and professional nursing care documentation in acute care settings. OJIN: The Online Journal of Issues in Nursing, 20(2). doi: 10.3912/OJIN.Vol20No02PPT04

Melas, C. D., Zampetakis, L. A., Dinopoulou, A., & Moustakis, V. S. (2014). The significance of attitudes towards evidence-based practice in information technology use in the health sector: an empirical investigation. Behaviour & Information Technology, 33(12), 1248-1260. doi: 10.1080/0144929X.2013.774048

McGonigle, D., & Mastrian, K. (2015). Nursing informatics and the foundation of knowledge (3rd ed.). Burlington, MA: Jones and Bartlett Learning.

National League for Nursing. (2008). Preparing the next generation of nurses to practice in a technology-rich environment: An informatics agenda. Retrieved from: http://www.nln.org/docs/default-source/professional-development-programs/preparing-the-next-generation-of-nurses.pdf?sfvrsn=6

National Research Council. (2012). Health IT and patient safety: Building safer systems for better care. Washington, DC: National Academies Press. Retrieved from http://www.modernhealthcare.com/Assets/pdf/CH76254118.PDF

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

Schwirian, P. M. (2013). Informatics and the future of nursing: Harnessing the power of standardized nursing terminology. Bulletin of the Association for Information Science & Technology, 39(5), 20-24.

Skiba, D. J., Connors, H. R., & Jeffries, P. R. (2008). Information technologies and the transformation of nursing education. Nursing Outlook, 56, 235-230. doi: 10.1016/j.outlook.2008.06.012

Staggers, N., Gassert, C. A., & Curran, C. (2001). Informatics competencies for nurses at four levels of practice. Journal of Nursing Education, 40(7), 303-316.

Sun, X., & Falan, S. (2013). What is your informatics skills level?—the reliability of an informatics competency measurement tool. Transactions of the International Conference on Health Information Technology Advancement. Paper 31. Retrieved from http://scholarworks.wmich.edu/cgi/viewcontent.cgi?article=1040&context=ichita_transactions   

Technology Informatics Guiding Education Reform. (2014). Informatics competencies for every practicing nurse: Recommendations for the TIGER collaborative. Retrieved from http://www.aacn.nche.edu/education-resources/tiger.pdf

 Tellez, M. (2012). Nursing informatics education past, present and future.CIN: Computers, Informatics, Nursing,30(5), 229-233. doi: 10.1097/NXN.0b013e3182569f42
Yoon, S., Yen, P. Y., & Bakken, S. (2009). Psychometric properties of the self-assessment of nursing informatics competencies scale. Studies in Health Technology Informatics, 146, 546-550.

Author Bios

Lynne E. Bryant, EdD, MSN, RN, CNE
Lynne Bryant is an Associate Professor at Nova Southeastern University in Ft. Lauderdale, FL. She earned a diploma in nursing from Johns Hopkins Hospital School of Nursing in Baltimore, MD, a BSN from the University of Maryland in Baltimore, MD, a MSN from the University of Pennsylvania in Philadelphia, PA, and an EdD from Florida International University in Miami, FL. Her scholarly interests focus on the issues involving nursing education.  

Jo Ann Kleier, PhD, EdD, ARNP, ACNP-BC
Jo Ann Kleier is a Professor and Director of Research at Nova Southeastern University in Ft. Lauderdale, FL. She earned a BSN from the University of South Florida in Ft. Myers, FL, a MSN from the University of Miami in Coral Gables, FL, an EdD from Florida International University in Miami, FL, and a PhD from Barry University in Miami Shores, FL. Her scholarly interests focus on the health protective behaviors of minorities. 

Diane K. Whitehead, DNP, EdD, RN, ANEF
Diane Whitehead is a faculty member at Walden University Doctor of Nursing Practice Program. She was previously a Professor of Nursing at Nova Southeastern University in Ft. Lauderdale, FL. She received a BSN from Florida State University in Tallahassee, FL, an MSN from the University of Miami in Coral Gables, FL, an EdD from Florida International University in Miami, FL, and a DNP from Walden University. Her areas of interest include nursing leadership and management and community health nursing.