Determining the Measures of Success for Interoperability

At its most basic, interoperability is the ability to share and access health information data across systems to optimize healthcare for both providers and patients. While interoperability is a work in progress, in order to know when our efforts toward interoperability have been effective, we need to ask a few questions.

The first is: What is the measure of success for interoperability maturity and what parameters or indicators would one measure? You could measure volume—such as the number of transactions exchanged between partners, like an external lab and provider or emergency department and physician—and the quality of the information exchanged, its utility and the impact of the exchanged information on patient care.

The next question could be: Does the specific EHR functionality, configuration and training support the role-based workflows required for optimal information exchange and usability?

Finally: What are the key performance indicators (KPIs)—or parameters—that one would use to characterize the measure of success? Not only would these KPIs be useful to potentially rate the overall implementation and interoperability reach, it could also be used to compare different agencies or commercial entities’ interoperable success. These comparisons could extend to EHR usability, exchange frameworks and networks, like health information organizations, Carequality, eHealth Exchange, CommonWell Health Alliance, etc., and modes of exchange, like direct messages or queries.

Starting With the Basics

In assessing these potential measures of success, we uncovered that a well-defined set of parameters to measure even the most basic metrics for volume doesn’t exist.

RELATED: What is Interoperability?

For example, you could ask a health information exchange (HIE) to list the transactions between its registry and a provider, or a health information service provider (HISP) to provide a number of transactions across two endpoints and both can probably give you a number, say, 10,000 transactions a day.

However, there is often no mechanism to consistently know what these transactions include (e.g., HL7 v2 messages, HL7 FHIR®) that would provide the granularity of the type of messages (e.g., orders, results, labs, etc.), the direction of exchange, or the documents included (e.g., CCDs).

From the raw numbers of messages successfully delivered, we also don’t know whether the documents were ever opened. If the information included discrete data, was this data ingested and used to reconcile a chart or update a longitudinal patient record? Was it actively used to care and manage a patient? Many messages might be hard to track, especially point-to-point messages as there is no standard logging or tracking mechanism in place.

Creating a Foundation for Measurement

Before we can even begin to parse out the quality, usability and other measures of success, these questions around the definition of the “volume” of health information exchanged must be addressed. Once we agree on the foundational measure of volume, we can work toward defining more criteria to fully understand the breadth and impact of interoperability within the healthcare system.

We recognize there are several organizations in the process of defining these volume measures, but may be doing so independently without the broader community’s knowledge. If we hope to move toward consistent understanding of our progress in interoperability, we must break down these silos and start a productive dialogue to define these measurement parameters.

By creating common definitions, we can provide a basis for the next steps to create the appropriate KPIs and work on quality, usability, utility and outcomes of interoperable exchange.

The views and opinions expressed in this blog or by commenters are those of the author and do not necessarily reflect the official policy or position of HIMSS or its affiliates.

Exploring the Measures of Success for Interoperability

The HIMSS Interoperability & HIE Committee is exploring the measurement needs of interoperability efforts and welcomes your voice and input. If you are currently involved in efforts to measure our national interoperability progress, we’d love to hear from you.

Connect with Us