Using a "think-aloud" process to aid co-design in development of a medicine information needs assessment tool


Medicine administration is a key component of nursing practice that encompasses consumer education. However, the information a consumer considers important in their health journey is difficult to ascertain in a timely manner. This paper describes one part of a multi-stage project aiming to develop a new tool to assist nurses, pharmacists and other healthcare professionals to better meet the medicine information needs of consumers. The development of this questionnaire involved a multi-step co-design process. Here we report on our findings to the research question of “How can co-design be used in the development of a patient-reported outcomes tool?”

Specifically, we report on the process of using a "think-aloud" process to aid co-design in the early testing and refining of the data collection tool (Medicines Information Needs for Individuals – Questionnaire: MINI-Q). Several rounds of "think-aloud" interviews with consumers were used to determine the tool’s acceptability for use, initially as a paper version and then as a digital version completed using a tablet device.

Consumers (n=40), with an average of three medical conditions and taking six regular medicines, were interviewed. They identified 315 issues with the MINI-Q. Their suggestions included clearer wording, reduced length, simplified sentence structure, changed formatting, type of response options, and additional information to collect. The "think-aloud" technique was helpful to improve the usability and acceptability of the tool. The next stage is to test the final version with a larger sample to determine acceptability and face validity.


Enhancing consumer engagement with their health care is expected to lead to improved health outcomes (Hopkinson, 2018). Engaged consumers should be viewed as active stakeholders in any health information development and therefore their perspectives need to be considered when developing information resources (Featherall et al., 2018; Horne, 2017). However, often health information initiatives are provider-driven, with consumer perspectives ignored. This approach risks consumer needs being inadequately addressed (Featherall et al., 2018; Lee et al., 2019; Young et al., 2018). The role of consumers in managing their own health has changed with increased access to on-line information (Honey & Westbrooke, 2016). They use the internet to explore aspects of their health, including to seek information about their medications (Honey et al., 2018; Kusch et al., 2018). However, accurate and inaccurate sources of medicine information can leave them with questions (Honey et al., 2018; Sheridan et al., 2015).

It is imperative to better understand consumers’ medicine information needs so that nurses, pharmacists and other health professionals can meet these needs and potentially provide information aligned to consumer priorities (Lee et al., 2019; Sze et al., 2018; Young et al., 2018). Not considering consumer needs may lead to poorer health outcomes, reduced medicine adherence, and resource wastage. This is particularly important when considering the creation of any health information assessment tool, as any design that does not involve consumers may lead to a tool that does not achieve its aim (Ioannidis, 2016). Using a co-design approach is a method that can ensure consumer engagement and acceptable tools (Nakarada-Kordic et al., 2017).

This paper reports our findings from addressing the research question “How can co-design be used in research and practice to improve the development of a patient-reported outcomes tool?” The paper first defines co-design and "think-aloud" processes before describing the application of these to gain an understanding of consumer perception of the Medicines Information Needs for Individuals – Questionnaire: MINI-Q. These co-design and "think-aloud" processes occur as part of a larger project, and so the context of the work presented in this paper is provided. The iterative "think-aloud" process used is described before presenting the results from the "think-aloud" process with consumers using a paper version (n=30) and a digital version (n=10). The discussion considers the results in light of wider literature before identifying limitations and the next steps in this project.


Co-design is described as involving collaboration between researchers and end users (Goodyear-Smith et al., 2015). Slattery et al. (2020, p. 2) define co-design as “meaningful end-user engagement in research design and includes instances of engagement that occur across all stages of the research process and range in intensity from relatively passive to highly active and involved.” Terms used to describe co-design approaches include participatory research methods; participatory action research; consumer, stakeholder or community engagement; and inclusive health research (Slattery et al., 2020). Therefore, a defining feature of co-design is that it is emergent and flexible as feedback from consumers is included (Goodyear-Smith et al., 2015). Using a co-design approach ensuring consumer understanding is central to the development of the questionnaire or research tool rather than the researcher or health professional making assumptions about a consumer’s interpretation or understanding. Consumer-centered co-design approaches can be incorporated into research in a variety of ways including consumer involvement in research planning, product design, stakeholder engagement, collaborative data collection, shared analysis, and distribution of results (Slattery et al., 2020).

A 2020 overview of 23 literature reviews (Slattery et al., 2020) identified there were no comparative studies indicating co-design led to better health outcomes or cost-effectiveness compared to traditional research approaches. However, the benefits of co-design included ensuring research topics, questions and design of materials were more applicable and acceptable; increased consumer knowledge and confidence about managing their health; improved ease of access to participants by researchers, particularly recruiting from hard-to-reach groups; and increased acceptability by consumers of research instruments that were developed using co-design (Slattery et al., 2020).

 "Think-aloud" process

One way to collaborate with consumers and gain insight into what they are thinking is to use the "think-aloud" process that was first ascribed to Ericsson and Simon (1980). Think-aloud techniques are derived from cognitive psychology with the basic procedure being to ask the participant to verbalize what they are doing as they complete a task (Mes et al., 2019). Verbalizations are considered to reflect the cognitive and metacognitive processes and strategies used as participants engaged with the task (Charters, 2003), therefore giving insight into the participant’s understanding. While there is some thought that the "think-aloud" can be carried out either alongside the task or soon after completing a task (Peute et al., 2015) evidence supports concurrent task and "think-aloud" processes (Charters, 2003; Peute et al., 2015). The "think-aloud" process has been used successfully to understand the thinking process of patients as they complete questionnaires (Mes et al., 2019), including the Interpersonal Skills Questionnaire (ISQ) (Al-Jabr et al., 2018), Illness Perception Questionnaire (Aujla et al., 2020) and Quality-of-Life Questionnaire (Zhang et al., 2017). Involving consumers in the co-design of questionnaires reduces common issues in questionnaire design such as response error and incorporates people’s real-world experiences into the questionnaire design process (Mes et al., 2019).

 The project to date

This paper reports part of a larger project that aims to develop a tool to assist nurses, pharmacists and other health professionals to meet the medicine information needs of consumers. There are four phases to this project. Phase One was a needs assessment to explore what information consumers wanted to know about their medicines, when they want this information, and in what format they prefer it. Data collection for this phase involved 60 face-to-face interviews (30 mental health service users and 30 medical in-patients). The findings highlighted that consumer medicine information requirements are diverse and need to be individualised (Ashmore-Price, et al., 2017; Chan et al., 2020; Honey et al., 2018; Sheed et al., 2017).
The second phase consisted of a literature search to identify suitable tools to address these diverse needs. Four tools were found that potentially could be suitable: The Satisfaction with Information about Medicines Scale (SIMS) (Horne et al., 2001); The Beliefs about Medicines Questionnaire (BMQ) (Horne et al., 1999); The Intrinsic Desire for Information Scale (IDI) (Duggan et al., 2002); and The Extent of Information Desired Scale (EID) (Duggan & Bates, 2000). These tools were then reviewed by the research team to decide whether these could be used as they are, would need adaptation, or if a bespoke tool was needed. Of the relevant tools identified, the SIMS was considered to have the best coverage of the needs identified in Phase One; however, the SIMS was developed in the United Kingdom (UK) nearly 20 years ago as a measurement tool for medicine information audit and research purposes (Horne et al., 2001). The 17 items in the scale are written in the second person and assess UK consumer satisfaction with medicine information retrospectively. This means consumer perception is gained after medicine information has been provided rather than assisting in the identification of preferences and priorities prior to receiving the medicine information. After reviewing the tool, the research team concluded that adaptation and customisation for our purpose and to the New Zealand (NZ) context was needed, which was Phase Three of this project (Amende et al., 2019; Brackley et al., 2019).

Phase Three began with the development of the first iteration of the Medicines Information Needs of Individuals Questionnaire (MINI-Q), which was informed by the aim of the project, the data collected from consumers in Phase One, and the literature identified in Phase Two, the SIMS in particular. This paper focuses on the consumer engagement aspect of Phase Three, during which the usability and acceptability of the MINI-Q by general medical and surgical in-patients at one large urban hospital were explored using a qualitative approach collecting data via a "think-aloud" interview process.

 The Medicines Information Needs of Individuals Questionnaire (MINI-Q)

The first version of the MINI-Q was a 20-item tool designed to be completed by the consumer initially on paper, with a digital version developed once the content of the tool was finalized. The long-term goal is that the information gathered will be incorporated into consumers’ health records and be available to nurses, pharmacists and other health professionals to guide their medicine information provision, so it is individualised to the consumer’s needs, preferences and immediate priorities. The decision to have a digital version was informed by the long-term aim of being able to integrate the findings with the electronic healthcare record. The MINI-Q comprises three main domains (Table 1): information about the consumer, information requested, and preference for how and when to receive the information. The MINI-Q is designed to be completed by the consumer, and responses represent their needs at that point in time. This paper reports on the use of consumer co-design to test and further develop the MINI-Q tool by informing question and questionnaire refinement and future implementation of the questionnaire in practice and research.

Table 1. MINI-Q Domains with examples of the information sought.


A staged iterative "think-aloud" interview process was used to explore consumers’ understanding of the MINI-Q questions and their use of the MINI-Q in the paper and digital versions. The focus of these interviews was to identify inappropriate items in the tool and those that lacked clarity or were hard for consumers to understand. In addition, we sought to identify topics considered important to them that were not included in the tool and whether the instructions for completion were easily understood. The core research team reviewed the data collected from consumers every five interviews and used the findings to refine the tool before the next five interviews were conducted, in an iterative cycle of tool development in line with cognitive ‘think aloud’ methodology (Mes et al., 2019). Ethical approval for this study was received from the NZ Health and Disability Ethics Committee (reference: 18/NTA/137).


Consumers, aged 18 years and over, admitted to a general medical or surgical ward at a large urban hospital during the study period (November 2018 to August 2019) who had previously taken medication were purposively recruited to capture the views of people with a range of ages, ethnicities and long-term conditions representing the population demographics of the study hospital catchment area. The aim of this purposive sampling was to be able to assess the usability of the questionnaire among a diverse range of interviewees to ensure its acceptability and applicability to all potential users of the tool. Potentially eligible consumers were identified from the daily patient list by clinical pharmacists. The researcher then discussed the consumer’s suitability to participate with the clinical pharmacist and nurse on each ward. Those invited to participate could consult with their family or support person before deciding to participate. They could also have a family member or support person present during the interview. The inclusion criteria for participation were 1) the ability to speak English at a sufficient level to understand the purpose of the study and give consent, 2) experience taking any medication (prescription, pharmacist-only, pharmacy-only or general sale), and 3) willingness to have the interview audio-recorded. Interviews were conducted at a time convenient for the consumer during the hospital stay in a private area of the ward. Any specific medicine information needs or concerns identified during the interview were reported to the ward clinical pharmacist for follow-up.

 Data collection

The interviews were undertaken by two researchers, and after initial training, two preliminary interviews were conducted as "pilots" to ensure researcher capability and consistency. Participants were given paper versions of the MINI-Q. Using the "think-aloud" technique, researchers conducted one-on-one semi-structured interviews in the ward. Interviewees were asked to share their thoughts as they completed the tool (Figure 1). Consumers were asked not to use a pen so that they would verbalise all thoughts and responses. Responses were audio-recorded, and field notes taken. Usability factors such as time taken to complete the questionnaire, question items skipped, and the order in which questions were answered were documented. Demographic data were collected on age, gender, ethnicity, long-term conditions, and number of medications. The prescription medicines reported by the consumer were then confirmed with their clinical record, with their permission. As a thank you for participation, a NZ$20 shopping voucher was offered to all participants.

Figure 1. Figure illustrating the think-aloud process undertaken and consumer input (co-design) into MINI-Q development

This process was undertaken in eight iterative stages with five participants at each stage (total of 40 participants; 30 participants for the paper version and 10 participants for the digital tablet version). After each stage the research team convened and decided on the changes to be made to the tool before the "think-aloud" task was repeated with the new version of the tool using another sample of five consumers. After six iterations using a paper-based version, the MINI-Q was transferred to a digital format. The iterations continued until few new comments or suggestions were received.


Consumer characteristics

In total, 46 consumers agreed to participate in this study. However, two withdrew due to fatigue, two were unable to fully participate, and two were discharged or transferred from the ward before the interview, leaving a total of 40 who participated. One interview was only partially completed due to the consumer not feeling well enough to continue, and these data were included. The interviewees had an average of three long-term conditions, with one having seven long-term conditions. The average number of prescribed medications taken by each were six, with the maximum being 12. Of these 40, a third were NZ European, just over half were female, and their ages ranged between 19 and 92 years old (Table 2).

Table 2. Frequency and percentage distribution of the characteristics of consumers participating in the co-design study (n=40)

"Think-aloud" interviews

The 40 interviews, which lasted from 20 to 75 minutes, identified a wide variety of issues and suggestions. Across all interviews, participants identified a total of 315 issues with the MINI-Q. The highest number of issues identified was from the first group of five consumers. The number of changes suggested to the tool reduced with each version of the MINI-Q, in conjunction with a somewhat reduced interview time (Table 3). Changes suggested concerned the wording, length and structure of sentences; order and type of response options; and formatting. Consumers also suggested including three additional medicine information questions. Additional statements added to the MINI-Q included “changes to my medicines, e.g. tablet colour, dose etc.,” “what impact taking my medicine will have on my lifestyle,” and an item on concurrent use of natural health products.

Table 3. Issues and changes made for each iteration

Six iterations of the MINI-Q were needed before only one new comment or suggestion was received. Iterations seven and eight were completed using a table. Overall, consumers found the MINI-Q easy to use, and there were few new comments suggesting possible changes (see Digital Issues section).

Issues about wording and the need to improve clarity led to refinement of the MINI-Q. For example, the introduction in the first iteration had 101 words and stated: “We would like to ask you what you would like to know about your medicines. People often have preferences about what information they would like to know about their medicines and how important this information is to them. This will help us understand what is important to you. There are no right or wrong answers. We are simply interested in what you think. Your answers will help our team to best understand and cater for your information needs. We may not always be able to meet all your requirements, but we will do our very best to meet them wherever possible.

This was finally modified and reduced to 46 words to read: “People often have preferences with the information they would like to know about their medicines. Your answers will help us understand what your preferences for medicines information are at the moment. There are no right or wrong answers; we are simply interested in what you think.

Table 4 provides a summary of the main sections of the MINI-Q and the key changes that were made as a result of the interviews and consumer feedback gathered during the "think-aloud" process. As seen from Table 4, the co-design process with consumers was a critical part of questionnaire development as over 30 edits were made across all questionnaire domains, ranging from the introduction to the wording of the items and the format of the questionnaire.

Table 4. Description of MINI-Q sections and a summary of the changes made

Digital issues

When the MINI-Q was transferred to the tablet device, a further 10 consumers were interviewed. Few problems with the MINI-Q itself were identified; however, some device-specific issues relating to the electronic format of the MINI-Q were found. These included a small sidebar that made it difficult for one consumer to scroll and navigate down the page. Another participant accidently opened a file on the tablet from the tool bar and then required help to close it, and another noted a preference for larger text. Another interviewee shared that he found it daunting to use the tablet at first, as he had not used one before and did not think he would be able to. He was surprised at how easy it was to read and understand. Further comments included “easy survey to use on the tablet” (consumer aged 71) and “easy to use and the tablet was simple to navigate” (consumer aged 21).
Generally, consumers read each question once and immediately provided an answer. A number of them experienced interpretation problems with questions assessing similar topics such as “what my medicine does” and “how my medicine works”.  As the “think-aloud” technique asked interviewees to verbalize all answers, problematic items, identified by a delayed response where a consumer appeared to read an item two or three times before providing a response, could be readily identified and followed up with sensitive probing by the researcher to ascertain the exact nature of the issue.


Interviews incorporating a staged iterative "think-aloud" process proved a useful technique for testing the usability and acceptability of a consumer-facing data collection tool and making tool iterative refinements in a time-efficient manner. The interviews resulted in many subtle refinements being made to the MINI-Q via a series of tool iterations. Many of the suggestions made either had not been considered by the research team or differing opinions existed within the team. Rather than the research team reaching a consensus on what wording or format was clearer, consumers themselves indicated the improvements needed. This led to a succession of improvements in the understandability of the MINI-Q that were led from a consumer perspective. These changes eventually resulted in a version of the MINI-Q that was deemed sufficiently understandable to be transformed into a digital version. The next sample of participants then completed the digital version of MINI-Q during the "think-aloud" interviews using a tablet supplied by the research team. We believe that this was a more inclusive, responsive and time-efficient way of receiving feedback than the traditional methods used during questionnaire design and development whereby often the first step is piloting one version of the tool with a small group of the target population (Polit & Beck, 2017).

With this method, clear communication between the researcher and interviewee is important. Spending time establishing a relationship with the participant and building rapport before a participant completed the MINI-Q was considered crucial by the researcher conducting the interviews. Establishing rapport enables the collection of rich data, i.e. comprehensive and truthful information, and a shared understanding of issues during the "think-aloud" interview process. This method also reduces the need to interpret participant responses as ambiguous since responses can be clarified during the interview. For consumers, rapport enabled them to feel comfortable answering questions and have open and easy communication and sharing. Similarly, establishing rapport with a healthcare professional increases patient satisfaction (Grobosch et al., 2018).

Increased consumer access to online information, including medicine information, brings challenges for nurses (Honey & Westbrooke, 2016). Consumers may need support to understand information they have located online, as well as assistance to discern if the information is relevant for them (Honey, et al., 2018; Kusch, et al., 2018; Sheridan, et al., 2015). Using tools, such as the MINI-Q, may support nurses, pharmacists and other health professionals to provide targeted information aligned to consumer priorities. In addition, providing the MINI-Q in a digital format could improve accessibility, as text can be enlarged if vision is impaired and a "read aloud" function can also be used. Having options to view the MINI-Q in different languages could further improve usefulness and accessibility of the tool, although information provided would likely have to be in the same language to ensure responsiveness to the consumers’ needs. The digital version may also be preferred by other groups. For example, there is evidence that young people prefer their medicine information in a digital format (Monkman et al., 2020). However, there have been reports of those with low health literacy using digital technologies less than those with higher health literacy (Levy et al., 2015).

As the MINI-Q is primarily intended for consumers to complete independently, consideration was given to the understandability of the MINI-Q by the development team. However, poor English literacy levels could still limit access to the MINI-Q as adequate skills in English reading and comprehension are required to complete it independently. In these cases, completion of the MINI-Q could be assisted by a health professional with the instructions, items and response options being read out and further explained if necessary. However, even if medicine information needs and preferences are identified and met, poor health literacy can be a barrier to understanding the medicine information provided (Kusch, et al., 2018). A North American study suggested that low health literacy, in terms of medicine information needs, results in poor understanding of medication information, forgetting to take medications, and ignoring side effects and instructions relating to food or interactions (Wali & Grindrod, 2016).
Nurses, pharmacists and other health professionals must be skilled in accessing and critically appraising, evidence to be able to answer consumer questions related to their medicines. These are the skills of evidence-based practice, yet research indicates that healthcare professionals vary in their ability to use these skills (Weng et al., 2013). This has implications for ensuring that evidence-based practice skills are included in the educational preparation of healthcare professionals, as well as in their continuing education. Being able to access credible medicine information will support healthcare professionals to better meet consumers' medicine information needs (Young, et al., 2018).

Limitations of this work include the small sample size, the use of purposive sampling, and the nature of the in-patient population that limits generalization of the findings to outpatients in the primary care setting. Purposive sampling aimed to ensure consumers represented the population characteristics and demographics of the study hospital catchment area. Such an approach in co-design is supported by Slattery et al. (2020), who considered co-design to be more effective and inclusive in reaching those who are sometimes omitted or under-represented. However, whilst purposive sampling led to the inclusion of consumers with diversity of ethnicities and ages, the sample size was small and cannot be considered representative. This study was undertaken with in-patients at one large urban hospital, and the participants were drawn only from general medical and surgical wards. Whilst these wards were selected to capture a range of consumers with a variety of health conditions, this approach may have limited participation of those with rarer health conditions. An area for future research and further development of the tool’s coverage and use will be to explore the medicine information needs of those with a broader range of health conditions, cultural backgrounds, ages and ethnicities. The next stage that is planned in the development of the MINI-Q is for a much larger group of health consumers with varying demographics and health conditions to complete the latest version of the MINI-Q independently across multiple sites. This should somewhat address this limitation, and a large sample may also support exploring reliability and validity.

The study reported here is part of Phase Three of the larger project. It is anticipated that the outcomes of this research project will be translated into practice and inform the development of services and strategies to address each patient’s specific medicine information needs. As mentioned above, the next step is to test the refined MINI-Q in a digital format using tablets with a large sample of consumers. In addition, feedback from nurses, pharmacists and other health professionals is needed to consider how the information arising from the completed MINI-Q can best be presented to allow health professionals to make use of it to meet the needs of consumers. This may be in a summarized or infographic format. Furthermore, this information could also be integrated into electronic health records. Long-term, it is hoped that the responses to the MINI-Q will be available to all nurses, pharmacists and other health professionals working in hospitals, so that individualized and targeted medicine information and education can be provided in the format the consumer prefers before discharge. Also, each consumer should be provided with a copy of their responses to share with other health professionals after discharge to facilitate communication between different points of care. Further research will be needed to assess the fit of the MINI-Q for community dwelling consumers.


Understanding consumers’ medicine information needs has the potential to benefit them as nurses, pharmacists and other healthcare professionals will be able to better meet their needs. The MINI-Q is a prototype medicine information needs assessment tool designed to collect data about the medicine information needs of individuals and present it to health professionals in a manner facilitating its use. Adopting a co-design approach and the "think-aloud" interview process with a small sample of consumers supported further development and refinement of the consumer-facing side of the MINI-Q. This study confirmed the consumer acceptability of the MINI-Q as later iterations of testing identified very few new issues. With further testing using a tablet and a larger sample of consumers, followed by co-designing the visualization of the outputs with health professionals, it is hoped this would maximize the potential of this tool prior to implementation into clinical practice.

Online Journal of Nursing Informatics

Powered by the HIMSS Foundation and the HIMSS Nursing Informatics Community, the Online Journal of Nursing Informatics is a free, international, peer reviewed publication that is published three times a year and supports all functional areas of nursing informatics.

Read the Latest Edition


Al-Jabr, H., Twigg, M. J., Saadvandi, R. A., & Desborough, J. A. (2018). Exploring what patients think when answering the Interpersonal Skills Questionnaire (ISQ): A 'think aloud' study. Research in Social and Administrative Pharmacy, 15(5), 619-622.

Amende, H., Aspden, T., Honey, M., Chan, A., Chang, R., Ashmore-Price, H., French, A. & Brackley, K. (2019, November 8-10). Determining patients’ medicines information needs efficiently – optimising a clinical assessment tool. Paper presented at the NZ Hospital Pharmacists Association Conference: Hui Taumata 2019: Back to the Future, Auckland, New Zealand.

Ashmore-Price, H., Aspden, T., Brackley, K., Chan, A., & Honey, M. (2017, September 22-24). Exploring medical patient medicines information need at Auckland District Health Board (ADHB) Paper presented at the New Zealand Hospital Pharmacists’ Association Conference, Auckland, New Zealand.

Aujla, N., Vedhara, K., Walker, M., & Sprigg, N. (2020). Evaluating a stroke-specific version of the Illness Perception Questionnaire–Revised, using the Think-Aloud method. Journal of Health Psychology, 25(12), 1989-2005.

Brackley, K., Aspden, T., Chan, A., Honey, M., Haua, R., Sheed, L., Chang, R., Ashmore-Price, H., Amende, H., French, A. & Horne, R. (2019, October 31st). The development of a patient-centred tool to meet medicine information needs. Paper presented at the Waitemata District Health Board Health Literacy Symposium, Auckland, New Zealand.

Chan, A. H. Y., Aspden, T., Brackley, K., Ashmore-Price, H., & Honey, M. (2020). What information do patients want about their medicines? An exploration of the perspectives of general medicine inpatients. BMC Health Services Research, 20(1131), 1-8. doi: 10.1186/s12913-020-05911-1.

Charters, E. (2003). The use of think-aloud methods in qualitative research. Brock Education Journal, 12(2), 68-82. doi:

Duggan, C., & Bates, I. (2000). The development and evaluation of a survey tool to explore patients' perceptions about their prescribed drugs and their need for drug information. International Journal of Pharmacy Practice, 8(1), 42-52.

Duggan, C., Bates, I., Sturman, S., Andersson, E., Åström, K., & Carlsson, J. (2002). Validation of a “desire for information” scale. International Journal of Pharmacy Practice, 10(1), 31-37.

Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215-251.

Featherall, J., Lapin, B., Chaitoff, A., Havele, S. A., Thompson, N., & Katzan, I. (2018). Characterization of patient interest in provider-based consumer health information technology: Survey study. Journal of Medical Internet Research, 20(4), e128.

Goodyear-Smith, F., Jackson, C., & Greenhalgh, T. (2015). Co-design and implementation research: Challenges and solutions for ethics committees. BMC Medical Ethics, 16(78), 1-5.

Grobosch, S., Kuske, S., Linnenkamp, U., Ernstmann, N., Stephan, A., Genz, J., Begun, A., Haastert, B., Szendroedi, J., Müssig, K., Burkart, V., Roden, M., Icks, A., for the GDS Group. (2018). What information needs do people with recently diagnosed diabetes mellitus have and what are the associated factors? A cross-sectional study in Germany. BMJ Open, 8(10), e017895.

Honey, M., Aspden, T., Brackley, K., Haua, R., Sheed, L., Ashmore-Price, H., & Chan, A. (2018). Patients' internet use in New Zealand for information about medicines: Implications for policy and practice. Journal of Health Policy and Technology, 7(2), 119-124.

Honey, M., & Westbrooke, L. (2016). Evolving national strategy driving nursing informatics in New Zealand. Studies in Health Technology and Informatics, 255, 183-187.

Hopkinson, D. (2018). Integration and perception of ehealth in the primary care setting. Value in Health, 21, S325.

Horne, R. (2017). The human dimension: Putting the person into personalised medicine. The New Bioethics, 23(1), 38-48.

Horne, R., Hankins, M., & Jenkins, D. P. (2001). The Satisfaction with Information about Medicines Scale (SIMS): A new measurement tool for audit and research. Quality and Safety in Health Care, 10(3), 135-140.

Horne, R., Weinman, J., & Hankins, M. (1999). The beliefs about medicines questionnaire: The development and evaluation of a new method for assessing the cognitive representation of medication. Psychology and Health, 14(1), 1-24.

Ioannidis, J. P. A. (2016). Why most clinical research is not useful. PLoS Medicine, 13(6), e1002049.

Kusch, M. K., Haefeli, W. E., & Seidling, H. M. (2018). How to meet patients' individual needs for drug information: A scoping review. Patient Preference and Adherence, 2018(12), 2339-2355.

Lee, L., El-Den, S., Horne, R., & Carter, S. R. (2019). Patient satisfaction with information, concerns, beliefs and adherence to topical corticosteroids. Patient Education & Counseling, 102(6), 1203-1209.

Levy, H., Janke, A. T., & Langa, K. M. (2015). Health literacy and the digital divide among older Americans. Journal of General Internal Medicine, 30(3), 284-289.

Mes, M. A., Chan, A. H., Wileman, V., Katzer, C. B., Goodbourn, M., Towndrow, S., Taylor, S. J. & Horne, R. (2019). Patient involvement in questionnaire design: Tackling response error and burden. Journal of Pharmaceutical Policy and Practice, 12(1), 1-4.

Monkman, H., Kushniruk, A. W., Borycki, E. M., Sheets, D., Barnett, J., & Nøhr, C. (2020). The medium is the message: How do young adults want digital medication information? Life, 10(12), 339.

Nakarada-Kordic, I., Hayes, N., Reay, S. D., Corbet, C., & Chan, A. (2017). Co-designing for mental health: Creative methods to engage young people experiencing psychosis. Design for Health, 1(2), 229-244.

Peute, L. W., de Keizer, N. F., & Jaspers, M. W. (2015). The value of retrospective and concurrent think aloud in formative usability testing of a physician data query tool. Journal of Biomedical Informatics, 55, 1-10.

Polit, D. F., & Beck, C. T. (2017). Essentials of nursing research: Appraising evidence for nursing practice (9th ed.). Wolters Kluwer Health/Lippincott Williams & Wilkins.

Sheed, L., Haua, R., Brackley, K., Honey, M., Aspden, T., & Chan, A. (2017, November 30-December 2). Exploring medicines information needs of patients receiving inpatient mental health services in a New Zealand hospital: Implications for adherence. Paper presented at the European Society for Patient Adherence, Compliance and Persistence (ESPACOMP) Annual Meeting, Budapest, Hungary.

Sheridan, N. F., Kenealy, T. W., Kidd, J. D., Schmidt-Busby, J. I. G., Hand, J. E., Raphael, D. L., McKillop, A. M. & Rea, H. H. (2015). Patients’ engagement in primary care: Powerlessness and compounding jeopardy: A qualitative study. Health Expectations, 18(1), 32-43.

Slattery, P., Saeri, A. K., & Bragge, P. (2020). Research co-design in health: A rapid overview of reviews. Health Research Policy and Systems, 18(17), 1-13.

Sze, W. T., Pudney, R., & Wei, L. (2018). A survey on inpatients’ preferred methods of receiving information about medicines in London, United Kingdom. Journal of Pharmacy Practice and Education, 1(13), 1-7.

Wali, H., & Grindrod, K. (2016). Don’t assume the patient understands: Qualitative analysis of the challenges low health literate patients face in the pharmacy. Research in Social and Administrative Pharmacy, 12(6), 885-892.

Weng, Y. H., Kuo, K. N., Yang, C. Y., Lo, H. L., Chen, C., & Chiu, Y. W. (2013). Implementation of evidence-based practice across medical, nursing, pharmacological and allied healthcare professionals: A questionnaire survey in nationwide hospital settings. Implementation Science, 8(112), 1-10.

Young, A., Tordoff, J., Leitch, S., & Smith, A. (2018). Do health professionals tell patients what they want to know about their medicines? Health Education Journal, 77(7), 762-777.

Zhang, L., Gallagher, R., Lowres, N., Orchard, J., Freedman, S. B., & Neubeck, L. (2017). Using the ‘Think Aloud’ technique to explore quality of life issues during standard Quality-of-Life Questionnaires in patients with atrial fibrillation. Heart, Lung and Circulation, 26(2), 150-156.

Author Bios

Michelle Honey
Dr. Michelle Honey is Associate Professor and Masters of Nursing Science programme director in the School of Nursing, University of Auckland, New Zealand (NZ). As a nurse, Michelle has experience in cardio-thoracic, intensive care, nurse education, and informatics. She has a PhD in nursing from the University of Auckland. Michelle has long been active in nursing informatics in New Zealand with leadership roles recognized by honorary member status of Health Informatics NZ (HiNZ) and fellowship status with the College of Nursing Aotearoa NZ. She continues to support the development of health informatics in NZ and serves on the HiNZ Nursing and Midwifery Executive Board and the NZ InterRAI Governance Board. Internationally, in 2004 Michelle received honorary member status of the International Medical Informatics Association Nursing Informatics group (IMIA-NI). Michelle is the NZ representative for the Technology Informatics Guiding Education Reform (TIGER) group and for IMIA-NI. She has recently been elected vice-chair of IMIA-NI.

Trudi Aspden
Dr. Trudi Aspden is a senior lecturer in Pharmacy Practice at the University of Auckland. Trudi graduated with a BPharm(hons), United Kingdom (UK), in 1991 and was awarded her PhD evaluating the use of chitosans in nasal peptide absorption systems in 1996 from the Department of Pharmaceutical Sciences, University of Nottingham. Trudi worked as a community pharmacist in the UK and New Zealand from 1992 to 2009 and has worked in academia since that time. Trudi’s research focusses on extending roles and services provided by community pharmacists to improve health outcomes and reduce inequities; pharmacy student recruitment; pharmacist and undergraduate cultural safety and competence; and the teaching of transferrable skills through art. She is a member of the Pharmaceutical Society of New Zealand and was awarded the 2013 Butland Award for Innovation in Teaching from the Faculty of Medical and Health Sciences at the University of Auckland.

Kim Brackley
Kim Brackley is Clinical Lead Pharmacist for education, training and practice research at the Auckland District Health Board. She holds an MSc in Clinical Pharmacy (London) and a PG diploma in Higher Education (NZ). Kim has over 30 years of experience in hospital pharmacy in the UK and NZ in both clinical practice and pharmacy education and is a past president of the New Zealand Hospital Pharmacists Association. Her research and practice interests include enabling effective medicine usage, supporting skill development, and the teaching of transferrable skills through art.

Hunter Amende
Hunter Amende was involved in this project as an undergraduate student completing his Bachelor of Pharmacy at the University of Auckland. He was funded by the project grant from A+ Trust as a summer student scholar. He has since completed his degree and is now working in practice.

Amelia French
Amelia French selected this project for her pharmacist internship in 2019, following completion of a Bachelor of Pharmacy (Hons) at the University of Auckland in 2018. She has since completed her internship and is currently working in practice.

Amy Chan
Dr. Amy Chan is a senior clinical research fellow at the School of Pharmacy, University of Auckland, NZ, and holds an honorary post at the Centre of Behavioural Medicine, University College London. She is a clinical pharmacist academic and holds a joint appointment between the University and Auckland District Health Board. Amy completed her PhD in Pharmacy in 2016, having investigated the effect of a digital adherence intervention on treatment adherence in asthma. Amy has nearly 15 years of experience in the public health service, where she led the clinical pharmacy service in mental health and was the co-convenor for the New Zealand Hospital Pharmacists' Association Mental Health Special Interest Group. Amy is currently the global lead for workforce transformation with the International Pharmaceutical Federation (FIP) and is the professional development and research lead for the Commonwealth Pharmacists’ Association (CPA). Amy also sits on the steering committee for ESPACOMP – the international organisation on adherence research.