INTRODUCTION
The use of oral anticancer agents is increasing worldwide and changing the treatment paradigm in oncology [1,2]. Self-administration of oral cytotoxic at home places the patient and caregivers on greater responsibility than infusion-based cancer therapies administered by a health care center [3]. Patients and family caregivers also receive less adherence and adverse effect monitoring supervision. Likewise, health workers face challenges in ensuring patient medication compliance and proper control of drug toxicities [4,5].
Medication adherence is a complex multidimensional process influenced by several factors such as the patient, the health care practitioner, and the health system [6]. Adherence to oral agents is vital for therapeutic efficacy because nonadherence is associated with poor treatment outcomes and consuming health care resources [7–9]. Several studies have shown that adherence to oral chemotherapy agents as a therapy for chronic diseases varies between 50% and 70% [10–15].
Interventions such as face-to-face counseling have been designed to improve adherence to oral chemotherapy [16–18]. However, most intended interventions are complex, time-consuming, labor-intensive, expensive, and require enormous resources, making them challenging to implement in clinical practice [19,20]. Technological interventions, such as mobile health, are becoming more common in health care and represent a novel approach that can support the unique needs of oncology patients [1,5]. It provides real-time data collection, cost-effectiveness, broad accessibility, and a helpful tool for delivering various educational and behavioral interventions while allowing healthcare providers to monitor behaviors and toxicity [1,21].
The development of mobile applications during chemotherapy treatment has proven essential for monitoring adherence, controlling adverse drug reactions, or providing patient self-care guidelines [1]. Badawy et al. [22] demonstrated that using a telephone application to improve medication adherence in adolescents with chronic health conditions was feasible, acceptable, and demonstrated modest efficacy. Mobile health interventions have also increased patient activation and engagement, making them a potential tool to enhance outcomes [23].
This study develops a digital health intervention based on a smartphone application with a user-centered design (UCD) approach. It is designed based on user needs, especially for cancer patients in Indonesia, to reinforce and improve patients’ understanding of the critical aspects of taking oral anti-cancer medications (OAMs). The Indonesian Ministry of Health reports that cancer prevalence reaches 1.79 per 1,000 population and as much as 136.2 per 100,000 population, which is expected to continue to increase [24]. Unfortunately, no commercial cancer applications are available in the Indonesian application market. Furthermore, no previous study is available on the overall development and usability of mobile applications that focus on monitoring the use and adherence of OAMs in Indonesia. This study aimed to comprehensively describe the process of designing, developing, and evaluating the usability and feasibility of smartphone applications, with a specific focus on cancer outpatients taking oral cytotoxic drugs.
MATERIALS AND METHODS
Setting
This mixed-methods study was conducted from January 2022 to April 2023 at the National Cancer Centre, Dharmais Cancer Hospital, Jakarta. The study was approved by the Ethics Committee of National Cancer Dharmais Hospital (Reference number 050/KEPK/V/2021).
Participants of the study
This study used purposive sampling to recruit respondents for user needs assessment, usability, and feasibility tests. The recruitment process was conducted during routine clinic visits. The participants obtained a face-to-face explanation of the study. Before any study procedure, all participants signed an informed consent form.
Participants for user need assessment and usability testing (Phase 1)
The participants in this phase included patients and their families and healthcare providers (pharmacists, doctors, and oncology nurses). Patients were eligible if they were cancer outpatients taking oral chemotherapy at least one cycle, usually used an application on a smartphone, were willing to participate in the study; and they signed the informed consent and showed no communication disorder. Pharmacists, oncologists, and nurses who participated in this study must have worked in ambulatory care with at least 2 years of experience.
Participants for feasibility test (Phase 2)
The participants in this phase were eligible if they were cancer outpatients on the second or third cycle of capecitabine therapy; they owned a smartphone and internet access at home; they had previously used an application on a smartphone; and they were willing to participate in the study and signed the informed consent. The exclusion criteria included not owning a smartphone, not using an Android-compatible device (Blackberry or an iOS device are not compatible with the apps), having cognitive impairments that prevented them from utilizing electronic devices, and declining to participate in the study.
Procedures
This study was conducted in two sequential phases. Phase I consisted of the smartphone app design and development process using a UCD approach and the evaluation of the app using the System Usability Scale (SUS). Phase II consisted of a feasibility test of the app using one group pre/post-test intervention design.
Phase 1: Development and evaluation of the smartphone app
Smartphone app development used a UCD approach in the design activities and processes focusing on user needs. Early in the development process, oncologist physicians, pharmacists, and patients were consulted as part of a UCD strategy. They were invited to participate in a semi-structured interview to explore user needs and barriers in oral cancer drugs, mainly related to adherence and the smartphone application’s expected feature and content requirements.
We designed the prototype based on the user need assessment and developed the app through an iterative process between the researchers and technical development teams. This application was created on the Android platform in Indonesia. The application design began with creating the Android wireframe and user interface (UI)/user experience (UX). The next stage was implementing the UI/UX design and coding of the mobile application, which was carried out using Android Studio by the mobile application development team. Applications developed were synchronized with the cloud. Application integration with the cloud database was built with a Representational State Transfer Representational-Application Programming Interface. The results of this stage were in the form of a software program for the Android operating system in the Android application package (APK) format, which would then be tested in the usability test.
This evaluation stage aimed to improve smartphone applications through an iteration process. We collected feedback from users through usability tests. The initial patient, family, and healthcare provider group was also invited to participate in the usability test. Participants with different levels of technological comfort were also recruited to increase generalizability. Researchers observed users while interacting with smartphone applications and guided the respondents whenever required. Respondents were asked to complete specific tasks in the application demo to get a UX for the application. The tasks on the app demo were as follows:
1. Download and install the mobile application prototype application on the respondent’s mobile device by downloading the APK file. The respondents had to download the APK file to install the SoBat Kanker on their mobile devices.
2. Open the application and register on the application.
3. Observe the application logos and existing menus.
4. Open the “Daftar Obat” menu and add one of your current drugs.
5. Open the information or video about what to do if they forget to take capecitabine medicine.
6. Open the “Laporan Efek Samping Obat” menu and enter data on side effects experienced.
7. Open the “Tanya Apoteker” menu and ask the pharmacist about therapy.
Furthermore, respondents were asked to uninstall the application from their mobile devices and to share their experiences as feedback by filling out an SUS questionnaire. The questionnaire contained ten questions representing the app’s use, perceived usefulness, and ease of use. The questionnaire had been validated in the Indonesian version and was reliable (Cronbach’s alpha = 0.841) [25]. Feedback from the usability test was then incorporated into the development and refinement cycle to improve the UX within the SoBat Kanker app. The research team reviewed multiple iterations of the app and provided the app’s appearance and functionality feedback to the technical team before the app was finalized. The app that has been refined and ready to use was uploaded to the Google Play Store for a pilot feasibility study.
Phase 2: Pilot feasibility study
A 6-week single-arm, prospective, pre-post-intervention study without a control group was conducted in this phase. In the intervention phase, patients used the SoBat Kanker app. The app was introduced to the participants during one in-person introduction session by a pharmacist, and they also got assistance downloading it from the Google Play Store and instructions on how to use it. We collected patients’ demographic and adherence baseline data. Patients were asked to complete the oral chemotherapy adherence scale at baseline and the sixth-week follow-up. This questionnaire contained 19 questions and had been validated in the Indonesian version (Cronbach’s alpha = 0.675).
Data analysis
Interviews were audio recorded, transcribed, and analyzed using qualitative content analysis to determine emerging codes, categories, and themes. Categorical variables were presented as frequency and percentage, and continuous variables were presented as mean and standard deviation. Statistical analyses were analyzed using the Statistical Program for Social Sciences. Pre- and postintervention adherence measurements were compared using the Wilcoxon signed-rank test. p values < 0.05 were considered statistically significant.
RESULTS
User-need assessment
Thirteen respondents were interviewed to explore user needs and obstacles. Respondents involved in this phase were patients and their families (n = 9), pharmacists (n = 2), doctors (n = 1), and nurses (n = 1).
Barriers to using oral cancer drugs at home
The main barriers identified were forgetting to take medication or taking medication not according to the schedule instructed by the doctor. In addition, side effects and their management hindered the use of OAM at home.
A patient stated about forgetting to take medicine.
“… I took my medicine on time, but sometimes I forgot to take it, and an hour later, I took it as soon as possible when I remembered. Everyday activities like cooking caused me to fail to take my medicine” (Participant 6).
Another patient also reported forgetting to refill the next cycle prescription.
“… I was confused about the schedule for refilling medication for the next cycle. I forgot when I had to return to the pharmacy to take medicine for my next period. At that time, I came to the pharmacy to refill the prescription, but the pharmacist could not serve me because the schedule did not match, so I had to go back and forth. Moreover, it wastes time and transportation costs” (Participant 8).
A doctor spoke about side effects as a barrier to the adherence factor.
“…, I think more about monitoring, especially side effects. Many patients have side effects, and then they do not comply. It will benefit doctors and pharmacists to collaborate to explain medication and side effects...” (Participant 3).
Feature requirements in the app
In our study, we also evaluated participants’ preferences for a mobile phone app that would help to promote adherence. Participants suggested several tools that help them in daily life that could be integrated into the app.
The participants said that a reminder feature is necessary.
“…. It would be good if a reminder were provided in the apps so that when you set the reminder, there will be a notification reminding you when to take medicine at certain hours” (Participant 10).
“A reminder to refill the medications for the next cycle is necessary. Pharmacy could provide for the following six-cycle schedule. Thus, what they wanted to do, they already knew the plans because patients often forget them. Suppose the patient refills the prescription too quickly. In that case, it will increase additional transportation costs, but if it is too late to refill the medicine, the patient can be late taking the medication. Well, that is troublesome” (Participant 5).
 | Figure 1. The example screenshots of the SoBat Kanker app. From the left (A) start page; (B) registration page; (C) homepage; and (D) list of patient medication. [Click here to view] |
Access to professional health care by chatting was also valued.
“…asking questions, consulting, or discussing with health workers, especially during an emergency, is very helpful and makes us feel unworried because we did not fully understand the effects of cancer drugs” (Respondent 13).
The patient and health worker felt reassured by the medication information.
“…information about drug side effects occurring, what to do, and how to deal with them is urgently needed. Patients do have to see a doctor, but initial treatment before seeing a doctor is also important” (Participant 13).
“It will be beneficial if doctors and pharmacists, through this application, collaborate to explain medication taking and drug side effects” (Participant 3).
Design and development of smartphone application prototype
This phase implemented all the contents and features highlighted in the user needs assessment. Figure 1 shows the example screenshots of the SoBat Kanker app. A summary of the features and functions of SoBat Kanker is described in Table 1.
Usability test of SoBat Kanker app
We collected feedback from users through usability tests using a questionnaire. Usability tests were conducted on 30 respondents, including patients or their families (n = 27) and healthcare providers (n = 3). All the respondents completed a total of seven tasks and a usability questionnaire. The respondent demographics are described in Table 2. The usability test of the SoBat kanker app based on SUS is at 85.33. It is indicated that the SoBat Kanker app has grade A and excellent system usability, which means it was acceptable. Table 3 represents the usability testing of the SoBat Kanker app.
 | Table 1. A summary of features and functions of SoBat Kanker. [Click here to view] |
 | Table 2. The sociodemographic of usability testing respondents (n = 30). [Click here to view] |
 | Table 3. The usability testing of the SoBat Kanker app (n = 30). [Click here to view] |
Feasibility pilot study of SoBat Kanker app
A total of 43 patients with capecitabine therapy between November 2022 and January 2023 were referred to potential study participants. Thirteen participants were excluded because they did not have a smartphone (n = 2), did not participate in the introduction session (n = 4), could not be contacted (n = 4), died (n = 1), and changed therapy (n = 2). Thirty patients were included in the study, received the app intervention, and completed the 6-week outcome measure. Figure 2 provides a summary of the recruitment and participant flow. The demographic of study participants in the feasibility test is presented in Table 4.
The participants who completed the SoBat Kanker app intervention within a 6-week study period had significantly higher adherence than baseline (p < 0.001). Preliminary pre-post-intervention findings at the 6-week follow-up are described in Table 5.
DISCUSSION
This study reports a novel smartphone application intended for cancer patients taking oral cytotoxic agents at home. This application contains features such as reminders, symptom reporting, self-management, and engagement with pharmacists. To the best of our knowledge, this app is the first to be developed and evaluated in Indonesia.
 | Figure 2. A summary of the recruitment and participant flow. [Click here to view] |
 | Table 4. Descriptive characteristics of study participants in feasibility test (n = 30). [Click here to view] |
 | Table 5. Pre-post-intervention changes in adherence outcome (n = 30). [Click here to view] |
The application was developed using the UCD approach. UCD is an iterative design method that actively seeks and incorporates user feedback to ensure the tool meets their needs and requirements [26]. The users involved in this study were patients and their families as caregivers who often accompanied the patients when using medicines at home and communicating with the health team. Healthcare providers, such as oncology doctors, nurses, and pharmacists, were also involved in obtaining input regarding cancer management at home. User involvement through interviews and feedback in this study provides essential direction for designing and developing interventions [27]. The participation of experts and users in developing apps has been discussed by Whittaker et al. [28]. Ensuring the involvement of patients as the actual target audience in clinical research is vital to effectively meet patient needs [29]. A participatory research paradigm aligns intervention goals more with end-user needs, health interventions are more widely accepted, and the intervention results are more easily transferable to practical settings [1]. An application that closely meets patient and healthcare provider expectations can improve app implementation in professional healthcare practice [30] and help improve participants’ health and well-being in the home setting [1,30].
We developed the SoBat Kanker application to improve patient medication adherence by providing automatic medication reminders personalized with medication usage information. Several studies have shown that active reminders can change patient behavior, reinforcing the benefits of medication adherence application [19,31]. A review by Santo et al. [32] found that only 56% of medication adherence applications accommodate flexible scheduling for medication reminders. Medication reminder with flexible scheduling allows the app’s personalization to suit individual needs. The medication alarm actively encourages participants to take their medication correctly and on time. It also aids ambulatory patients on long-term medication to improve medication adherence [33,34].
In addition, with these apps, patients periodically send adherence reports to the pharmacist so that the pharmacist can help tackle patients’ obstacles and noncompliance issues. The application is also equipped with a symptom-reporting feature, which will be received by pharmacists so that they can provide feedback and follow-up on acute or chronic symptoms on patients’ next visit to the hospital. This application also facilitates patient or caregiver communication with pharmacists regarding the use of drugs at home.
Educational features in the form of articles and videos are also available in the application so that users can access medication information at any time and address the lack of information regarding the therapy. This feature contains information, such as drug use, side effects, their management, and drug interactions. With the advancement in mobile technology, primarily through applications, monitoring of patient adherence, control of adverse drug reactions, or self-care guidelines, patients taking oral chemotherapy can be improved [1]. Smartphone application for medication adherence potentially consolidates the user’s medication-specific information, providing a more streamlined process to educate the individual about their disease or therapy [5].
The final phase of this study was the usability test. The usability test is fundamental to the design process of any user-centered interactive system. It ensures that the interactive system is adaptable, can be used by the users, is effective, efficient, and fits the purpose [32]. This test is also helpful for evaluating the suitability of the application to user needs [35]. Usability has been recognized as a critical element of good practices for developing digital applications [33]. Several published standards have identified usability as an essential criterion for assessing digital applications in the health field [36].
The usability test of SoBat Kanker showed that the overall design and layout of the apps were simple and user-friendly for participants because they succeeded in completing the assigned tasks. The perception that a mobile application is easy to use will encourage the users to utilize it despite their limited knowledge [36,37]. In addition, it is important that the app use and navigation can be clear even without instruction [26]. All respondents agreed that they quickly learned how to use SoBat Kanker, which is probably due to the simple, familiar, and consistent design of SoBat Kanker.
In the pilot test, the sample size was limited and involved participants with specific types of oral chemotherapy to assess the initial feasibility and acceptability of the apps. The results showed that the mean adherence score was significantly higher after 6 weeks of intervention than the baseline (p < 0.001). It may be due to the application’s features that can potentially improve patient behavior in taking medication. The Sobat Kanker app allows patients to take medicines on time by setting reminder medication, receive a one-click consultation on the issue, access medication information at any time, and report medication adherence periodically. It allows effective communication between patients and pharmacists [38,39].
This finding was also reported by the Collado-Borrel et al. [40] study, which used a smartphone app called e-OncoSalud app, in which adherence was significantly higher (p = 0.002) after they used the app. In addition, recent meta-analyses on the effectiveness of mobile device apps showed that app users demonstrated higher medication adherence [41–43]. Furthermore, the efficacy of a mobile app for monitoring chronic diseases was demonstrated in a randomized controlled trial (RCT) by Bloss et al. [44]. It was shown that using such applications improved self-controlled health indicators without increasing costs. This finding also correlates with the previous study, which demonstrated a significant increase in the percentage of adherent patients during the intervention phase using a web and smartphone-based medication self-management platform for chronically ill patients [38].
Study limitations and strengths
Some limitations have been identified in this study and need to be considered. Respondents who took part in the user interviews were a convenience sample, which can lead to bias. The feasibility study was not randomized, and the intervention was given to every participant; thus, it could not be generalized. On the other hand, this study employed vital stakeholder involvement from the beginning, including cancer patients, healthcare providers, and IT developers. This approach resulted in better user engagement. Engaging users from the beginning and combining user input with evidence-based concepts and content likely increases the potential for effective interventions. The study also established usability tests with good utility due to the easiness and acceptable system usage. A future large-scale RCT with long-term follow-up and clinical outcome measures is necessary to elaborate on UXs as well as assess efficacy. A qualitative interview for more data exploration is also recommended.
CONCLUSION
This study described the development of a user-centered smartphone app, SoBat Kanker, to support medication adherence in cancer patients. The usability study showed that the SoBat Kanker app was acceptable, useful, and easy to use, with excellent user satisfaction. The feasibility pilot study demonstrated that the apps improved medication adherence. A large-scale RCT, including efficacy assessment, is required for further research.
ACKNOWLEDGMENT
The authors would like to thank Lembaga Pengelola Dana Pendidikan (LPDP), Ministry of Finance, Republic of Indonesia.
AUTHOR CONTRIBUTION
All the authors made substantial contributions to the conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; agreed to submit to the current journal; gave final approval of the version to be published; and agreed to be accountable for all aspects of the work. All the authors are eligible to be authors as per the International Committee of Medical Journal Editors (ICMJE) requirements/guidelines.
FINANCIAL SUPPORT
The authors would like to thank Lembaga Pengelola Dana Pendidikan (LPDP), Ministry of Finance, Republic of Indonesia (No. SKPB1107/LPDP/LPDP.3/2023)
CONFLICTS OF INTEREST
The authors declare no conflicts of interest.
ETHICAL APPROVAL
Ethical approval was obtained from the Ethics Committee of the National Cancer Dharmais Hospital (Reference number 050/KEPK/V/2021).
DATA AVAILABILITY
All data generated and analyzed are included in this research article.
PUBLISHER’S NOTE
This journal remains neutral with regard to jurisdictional claims in published institutional affiliations.
REFERENCES
1. Magalhães B, Fernandes C, Martinez-Galiano JM, Santos C. Exploring the use of mobile applications by cancer patients undergoing chemotherapy: a scoping review. Int J Med Inform. 2020;144:104293. CrossRef
2. Jupp JCY, Sultani H, Cooper CA, Peterson KA, Truong TH. Evaluation of mobile phone applications to support medication adherence and symptom management in oncology patients. Pediatr Blood Cancer. 2018;65(11):e27278. CrossRef
3. Patel K, Foster NR, Farrell A, Le-Lindqwister NA, Mathew J, Costello B, et al. Oral cancer chemotherapy adherence and adherence assessment tools: a report from north central cancer group trial N0747 and a systematic review of the literature. J Cancer Educ. 2013;28(4):770–6. CrossRef
4. Wickersham KE. A study of medication-taking for patients with non-small cell lung cancer receiving oral targeted therapy. [Internet]. search.proquest.com; 2012. Available from: https://search.proquest.com/openview/20da6b09b2a6ece2b63521567788c1f6/1?pq-origsite=gscholar&cbl=18750&diss=y
5. Dayer L, Heldenbrand S, Anderson P. Smartphone medication adherence apps: potential benefits to patients and providers. J Am Pharm Assoc. 2013;53(4):345. CrossRef
6. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487–97. CrossRef
7. Lam WY, Fresco P. Medication adherence measures: an overview. Biomed Res Int. 2015;2015:1–12. CrossRef
8. Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J. 2003;20:54–61. CrossRef
9. Latremouille-Viau D, Guerin A, Gagnon-Sanschagrin P, Dea K, Cohen BG, Joseph GJ. Health care resource utilization and costs in patients with chronic myeloid leukemia with better adherence to tyrosine kinase inhibitors and increased molecular monitoring frequency. J Manag Care Spec Pharm. 2017;23(2):214–24. CrossRef
10. Bhattacharya D, Easthall C, Willoughby KA, Small M, Watson S. Capecitabine non-adherence: exploration of magnitude, nature and contributing factors. J Oncol Pharm Pract. 2012;18(3):333–42. CrossRef
11. Partridge AH, Avorn J, Wang PS, Winer EP. Adherence to therapy with oral antineoplastic agents. J Natl Cancer Inst. 2002;94(21):1652. CrossRef
12. Hartigan K. Patient education: the cornerstone of successful oral chemotherapy treatment. Clin J Oncol Nurs. 2003;7(6 Suppl):21–4. CrossRef
13. D’Amato S. Improving patient adherence with oral chemotherapy. Oncol Issues. 2008;23(4):42–5. CrossRef
14. Escalada P, Griffiths P. Do people with cancer comply with oral chemotherapy treatments? Br J Community Nurs. 2006;11(12):532–6. CrossRef
15. Nilsson JLG, Andersson K, Bergkvist A, Björkman I, Brismar A, Moen J. Refill adherence to repeat prescriptions of cancer drugs to ambulatory patients. Eur J Cancer Care (Engl). 2006;15(3):235–7. CrossRef
16. Colombo LRP, Aguiar PM, Lima TM, Storpirtis S. The effects of pharmacist interventions on adult outpatients with cancer: a systematic review. J Clin Pharm Ther. 2017;42(4):414–24. CrossRef
17. McCue DA, Lohr LK, Pick AM. Improving adherence to oral cancer therapy in clinical practice. Pharmacotherapy. 2014;34(5):481–94. CrossRef
18. Mackler E, Segal EM, Muluneh B, Jeffers K, Carmichael J. 2018 Hematology/oncology pharmacist association best practices for the management of oral oncolytic therapy: pharmacy practice standard. J Oncol Pract. 2019;15(7):E346–55. CrossRef
19. Vervloet M, Linn AJ, van Weert JCM, de Bakker DH, Bouvy ML, van Dijk L. The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature. J Am Med Informatics Assoc. 2012;19(5):696–704. CrossRef
20. Fishbein JN, Nisotel LE, MacDonald JJ, Amoyal Pensak N, Jacobs JM, Flanagan C, et al. Mobile application to promote adherence to oral chemotherapy and symptom management: a protocol for design and development. JMIR Res Protoc. 2017;6(4):e62. CrossRef
21. Haase J, Farris KB, Dorsch MP. Mobile applications to improve medication adherence. Telemed e-Health. 2017;23(2):75–9. CrossRef
22. Badawy SM, Barrera L, Sinno MG, Kaviany S, O’dwyer LC, Kuhns LM. Text messaging and mobile phone apps as interventions to improve adherence in adolescents with chronic health conditions: a systematic review. JMIR Mhealth Uhealth. 2017;5(5):1–17. CrossRef
23. Alberts NM, Badawy SM, Hodges J, Estepp JH, Nwosu C, Khan H, et al. Development of the in charge health mobile app to improve adherence to hydroxyurea in patients with sickle cell disease: user-centered design approach. JMIR Mhealth Uhealth. 2020;8(5):e14884. CrossRef
24. Kementerian Kesehatan RI. Penyakit Kanker di Indonesia Berada Pada Urutan 8 di Asia Tenggara dan Urutan 23 di Asia [Internet]. Jakarta, Indonesia: Direktorat Jenderal Pencegahan dan Pengendalian Penyakit Kementerian Kesehatan Republik Indonesia; 2019. Available from: http://p2p.kemkes.go.id/penyakit-kanker-di-indonesia-berada-pada-urutan-8-di-asia-tenggara-dan-urutan-23-di-asia/
25. Sharfina Z, Santoso HB. An Indonesian adaptation of the System Usability Scale (SUS). 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Malang, Indonesia; 2017. pp 145–8. CrossRef
26. Griffin L, Lee D, Jaisle A, Carek P, George T, Laber E, et al. Creating an mHealth app for colorectal cancer screening: user-centered design approach. JMIR Hum Factors. 2019;6(2):e12700. CrossRef
27. Testing U, Børøsund E, Mirkovic J, Clark MM, Ehlers SL. A stress management app intervention for cancer survivors: design, development, and usability testing. JMIR Form Res. 2018;2(2):e19, p1–16. CrossRef
28. Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mhealth interventions: examples from New Zealand. J Health Commun. 2012;17(SUPPL. 1):11–21. CrossRef
29. Skilton E, Aslam M, Yeung J, Gao F, Melody T. Embedding patient and public involvement within research—how to set up a research patient ambassador group within a NHS trust. J Intensive Care Soc. 2016;17(3):234–7. CrossRef
30. Moses JC, Adibi S, Shariful Islam SM, Wickramasinghe N, Nguyen L. Application of smartphone technologies in disease monitoring: a systematic review. Healthcare (Basel). 2021;9(7):1–19. CrossRef
31. Dayer LE, Shilling R, Van Valkenburg M, Martin BC, Gubbins PO, Hadden K, et al. Assessing the medication adherence app marketplace from the health professional and consumer vantage points. JMIR Mhealth Uhealth. 2017;5(4):e45. CrossRef
32. Santo K, Richtering SS, Chalmers J, Thiagalingam A, Chow CK, Redfern J. Mobile phone apps to improve medication adherence: a systematic stepwise process to identify high-quality apps. JMIR Mhealth Uhealth. 2016;4(4):e132. CrossRef
33. Brown W, Yen PY, Rojas M, Schnall R. Assessment of the health IT usability evaluation model (Health-ITUEM) for evaluating mobile health (mHealth) technology. J Biomed Inform [Internet]. 2013;46(6):1080–7. CrossRef
34. Chew S, Lai PSM, Ng CJ. Usability and utility of a mobile app to improve medication adherence among ambulatory care patients in Malaysia: qualitative study. JMIR Mhealth Uhealth. 2020;8(1):e15146. CrossRef
35. Bastien JMC. Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform [Internet]. 2010;79(4):e18–23. CrossRef
36. Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: a scoping review. Int J Med Inform [Internet]. 2019;126:95–104. CrossRef
37. Muhamat NA, Hasan R, Saddki N, Arshad MRM, Ahmad M. Development and usability testing of mobile application on diet and oral health. PLoS One [Internet]. 2021;16:1–21. CrossRef
38. Anglada-Martínez H, Martin-Conde M, Rovira-Illamola M, Sotoca-Momblona JM, Sequeira E, Aragunde V, et al. Feasibility and preliminary outcomes of a web and smartphone–based medication self-management platform for chronically ill patients. J Med Syst. 2016;40(4):1–14. CrossRef
39. Alasfour M, Almarwani M. The effect of innovative smartphone application on adherence to a home-based exercise programs for female older adults with knee osteoarthritis in Saudi Arabia: a randomized controlled trial. Disabil Rehabil [Internet]. 2022;44(11):2420–7. CrossRef
40. Collado-Borrell R, Escudero-Vilaplana V, Ribed A, Gonzalez-Anleo C, Martin-Conde M, Romero-Jimenez R, et al. Effect of a mobile app for the pharmacotherapeutic follow-up of patients with cancer on their health outcomes: quasi-experimental study. JMIR Mhealth Uhealth. 2020;8(10):e20480. CrossRef
41. Li A, Del Olmo MG, Fong M, Sim K, Lymer SJ, Cunich M, et al. Effect of a smartphone application (Perx) on medication adherence and clinical outcomes: a 12-month randomised controlled trial. BMJ Open. 2021;11(8):e047041. CrossRef
42. Armitage LC, Kassavou A, Sutton S. Do mobile device apps designed to support medication adherence demonstrate efficacy? A systematic review of randomised controlled trials, with meta-analysis. BMJ Open. 2020;10(1):e032045. CrossRef
43. Zhao J, Freeman B, Li M. Can mobile phone apps influence people’s health behavior change? An evidence review. J Med Internet Res. 2016;18(11):1–12. CrossRef
44. Bloss CS, Wineinger NE, Peters M, Boeldt DL, Ariniello L, Kim JY, et al. A prospective randomized trial examining health care utilization in individuals using multiple smartphone-enabled biosensors. PeerJ. 2016;2016(1):1–16. CrossRef