INTRODUCTION
The use of online resources to provide and administer health care services is known as electronic health, or eHealth (World Health Organization, 2016). This technology seeks to provide the patient, health care service user, or client with greater responsibility, power, and information, helping them to take an active role in making decisions about their health (Scheibner et al., 2021). eHealth also contributes to greater efficiency and effectiveness and enhances the interaction between the patient and the primary and secondary health care provider (Abolade & Durosinmi, 2018). In mental health, various eHealth strategies are utilized in the design, evaluation, and practical implementation of internet interventions. The use of eHealth in mental health is known as eMental Health (Blankers, 2011).
A range of internet-based psychological interventions (IPIs) exist. These include telephone calls, videoconferencing, text messaging for cognitive behavioral interventions, text messages or applications that send reminders, automated messages with general and personalized information, interventions through social media, virtual reality, and gaming. They provide services that include psychoeducation or more complex interventions (Mohr et al., 2014).
Although most published studies on the use of the electronic strategies offered in mental health have focused on evaluating their efficiency and effectiveness, an additional area of interest is the identification of factors that enhance their adoption by patients, such as acceptability (Musiat et al., 2014; Simon et al., 2019; Sobowale et al., 2016).
The concept of acceptability is heterogeneous and can be confused with terms such as treatment satisfaction, engagement, usability, and feasibility (Ellis & Anderson, 2023; Ng et al., 2019), which can hamper its evaluation (Ellis & Anderson, 2023). For this research, acceptability refers to attitudes or beliefs about the use of this type of intervention (Molloy & Anderson, 2021; Schröder et al., 2015).
Evaluating the acceptability of IPIs is essential since it can influence adherence and intervention results (Santana & Fontenelle, 2011).
There are two types of procedures for developing assessment instruments to study acceptability as a construct: 1) acceptability assessment of different types of ICTs without a theoretical basis, and 2) theory-based acceptability assessment.
In regard to the first group, in 2010, Banna et al. (2010) explored perceptions about eHealth among the general population in Australia, finding that accessibility was the most important advantage, since it enables people to make decisions about their health. Dinesen et al. (2013) investigated patients with chronic obstructive pulmonary disease (COPD) to evaluate their attitudes toward telerehabilitation in the Danish TELEKAT project, finding four types of attitudes: indifference, learning, feelings of security, and motivation to engage in physical training. Lee et al. (2014) evaluated the attitudes and preferences of older adults in the United States undergoing warfarin treatment about the use of mHealth technology and health games for acquiring self-control skills. Their findings indicated that study participants believed that mHealth could be useful for managing medication, and that they needed help from their families in using technological devices. Simon et al. (2009) evaluated the opinions and preferences of patients in the United States concerning the electronic provision of health information. They found that patients were enthusiastic about the process, recognizing its capacity for improving the safety and quality of health care, although they also expressed concern about privacy and improper use of their data.
Kok et al. (2014) found that users in the Netherlands rated this type of intervention as acceptable in terms of difficulty, time spent on each module, and usefulness. In a study of the general population in England, Musiat et al. (2014) assessed the acceptability of different types of mental health care: 1) in-person therapy, 2) self-help books, 3) eMental Health (internet-based interventions), and 4) mMental Health (smartphone apps). They evaluated a variety of criteria, including whether they helped solve the problem, provided motivation to improve, were credible, accessible without waiting periods, and available at convenient times, at no cost, and in convenient locations. They also explored whether they could be used anonymously, whether they included individual support, provided feedback, and were adaptable to individual learning styles. They found that participants did not believe that computerized treatment or mental health apps met these criteria, except for accessibility, and that they were unlikely to use computerized treatments for mental health in the future.
Regarding the use of videoconferencing to provide psychotherapy, Morland et al. (2015) evaluated the acceptability of this resource among women with PTSD using the Telemedicine Satisfaction and Acceptability Scale (TSAS, Frueh et al., 2005). The results indicated that participants were satisfied with the intervention and would recommend it to family and friends. Olden et al. (2017) also administered the TSAS to patients with post-traumatic stress disorder receiving the intervention. They found high satisfaction with and acceptability of the clinical interaction in the videoconferencing intervention. These studies provided valuable information for understanding the acceptability of the electronic strategies offered in mental health but did not use the theory and models for the variables associated with information and communication technologies (ICT) in health care. These include the theory of reasoned action (Fishbein & Ajzen, 1980), the technology acceptance model (Davis, 1989), the extended technology acceptance model (Venkatesh & Davis, 2000), the innovation diffusion model (Rogers, 1963), and the unified theory of acceptance and use of technology (UTAUT; Venkatesh et al., 2003), as described in Table 1.
Some studies have used this theory. Jung and Loria (2010) investigated the acceptance of eHealth services among seniors in Sweden using the technology acceptance model (TAM), finding that 1) usefulness, perceived ease of use, compatibility of services with user needs, and trust in the service provider are the major determinants of intention to use the service, and 2) most of those interviewed expressed positive attitudes toward the use of eHealth services, finding them useful, convenient, and easy to use. In an evaluation of the acceptability of online therapy for depression in Indonesia, Arjadi et al. (2018) also used the TAM, find that people were open to using this type of intervention, and that personal acceptance of online services, understood as the degree of individual predisposition or an attitude reflecting the tendency to experiment with mobile health care technologies, regardless of the experience reported by others, is the strongest factor predicting use.
Zhang et al. (2015) used Rogers’ innovation diffusion model in a study of factors influencing the acceptance and use of electronic resources for health care among primary care patients in Australia. Their main findings included an association between a low adoption rate and the inability to use the service to make an online appointment, the preference of most patients to use the telephone, the incompatibility of the new service with patients’ preference for oral communication with a receptionist, and patient characteristics such as unfamiliarity with the internet and lack of experience with online health care services. Lamela et al. (2020) used the UTAUT to evaluate the acceptability of IPIs for depression among the Portuguese population, finding that the expectation of efficiency, social influence, and the stigmatization of depression were significantly associated with the acceptance of this type of intervention.
Many studies on the acceptability of IPIs have been conducted internationally. Among the Mexican population, exploratory and descriptive analyses have been conducted. Although they constitute an initial approach to the topic (Lara et al., 2022), they are not based on a specific theoretical framework that would provide a more in-depth understanding of acceptability or its relationship with other variables predicting the adoption of IPIs in the general population.
The literature review identified studies focused on assessing the acceptability of psychological service provision through technological tools that included items without a theoretical foundation to support their psychometric properties. It was not possible to locate psychometric instruments developed in Mexico to assess IPI acceptability, the study of which could benefit the implementation of IPIs for people with limited access to in-person services.
The purpose of the present study was to evaluate the acceptability of IPIs, with a focus on the construction and assessment of the psychometric properties of theory-based instruments. It was conducted in three stages. The first involved the design of items for the instrument based on the concepts of the TAM. The second focused on evaluating psychometric properties and exploring the dimensions of the instrument based on an exploratory factor analysis. The third used a confirmatory factor analysis to determine the factor structure of the instrument.
METHOD
Phase 1. Construction of items for the instrument
Design of the study
An exploratory-descriptive study was conducted to identify the key words and phrases for an instrument to assess the acceptability of IPI use in Mexico.
Participants
A convenience sample was formed of undergraduate students aged 18 years and over requiring psychological care.
Measurements
Six open, self-report questions were designed and administered based on 1) beliefs, 2) attitudes, and 3) subjective norms concerning the use of psychological interventions through information technologies. These included the following: “List the advantages of having access to a psychological service based on information technologies,” “List the negative adjectives associated with having access to a psychological service based on information technologies,” and “Describe who would disapprove if you used a psychological service based on information technologies.”
Procedure
Prospective participants were contacted at various faculties at a public university, informed of the objectives of the study, and told that participation was voluntary. The process took an average of ten minutes, and data collection was face to face.
Data analysis
A content analysis was conducted of participants’ responses (Neuendorf, 2019).
Phase 2. Analysis of Psychometric Properties and Exploratory Factor Analysis (EFA)
Study Design
A non-experimental, descriptive, exploratory analysis was conducted to evaluate the internal consistency and validity of an instrument to determine the acceptability of use of internet-based psychological interventions in Mexico.
Participants
A convenience sample was formed of university students contacted at various faculties in a public university.
Procedure
Prospective participants were contacted in person and told of the objectives of the project. Those who were interested filled in the informed consent form and subsequently completed the questionnaire. The process took an average of twenty minutes, and data collection was in-person and conducted on the university premises.
Measurements
Sociodemographic data questionnaire: four items about sex, age, academic major, and year students were enrolled in at university.
Questionnaire on the acceptance of internet-based psychological interventions: forty-two items with responses on a five-point Likert scale, where 1 is “completely disagree” and 5 is “completely agree.” The following definition of psychological intervention on the Internet was given at the beginning of the questionnaire: Psychological treatment that is not provided in a traditional face-to-face setting and instead uses information and communication technologies, specifically the Internet, with tools such as chat, video calls (Zoom, Skype) and self-help programs (interactive activities integrated in a program) with and without the intervention of a health professional.
Statistical analysis
Frequency, asymmetry, discrimination and item directionality were analyzed using Cronbach’s alpha, and exploratory factor analysis (EFA), Bartlett’s test of sphericity and the Kaiser-Meyer Olkin (KMO) test of sampling adequacy. Factor analysis was performed with the extraction of principal components and varimax rotation; items with factor loads of less than 0.4 or more than 0.4 in two or more factors were eliminated.
Phase 3. Confirmatory Factor Analysis
Design of the study
A non-experimental, descriptive, exploratory study was conducted to confirm the factor structure of the instrument.
Participants
A convenience sample was drawn from mental health service users contacted through universities in Mexico City.
Measurements
Sociodemographic questionnaire: twelve items on age, sex, educational attainment, profession, and residence.
Internet use questionnaire: sixteen questions used in other studies, exploring internet use, place of connection, ease of access, and activities on the internet. These included “Do you know how to use the internet?”, “Do you use the internet regularly?”, “Where do you connect to the internet?”, and “How often do you do the following activities on the internet?” (Tiburcio et al., 2018).
Questionnaire on the acceptability of internet-based psychological interventions: fourteen items with responses on a five-point Likert scale where 1 is “completely disagree” and 5 is “completely agree.” The following definition of psychological intervention on the Internet was given at the beginning of the questionnaire: Psychological treatment not conducted in a traditional face-to-face setting but through the use of information and communication technologies, specifically the Internet, using tools such as chats, video calls (Zoom, Skype) and self-help programs (interactive activities incorporated into a program) with and without the intervention of a health professional.
Procedure
Prospective participants were contacted in person and informed of the objectives of the project. Those who were interested answered the informed consent form and subsequently completed the questionnaire. The process took an average of twenty minutes. The data collection was face-to-face and conducted on university premises.
Statistical analysis
The confirmatory factor analysis (CFA) used the maximum likelihood estimation method. The goodness of fit indicators were: 1) χ² with degrees of freedom ≤ 5 for acceptable fit, ≤ 3 for perfect fit; root mean square error of approximation (RMSEA) ≤ .10 for weak fit, ≤ .08 for good fit, ≤ .05 for perfect fit; comparative fit index (CFI) ≥ .90 for acceptable fit, ≥ .95 for good fit, ≥ .97 for perfect fit; and Tucker Lewis index (TLI/NNFI) ≥ .90 for acceptable fit, ≥ .95 for good fit (Hu & Bentler, 1999). Internal consistency overall and for each factor were analyzed using Cronbach’s alpha. Data analysis was performed with Stata.
Ethical considerations
The study protocol was approved by the Committee on Research Ethics of the Ramón de la Fuente Muñiz National Institute of Psychiatry (CEI/C/015/2015).
RESULTS
Phase one
The sample included fifty participants: twenty-five female and twenty-five male students majoring in range of subjects at a public university. Their answers were grouped into four categories: 1) usefulness, 2) obstacles to use, 3) risks, and 4) approval of use (Table 2). Based on this analysis, forty-two items were drafted with responses on a five-point Likert scale, in an initial version of a questionnaire on the acceptability of online psychological interventions.
Phase two
The sample comprised 223 participants, 50.9% of whom were women ages18-26. The largest proportion were 20 years old (21.9%), followed by those ages 21 (21%) and 19 (17.4%). Participants were pursuing undergraduate degrees in law (15.6%), chemistry (12.1%), engineering (11%), education (6.7%), medicine (5.8%), biopharmaceutical chemistry (5.8%), architecture (5.4%), and other fields (37.4%).
Table 3 shows that the different response options were used in the forty-two items in the instrument. The asymmetry analysis found that nine items showed a typical bias and that thirty-three had a normal distribution. Eight items (10, 13, 18, 24, 30, 36, 37, 38) were eliminated based on the discriminant analysis because no significant difference was found in the group means. The directionality analysis found a clear trend in thirty-four items. Three items (11, 21, and 35) were eliminated based on the initial analysis of internal consistency.
The test of sampling adequacy found thirty-one items with values within the parameters, KMO = .90 and test of sphericity 3365.15 (df = 465, p =.001). Exploratory factor analysis found that seven components explained 51.6% of the variance, with eleven items with factor loads of less than .40 or showing loads in more than one factor being eliminated (1, 3, 8, 12, 14, 17, 32, 39, 40, 41, 42). Six items (5, 15, 19, 20, 27, 31) were eliminated that were grouped in a way that did not constitute a factor (Table 2). The reliability test of the fourteen items on the final scale showed an alpha of .89. The alpha for each of the factors was .89 for factor 1, .62 for factor 2, and .53 for factor 3 (Table 4).
Phase three
The sample comprised 201 participants, of whom 51.2% were male, 73.1% lived in Mexico City, and 78.1% were single. The largest proportion were students (48.8%), 56.7% of whom were undergraduate and 21.1% high school students. The results of the CFA showed the following measures of goodness of fit: χ² / df = 168.92/74 = 2.28, CFI = .935,
TLI = .920, and RMSEA = .080 (95% CI [.064, .096]). The analysis of internal consistency showed an alpha of .91 overall, .91 for factor 1, “approval of use”; .79 for factor 2, “perceived usefulness”; and .59 for factor 3, “perceived risk” (Figure 1).
DISCUSSION AND CONCLUSION
This study constructed and evaluated the psychometric properties of an instrument for exploring the acceptability of internet-based psychotherapy to users of mental health services. The results showed that the instrument has a good index of reliability (.86) and an internal structure of three factors: approval of use, perceived usefulness, and perceived risk. These are factors in the acceptance of technology model designed by Davis (1989), which has been extensively applied to predict the use of new technologies in different areas, including psychological care. In addition, it contributes to the importance mentioned in the introduction of having theory-based psychometric instruments to support their factor structure.
This study is one of the first in Mexico to provide a means of exploring the acceptability of IPIs. Acceptability assessment would make it possible to identify those who would adhere better to their treatment. It could also serve as an approach for providing psychoeducation to patients who perceive IPIs as risky so that they can benefit from them.
Developing an instrument with these characteristics is relevant as IPI use increased sharply as a result of the confinement measures due to the COVID-19 pandemic. This radical shift in psychological care provision from in-person to online was observed both in Mexico and internationally (de la Rosa-Gómez & Waldherr, 2023). This change has been maintained in several spheres of private consultation where tools such as Zoom, Skype, and WhatsApp are regularly used as a result of the multiple benefits offered by online psychotherapy. This includes increased privacy and decreased stigma towards psychological care, schedule flexibility, low costs, caution regarding therapist-patient proximity, better access for patients in remote areas to psychological care centers (Rojas-Jara et al., 2022).
Additionally, in Mexico, the implementation of online interventions is expected to expand in comparison with traditional interventions, since there are various online programs designed to treat mental health problems, such as the online self-help program for alcohol use (Schaub et al., 2021), the self-help program for drug use (Tiburcio et al., 2018), the self-help program for depression, the online self-help program to address the emotional health of adolescents in the pandemic (de la Rosa-Gómez et al., 2020), intervention for adults who lost a loved one to COVID-19 (Dominguez-Rodriguez et al., 2023); and psychological assistance via chats from a mental health platform in regard to COVID-19 (Arenas-Landgrave et al., 2022).
Moreover, Internet use is steadily increasing in Mexico. In 2016, the Mexican Internet Association reported that there were approximately 65 million internet users, increasing to 79 million by 2018, an annual growth rate of 17.5% vs. 12% respectively. Connection time also increased during this period, from seven hours and fourteen minutes a day to eight hours and twelve minutes. In both years, the principal online activities included social networks (79% vs. 89%), sending and receiving emails (70% vs. 84%), sending and receiving messages (68% vs. 83%), and searching the internet for information (64% vs. 82%; AMIPCI, 2016 ; 2018).
We recommend additional research to enable the identification of psychosocial and clinical variables that will predict greater acceptability, so that this instrument can serve as a screening tool. Further research should also complement the development of instruments for the evaluation of other ICTs, such as the use of WhatsApp as a means of communication between therapists and patients via messages or the use of mobile applications to monitor patients’ moods.
It is important to note that this study has limitations. Firstly, participants were not randomly selected. Some of them were already familiar with traditional psychological care and given that this study was conducted prior to the pandemic, some were unfamiliar with internet-based psychotherapy. Strengths of the study include the fact that it was based on the conditions of the Mexican population, and that its various phases were sufficient to obtain information on its psychometric properties.
It is important to note that the sample used to design the instrument primarily comprised college students, which affects the generalization of the results. In this respect, it would be useful to test the factor structure with other populations that could benefit from IPIs, such as the elderly.
This study found evidence for the validity and reliability of th questionnaire on the acceptability of online psychotherapy, one of the first Spanish-language psychometric instruments for evaluating this construct. Use of this instrument could yield multiple benefits, such as improving adherence to this type of intervention and increasing awareness among potential mental health service users of the possibilities of online psychotherapy.