Classic test and generalizability theories are both useful for cross-cultural adaptation of an allergic rhinitis health-related quality of life questionnaire

Open AccessPublished:November 29, 2021DOI:https://doi.org/10.1016/j.waojou.2021.100612

      Abstract

      Background

      An instrument to assess Allergic Rhinitis (AR) Health-Related Quality of Life (HRQL) in adult patients was developed in Spain. No validated instrument is currently cross-culturally adapted for use in daily practice to assess HRQL in AR patients in Colombia.

      Purpose

      The aim of this study was to evaluate the measurement performance of an AR-HRQL specific questionnaire, ESPRINT-15 (Cuestionario ESPañol de Calidad de Vida en RINiTis), in Colombian adult patients with AR using the Classic Test Theory (CTT) and the Generalizability theory (G-theory) frameworks.

      Methods

      We conducted the cross-cultural adaptation in 2 stages. In stage 1, we evaluated comprehensibility, acceptability, and feasibility of ESPRINT-15 in healthy adults and adult patients with AR. In stage 2, we examined both reliability and validity of ESPRINT-15 scores using CTT and overall reliability applying the G-theory in adult patients with AR.

      Results

      For feasibility and acceptability, all items showed a higher than 95% level of understanding, and modifications in the original questionnaire were unnecessary. Reliability and validity using CTT showed a high internal consistency (Cronbach's alpha and Mc McDonald's omega = 0.95) and test-retest reliability (scores from 0.70 to 0.76). The overall reliability score using G-theory was 0.75, and G-coefficients scores associated with internal consistency and test-retest reliability measures were 0.96 and 0.61, respectively. Validity using exploratory factor analysis (EFA) identified 2 factors instead of the original ESPRINT-15 4 domains. However, confirmatory factor analysis (CFA) showed good fit regarding the original model.

      Conclusions

      The proposed cross-cultural adaptation of ESPRINT-15 showed good reliability and validity measures. Additionally, it was easy to use and administer. ESPRINT-15 can be used clinically and for research in Colombian adults' patients with AR. CTT and the G-theory can be used in epidemiological studies to adapt AR-HRQL questionnaires cross-culturally in adult patients with AR.

      Keywords

      Background

      The high prevalence of allergic rhinitis (AR) and its effects on health-related quality of life (HRQL) have positioned this disease as one of the main chronic respiratory diseases worldwide.
      • Bousquet J.
      • Van Cauwenberge P.
      • Khaltaev N.
      • Aria Workshop G.
      World Health O
      Allergic rhinitis and its impact on asthma.
      Globally, the prevalence of AR is around 400 million people and AR symptoms are not well recognized by patients;
      • Demoly P.
      • Bosse I.
      • Maigret P.
      Perception and control of allergic rhinitis in primary care.
      in Colombia, the prevalence of AR symptoms ranges from 29.5% to 33.9% for the entire population, and 29.2%–32.4% in the pediatric population.
      • Dennis R.J.
      • Caraballo L.
      • Garcia E.
      • et al.
      Prevalence of asthma and other allergic conditions in Colombia 2009-2010: a cross-sectional study.
      ,
      • Penaranda A.
      • Aristizabal G.
      • Garcia E.
      • Vasquez C.
      • Rodriguez-Martinez C.E.
      • Satizabal C.L.
      Allergic rhinitis and associated factors in schoolchildren from Bogota, Colombia.
      AR is associated with impaired social life, sleep, school, and work
      • Valero A.
      • Izquierdo I.
      • Giralt J.
      • Bartra J.
      • del Cuvillo A.
      • Mullol J.
      Rupatadine improves nasal symptoms, quality of life (ESPRINT-15) and severity in a subanalysis of a cohort of Spanish allergic rhinitis patients.
      and several studies have explored how AR may negatively affect sleep, concentration, performance at work, learning and school activities, social life, sexuality, and sports.
      • Stuck B.A.
      • Czajkowski J.
      • Hagner A.E.
      • et al.
      Changes in daytime sleepiness, quality of life, and objective sleep patterns in seasonal allergic rhinitis: a controlled clinical trial.
      • Juniper E.F.
      • Stahl E.
      • Doty R.L.
      • Simons F.E.
      • Allen D.B.
      • Howarth P.H.
      Clinical outcomes and adverse effect monitoring in allergic rhinitis.
      • Shedden A.
      Impact of nasal congestion on quality of life and work productivity in allergic rhinitis: findings from a large online survey.
      • Bender B.G.
      • Fisher T.L.
      Differential impacts of allergic rhinitis and allergy medication on childhood learning.
      • Bousquet J.
      • Bullinger M.
      • Fayol C.
      • Marquis P.
      • Valentin B.
      • Burtin B.
      Assessment of quality of life in patients with perennial allergic rhinitis with the French version of the SF-36 Health Status Questionnaire.
      • Kirmaz C.
      • Aydemir O.
      • Bayrak P.
      • Yuksel H.
      • Ozenturk O.
      • Degirmenci S.
      Sexual dysfunction in patients with allergic rhinoconjunctivitis.
      • Bonini S.
      • Bonini M.
      • Bousquet J.
      • et al.
      Rhinitis and asthma in athletes: an ARIA document in collaboration with GA2LEN.
      HRQL is a critical patient-important outcome measurement that describes how individuals and groups perceive physical and mental health over time.
      • Schipper H.
      • Clinch J.
      • Olweny C.
      Quality of life studies: definitions and conceptual issues.
      Moreover, HRQL data allows clinicians and patients to recognize and monitor the impact of diseases and might improve patients' commitment to disease management.
      • Ansotegui I.J.
      • Braido F.
      • Molinengo G.
      • et al.
      Cross-cultural adaptation and validation of the RhinAsthma patient perspective (RAPP) in Spanish.
      • Zhu R.
      • Wang J.
      • Wu Y.
      • et al.
      The allergic rhinitis control test questionnaire is valuable in guiding step-down pharmacotherapy treatment of allergic rhinitis.
      • Lim M.C.
      • Baiardini I.
      • Molinengo G.
      • Navarro-Locsin C.G.
      • Canonica G.W.
      • Braido F.
      The cross-cultural validation of the English version of RhinAsthma patient's perspective (RAPP).
      AR's significant impact on HRQL and the cost of HRQL costs suggests an important public health burden.,
      • Verheggen B.G.
      • Westerhout K.Y.
      • Schreder C.H.
      • Augustin M.
      Health economic comparison of SLIT allergen and SCIT allergoid immunotherapy in patients with seasonal grass-allergic rhinoconjunctivitis in Germany.
      For instance, AR has a significant economic burden on society; in the United States, the estimated annual cost of AR is about US$2-5 billion; and in Latin-American countries, intranasal corticosteroid treatment can cost at least US$230 per annum per patient.
      • Reed S.D.
      • Lee T.A.
      • McCrory D.C.
      The economic burden of allergic rhinitis: a critical evaluation of the literature.
      ,
      • Rodriguez-Martinez C.E.
      • Sossa-Briceno M.P.
      • Vladimir Lemos E.
      Cost-effectiveness analysis of mometasone furoate versus beclomethasone dipropionate for the treatment of pediatric allergic rhinitis in Colombia.
      In consequence, the World Allergy Organization (WAO) and the Allergic Rhinitis and its Impact on Asthma (ARIA) clinical practice guidelines recommend classifying the severity of AR based on HRQL measures.
      • Canonica G.W.
      • Cox L.
      • Pawankar R.
      • et al.
      Sublingual immunotherapy: world Allergy Organization position paper 2013 update.
      • Brozek J.L.
      • Bousquet J.
      • Agache I.
      • et al.
      Allergic rhinitis and its impact on asthma (ARIA) guidelines-2016 revision.
      • Brozek J.L.
      • Bousquet J.
      • Baena-Cagnani C.E.
      • et al.
      Allergic rhinitis and its impact on asthma (ARIA) guidelines: 2010 revision.
      While multiple specific AR-HRQL questionnaires exist, most of them are heavily influenced by the cultural characteristics. In consequence, it is critical to validate and adapt their use to new populations.
      • Juniper E.F.
      • Thompson A.K.
      • Ferrie P.J.
      • Roberts J.N.
      Validation of the standardized version of the rhinoconjunctivitis quality of life questionnaire.
      • Baiardini I.
      • Pasquali M.
      • Giardini A.
      • et al.
      Rhinasthma: a new specific QoL questionnaire for patients with rhinitis and asthma.
      • Juniper E.F.
      • Thompson A.K.
      • Ferrie P.J.
      • Roberts J.N.
      Development and validation of the mini rhinoconjunctivitis quality of life questionnaire.
      Cross-cultural adaptation of HRQL questionnaires is less expensive and less time-consuming than developing a new tool. It allows the use of HRQL questionnaires adapted in cross-national research. Such adaptation should be made using reliable methods that measure the new version's psychometric characteristics.
      • Guillemin F.
      • Bombardier C.
      • Beaton D.
      Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines.
      ,
      • Hunt S.M.
      • Alonso J.
      • Bucquet D.
      • Niero M.
      • Wiklund I.
      • McKenna S.
      Cross-cultural adaptation of health measures. European group for health management and quality of life assessment.
      The application of an instrument in another target population, another language, or another form of administration is considered a new situation. A well-known type of validation is cross-cultural validation, ie, validation when an instrument is applied in countries with different cultures and languages.
      • de Vet H.C.W.
      • Terwee C.B.
      • Mokkink L.B.
      • Knol D.L.
      Measurement in Medicine: A Practical Guide.
      Classic Test Theory (CTT) and the Generalizability theory (G-theory) have similarities and use measurement procedures to yield reliable data.
      • Wan C.
      • Li H.
      • Fan X.
      • et al.
      Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and Generalizability Theory.
      Particularly, G theory enables researchers to quantify and identify the sources of inconsistencies in observed scores that arise, or could arise, over replications of a measurement procedure.
      • Brennan R.L.
      Generalizability theory and classical test theory.
      Although both frameworks assess psychometric properties of HRQL questionnaires, their similarities in terms of reliability measures have never been compared in a cross-cultural adaptation study.
      Heine et al
      • Henrich J.
      • Heine S.J.
      • Norenzayan A.
      The weirdest people in the world?.
      described that some individual functional psychological characteristics vary across different people and societies. For example, the population of Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies exhibits different psychological characteristics when compared to the population of non-WEIRD societies. These differences have taken place in domains such as social decision making (fairness, cooperation, and punishment), independent versus interdependent self-concepts (and associated motivations), analytic versus holistic reasoning, and moral reasoning.
      • Henrich J.
      • Heine S.J.
      • Norenzayan A.
      The weirdest people in the world?.
      Two generic HRQL measurement questionnaires have been translated into Spanish and cross-culturally adapted to Colombia.
      • Lugo A.L.H.
      • García G.H.I.
      • Gómez R.C.
      Confiabilidad del cuestionario de calidad de vida en salud SF-36 en Medellín, Colombia.
      ,
      • Quintero C.A.
      • Lugo L.H.
      • García H.I.
      • Sánchez A.
      Validación del cuestionario KIDSCREEN-27 de calidad de vida relacionada con la salud en niños y adolescentes de Medellín, Colombia.
      Regarding AR, a variety of specific HRQL questionnaires exist, but none have been developed or adapted to measure HRQL in Colombian adults with AR. ESPRINT-15 (Cuestionario ESPañol de Calidad de Vida en RINiTis), designed in Spain, is a validated questionnaire developed to measure HRQL in Spanish adults with AR.
      • Valero A.
      • Izquierdo I.
      • Sastre J.
      • et al.
      ESPRINT-15 questionnaire (Spanish version): reference values according to disease severity using both the original and the modified ARIA classifications.
      Scores from ESPRINT-15 are associated with high internal consistency (Cronbach's alpha: 0.92) and reproducibility (intraclass correlation coefficient: 0.80).
      • Schipper H.
      • Clinch J.
      • Olweny C.
      Quality of life studies: definitions and conceptual issues.
      ESPRINT-15 has also demonstrated moderate to strong correlations to symptom score questionnaires such as visual scale analog and generic HRQL questionnaires.
      • Valero A.
      • Baro E.
      • Sastre J.
      • et al.
      Reference values for facilitating the interpretation of the ESPRINT-15 questionnaire (Spanish version).
      Such findings support the applicability of ESPRINT-15 as a tool that can provide reliable and valid HRQL measurements in Colombian adults with AR.
      In this article, considering that the Colombian population with AR has different cultural values than the Spanish population with the same disease, we conducted a cross-cultural adaptation process of the ESPRINT-15 in Colombian adults (over 18-years old) with AR using the using classic test theory and g-theory. We analyzed its psychometric measures such as comprehensibility, feasibility, acceptability, reliability, and validity.

      Methods

      From each participant, we obtained demographic data, place of residence, duration of AR, and severity of AR according to the ARIA criteria.
      • Bousquet J.
      • Khaltaev N.
      • Cruz A.A.
      • et al.
      Allergic rhinitis and its impact on asthma (ARIA) 2008 update (in collaboration with the world health organization, GA(2)LEN and AllerGen).
      We assessed asthma and its severity in each participant based on the Global Strategy for Asthma Management and Prevention (GINA) criteria.
      Global Initiativa for Asthma
      Global Strategy for Asthma Management and Prevention.
      The Total 4 Symptom Score (T4SS) was calculated as the sum of the 4 main nasal symptoms of rhinitis: nasal congestion, rhinorrhea, nasal itching, and sneezing. Symptoms were scored from 0 to 3 (0, no symptoms; 1, mild; 2, moderate; 3, severe), resulting in a T4SS ranging from 0 to 12. Increasing values are associated with increasing severity of disease.

       Overall study design

      ESPRINT-15 is a questionnaire that includes 15 items distributed in the following domains: symptoms (5 items), daily activities (3 items), sleep (3 items), psychological impact (3 items), and overall score (1 item). The questionnaire provides domain scores ranging from 0 to 6 and an overall score ranging from 0 to 5.8. Lower scores indicate a better quality of life. The measurement goal of ESPRINT-15 is both discrimination and evaluation.
      • Group E.S.
      • Investigators
      • Valero A.
      • et al.
      Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
      The ESPRINT-15 questionnaire that we used was provided by the original developer, the Spanish Society of Allergy and Clinical Immunology. In order to get a conceptual equivalence, the original Spanish version was reviewed before it was implemented in this study.
      We carried out a cross-cultural adaptation in 2 stages. Stage 1 examined the comprehensibility, acceptability, and feasibility of the ESPRINT-15 questionnaire. Stage 2 explored its reliability and validity of ESPRINT-15. All patients signed an informed consent before their inclusion in the study. Fig. 1 shows the steps developed during the cross-cultural adaptation.
      Fig. 1
      Fig. 1Development process of the cross-cultural adaptation for the ESPRINT-15 questionnaire in Colombia

       Stage 1: Comprehensibility, acceptability, and feasibility

      Stage 1 was carried out by 2 researchers (CGG, JJYN) fluent in Spanish and knowledgeable in allergic diseases and HRQL.

       Procedure

      We examined comprehensibility in 2 pilot tests at this stage. In the first pilot test, we wanted to review any preliminary comprehensibility problems with the questionnaire, and for this purpose we asked twelve adults who did not suffer from the disease to fill out the questionnaire. After reviewing their comments, we conducted a second pilot test including 20 adults with AR to judge comprehensibility, relevance, and completeness of the ESPRINT-15 questionnaire.
      • de Vet H.C.W.
      • Terwee C.B.
      • Mokkink L.B.
      • Knol D.L.
      Measurement in Medicine: A Practical Guide.
      We conducted the pilot tests in an Allergy Unit located in a private health care institution in Medellín, Colombia.
      In both pilot tests, we included 3 questions after each item. First, patients answered yes or no to the question “Did you understand the item?”. The other 2 questions were open-ended: “What did you not understand?” and “What suggestions do you have to change it?”. Upon questionnaire completion, a verbal pre-testing was carried out by asking the patients to comment on the questionnaire items and their perceived meaning. We reviewed the answered questionnaires for problematic items, responses, statements, and words in both stages.
      We assessed acceptability by asking the same 32 respondents whether they were willing to fill out the questionnaire by themselves. Feasibility was evaluated by measuring the completion time of ESPRINT-15.

       Stage 2: Reliability and validity

      Five co-authors (MM, LM, EM, CG, JYN) were trained to collect information from interviews, specifically clinical characteristics of the disease, but they did not interfere with questionnaire completion.

       Procedure

      We conducted the second stage at the same health care institution where stage 1 was developed. We enrolled new adult patients with AR when they attended the health care institution to start allergenic immunotherapy treatment. Patients were assessed twice with a 4-week interval between visits. This interval was chosen according to a time interval applied in a previous study to assess test-retest reproducibility and sensitivity to change of ESPRINT-15.
      • Group E.S.
      • Investigators
      • Valero A.
      • et al.
      Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
      For test-retest reproducibility, we considered that patients did not change their clinical condition between both visits because onset of clinical efficacy occurs within 2–4 months.
      • Penagos M.
      • Durham S.R.
      Duration of allergen immunotherapy for inhalant allergy.
      At both visits, patients were invited to fill in a hard copy version of the ESPRINT-15 questionnaire. All returned questionnaires were examined for missing answers using a consistent protocol performed by the 5 co-authors. Patients were telephoned, up to 3 times, to resolve ambiguous responses. If it was not possible to clarify the answers, the data were entered as ‘0ʼ in the database, resulting in participant exclusion.

       Sample size estimation

      We did not calculate a sample size calculation for stage 1. In stage 2, we applied a method developed to calculate the approximate number of patients required to obtain an exact confidence interval of the desired width for a certain type of intraclass correlation in one-way and two-way analysis of variance (ANOVA) model.
      • Bonett D.G.
      Sample size requirements for estimating intraclass correlations with desired precision.
      Setting the level of significance at α = 0.05, for intraclass correlation 0.7 (p˜I=0.7), with a width of 0.2 (ω = 0.2) and if each patient is measured by the same set of κ questionnaires twice (κ = 2) which are the only questionnaires of interest, 104 patients would be required (η = 104). According to the authors of the model, the sample size approximation is less least accurate for κ = 2 and p˜I≥ 0.7 and its accuracy can be improved by adding 5 p˜I to η in special cases. Therefore, we estimated a final sample size of 109 patients (η = 109). We applied a convenience sampling to include healthy Colombian adults and adult patients with AR in both stages.

       Statistical analysis

      Statistical analyses were performed using IBM SPSS 27 for Mac (IBM, Corp, Armonk, NY) while confirmatory facto analysis was performed using an IBM SPSS Amos 26 (IBM, Corp, Armonk, NY). To calculate generalizability coefficients, we utilized G-String IV software (available at https://healthsci.mcmaster.ca/merit/research/g_string_v). We used descriptive statistics of mean, standard deviation (SD), frequencies, and percentages to summarize the data.

       Comprehensibility

      We assessed comprehensibility in stage 1 to ensure that the original meaning of ESPRINT-15 questionnaire was not lost or altered while reaching cultural equivalence.
      • de Vet H.C.W.
      • Terwee C.B.
      • Mokkink L.B.
      • Knol D.L.
      Measurement in Medicine: A Practical Guide.
      We calculated an average number of “yes” responses for each ESPRINT item to the question, “Did you understand the previous item?”. We established a threshold of 95% to accept comprehensibility for each item.

       Acceptability and feasibility

      Acceptability refers to the question of whether or not patients were willing to do something, and feasibility refers to whether or not they were able to do it.
      • de Vet H.C.W.
      • Terwee C.B.
      • Mokkink L.B.
      • Knol D.L.
      Measurement in Medicine: A Practical Guide.
      To evaluate acceptability, we assessed completeness of data in stage with a criterion for missing data <10%. Feasibility was assessed by calculating the time patients took to complete the questionnaire both in stage 1 and stage 2.

       Reliability

      Broadly conceived, reliability provides measures of consistency of questionnaire scores. The reliability of ESPRINT-15 was assessed in stage 2 using both CTT and G-theory.
      Classical Test Theory (CTT) evaluates reliability by measuring internal consistency and reproducibility of scores. Internal consistency refers to the consistency among items on the questionnaire, while reproducibility (the test-retest reliability) establishes the stability of the questionnaire over time in a stable population.
      • Aaronson N.
      • Alonso J.
      • Burnam A.
      • et al.
      Assessing health status and quality-of-life instruments: attributes and review criteria.
      In the present study, we assessed internal consistency by Cronbach's alpha and McDonald's Omega coefficients for each domain as well as the overall test. McDonald's Omega is considered one the best alternatives to calculate reliability.
      • Trizano-Hermosilla I.
      • Alvarado J.M.
      Best alternatives to Cronbach's alpha reliability in realistic conditions: congeneric and asymmetrical measurements.
      ,
      • Revelle W.
      • Zinbarg R.E.
      Coefficients alpha, beta, omega, and the glb: comments on sijtsma.
      A Cronbach's alpha value between 0.7 and 0.9 was considered enough to achieve a proper level of internal consistency.
      • Tangarife J.
      • Arias J.
      Construcción y validación de escalas de medición en salud: revisión de propiedades psicométricas.
      Reproducibility was evaluated using intra-class correlations (ICC). To assess the internal consistency of ESPRINT-15 scores, we took the average score between the first and second measurements.
      Generalizability Theory. In addition to using CTT, we used G-theory
      • Brennan R.L.
      Generalizability Theory.
      ,
      • Chronbach L.
      • Gleser G.
      • Nanda H.
      • Rajaratnam N.
      The Dependability of Behavioral Measurements: Theory of Generalizability of Scores and Profiles.
      to determine the overall reliability of ESPRINT-15, as well as internal consistency of ratings. G-theory allows for the estimation of multiple sources of variance, and therefore provides a useful framework for estimating the reliability of various types of assessments. In the present study design, individual items (i) are nested in a survey domain (i:d) and are crossed with time (t) and patients (p).

       Validity

      Validity is the degree to which the questionnaire measures what it is supposed to measure.
      • Aaronson N.
      • Alonso J.
      • Burnam A.
      • et al.
      Assessing health status and quality-of-life instruments: attributes and review criteria.
      In stage 2, we conducted exploratory factor analysis (EFA). We applied the Bartlett's Sphericity Test and Kaiser–Meyer–Olkin (KMO) Test to determine if the data was suitable to perform an EFA. The extraction of the axes for the factor analysis was based on the choice of eigenvalues greater than 1 and parallel analyses were used to confirm how many factors to retain. A rotation of the axes was be carried out to obtain greater representativeness in terms of factor loads (rotation promax, varimax or quartimax). The items that sufficiently saturated the retained factors were selected, setting a limit of 0.30 or higher for the selection. The analysis was completed with a scree plot. We conducted a confirmatory factor analysis (CFA) via Structural Equation Modelling (SEM) to test the questionnaire's structure. We used statistical tests to determine and compare goodness of fit of the models in absolute fit indices including Chi-squared test, χ2/df ratio, and standardized root mean square residual (SRMR). We used for relative fit indices, the Tucker-Lewis index (TLI), and indices of fit like the root mean square error of approximation (RMSEA), Bentler's Comparative Fit index (CFI), and the coefficient of determination (CD). RMSEA values lower than 0.05 indicate a good fit to the data, values between 0.05 and 0.08 an acceptable fit, values between 0.08 and 0.10 a marginal fit, and values greater than 0.10 a poor fit.
      • Fabrigar L.R.
      • Wegener D.T.
      • MacCallum R.C.
      • Strahan E.J.
      Evaluating the use of exploratory factor analysis in psychological research.
      For the CFI, NFI, and TLI, values greater than 0.90 indicate a good fit to the data.
      • Bentler P.M.
      Comparative fit indexes in structural models.
      To extract the variances from the data and to examine the factor structure, we used robust maximum likelihood estimates (MLR).
      • Stevens J.P.
      Applied Multivariate Statistics for the Social Sciences.
      We analyzed the reliability and validity of ESPRINT-15 scores using the first 14 items of the ESPRINT-15 questionnaire. We excluded the last item from the analyses because it represents an overall state of health and the measurement scale for retrieving the score was different from that used in the other 14 items. An English version of the ESPRINT-15 questionnaire is shown in Supplement 1.

       Concurrent validity

      We assessed concurrent validity using Spearman's rho coefficient to establish the correlation between the first 14 items of the ESPRINT-15 questionnaire and the overall score (item 15).

       Discriminatory ability

      We performed a Kruskal-Wallis test to evaluate the discriminant validity of the questionnaire and to compare the different median values obtained from patients with different severity of AR. Finally, a Mann Whitney U test was also conducted to explore if the ESPRINT-15 domains values can discriminate between patients with AR and history of asthma compared with patients with AR without having asthma.

      Results

       Baseline socio-demographic and clinical characteristics

      Two hundred sixty-seven patients were included in the study; 32 patients were enrolled in stage 1 and 235 adult patients with AR in stage 2. Table 1 describes the characteristics of the patients including the classification of the severity of AR, the T4SS values, and the presence of asthma as comorbidity. In stage 1, the median age of patients was 32 (IQR, 22.75–43) and the majority were female (n = 21, 65.59%). Moderate/severe intermittent and persistent AR was prevalent in this sample (n = 5, 75% in each classification). All patients completed the questionnaire. In stage 2, the questionnaire was administered to 265 patients. Forty patients were excluded from the analysis due to inadequate questionnaire completion. In total, data from 235 patients who answered 2 times the ESPRINT-15 questionnaire without missing values was selected for this stage analysis. Most patients at this stage were females (n = 181, 77%), their median age was 28 (IQR, 22–36) with the majority living in an urban area (n = 181, 77.02%). The most common type of AR classification was persistent moderate/severe (n = 191, 81.28%) and most patients reported they had asthma under control (n = 84, 71.18%). In stages 1 and 2, the median T4SS values reported by patients were 5
      • Dennis R.J.
      • Caraballo L.
      • Garcia E.
      • et al.
      Prevalence of asthma and other allergic conditions in Colombia 2009-2010: a cross-sectional study.
      • Penaranda A.
      • Aristizabal G.
      • Garcia E.
      • Vasquez C.
      • Rodriguez-Martinez C.E.
      • Satizabal C.L.
      Allergic rhinitis and associated factors in schoolchildren from Bogota, Colombia.
      • Valero A.
      • Izquierdo I.
      • Giralt J.
      • Bartra J.
      • del Cuvillo A.
      • Mullol J.
      Rupatadine improves nasal symptoms, quality of life (ESPRINT-15) and severity in a subanalysis of a cohort of Spanish allergic rhinitis patients.
      • Stuck B.A.
      • Czajkowski J.
      • Hagner A.E.
      • et al.
      Changes in daytime sleepiness, quality of life, and objective sleep patterns in seasonal allergic rhinitis: a controlled clinical trial.
      • Juniper E.F.
      • Stahl E.
      • Doty R.L.
      • Simons F.E.
      • Allen D.B.
      • Howarth P.H.
      Clinical outcomes and adverse effect monitoring in allergic rhinitis.
      • Shedden A.
      Impact of nasal congestion on quality of life and work productivity in allergic rhinitis: findings from a large online survey.
      and 6 (4–9), respectively.
      Table 1Socio-demographic and clinical characteristics of Colombian adults with allergic rhinitis.
      Comprehensibility. Acceptability, and Feasibility (phase 1)Reliability and Validity (phase 2)
      Sample size, n32235
      Age, years, Me (IQR)32 (22.75–43)28 (22–36)
      Gender, male: female, n, (%)11 (34.41), 21 (65.59)54 (23), 181 (77)
      Classical AR classification n (%)
      Data reported for 20 patients with allergic rhinitis
       Perennial20 (100)
      Data reported for 20 patients with allergic rhinitis
      231 (98.30)
       Seasonal0 (0)
      Data reported for 20 patients with allergic rhinitis
      4 (1.70)
      ARIA AR classification, n (%)
      Data reported for 20 patients with allergic rhinitis
       Mild Intermittent5 (25)
      Data reported for 20 patients with allergic rhinitis
      19 (8.09)
       Moderate/severe Intermittent8 (40)
      Data reported for 20 patients with allergic rhinitis
      10 (4.26)
       Mild Persistent2 (10)
      Data reported for 20 patients with allergic rhinitis
      15 (6.38)
       Moderate/severe Persistent5 (25)
      Data reported for 20 patients with allergic rhinitis
      191 (81.28)
      GINA asthma symptoms, n (%)
       Yes20 (100)
      Data reported for 20 patients with allergic rhinitis
      118 (50.21)
       No0 (0)117 (49.79)
      GINA asthma classification, n (%)
       Well controlled20 (100)
      Data reported for 20 patients with allergic rhinitis
      84 (71.18)
       Partly controlled0 (0)
      Data reported for 20 patients with allergic rhinitis
      33 (27.96)
       Uncontrolled0 (0)
      Data reported for 20 patients with allergic rhinitis
      1 (0.86)
      Nasal symptoms present in last 2 weeks, n (%)
       Sneezing2 (1–2)
      Data reported for 20 patients with allergic rhinitis
      2 (1–2)
       Rhinorrhea1 (1–2)
      Data reported for 20 patients with allergic rhinitis
      2 (1–2)
       Nasal pruritus2 (1–2)
      Data reported for 20 patients with allergic rhinitis
      2 (1–3)
       Nasal congestion1 (0–2)
      Data reported for 20 patients with allergic rhinitis
      2 (1–3)
       Olfactory loss0.5 (0–2)
      Data reported for 20 patients with allergic rhinitis
      1 (0–3)
      Ocular symptoms present in last 2 weeks, n (%)
       Ocular pruritus2 (1–3)
      Data reported for 20 patients with allergic rhinitis
      2 (1–3)
       Watery eyes1 (0–2)
      Data reported for 20 patients with allergic rhinitis
      1 (0–2)
       Red eyes1 (0–2)
      Data reported for 20 patients with allergic rhinitis
      2 (1–3)
      T4SS, Me (IQR)5 (3–8)
      Data reported for 20 patients with allergic rhinitis
      6 (4–9)
      Habitation, n (%)
       Urban area32 (100)181 (77.02)
       Suburban area0 (0)38 (16.17)
       Rural area0 (0)9 (3.83)
       No reported0 (0)7 (2.98)
      Education level, n (%)
       Primary school3 (9.38)4 (1.70)
       Secondary school12 (37.50)24 (10.21)
       Technical institute7 (21.88)81 (3447)
       University10 (31.25)126 (53.62)
      AR: Allergic Rhinitis; GINA: Global Strategy for Asthma Management and Prevention; Me: Median; IQR: Interquartile Rank; SD: Standard Deviation; T4SS: Total 4 Symptoms Score.
      a Data reported for 20 patients with allergic rhinitis

       Comprehensibility

      The comprehensibility of ESPRINT-15 was easy as there were no problems with its understanding. Respondents found minor issues in items 3, 4, and 7 due to these items included some words that are culturally used in Spain. However, all items showed a higher than 95% level of comprehensibility in all respondents; therefore, we did not modify any items of the questionnaire. In general, respondents described that the open-ended questions were understandable and did not provide additional feedback to improve their understandability. The original items of the complete results of the pilot comprehension test are presented in Supplement 2.

       Acceptability

      Both healthy adults and patients with AR were instructed to fill out in the questionnaire. There were not missing data for all items of ESPRINT-15.

       Feasibility

      Overall, the questionnaire was well understood. Median time to completion was 6.5 (IQR, 4–10) minutes in stage 1, and 10 (IQR, 7–10) minutes in stage 2. Respondents did not report issues regarding the time spent to complete the ESPRINT-15 questionnaire.

       Reliability

       Classical test theory

       Internal consistency

      ESPRINT-15 achieved good internal consistency and acceptable test-retest reliability applying statistical methods of the classical test theory. Item correlations ranged from 0.49 to 0.83. The lower item correlation was reported for the overall score item (item 15). The overall consistency of the instrument was 0.95 for Cronbach's alpha and for McDonald's Omega. When each item was removed, the values of the alpha coefficient ranged between 0.94 and 0.95. In the case of the Omega coefficients, all coefficients reported lower values than the Cronbach's alpha ones with a range between 0.89 and 0.91. The removal of an item from the questionnaire did not reduce the level of overall consistency of the instrument (Table 2).
      Table 2Cronbach's alpha and McDonald's Omega values the ESPRINT-15 questionnaire
      ItemsItem scale correlationCronbach' s alpha (one item removed)McDonald's Omega (one item removed)
      Item 10.580.950,91
      Item 20.690.950,90
      Item 30.780.950,90
      Item 40.550.950,91
      Item 50.710.950,90
      Item 60.820.950,90
      Item 70.850.940,89
      Item 80.850.940,89
      Item 90.800.950,90
      Item 100.700.950,90
      Item 110.820.950,90
      Item 120.780.950,90
      Item 130.750.950,90
      Item 140.830.950,89
      Item 150.490.950,90

       Reproducibility (test-retest reliability)

      Two hundred thirty-five patients answered the questionnaire twice, with an interval of approximately 4 weeks. ICC achieved values ranged from 0.70 (95% CI 0.61–0.77) (symptoms domain) to 0.76 (95% CI 0.69–0.82) (psychological affectation domain) for individual domains scores (Table 3).
      Table 3Intraclass correlation coefficient.
      ESPRINT 15 domainsICC95% CI
      Symptoms0.700.61–0.77
      Daily activities0.710.63–0.78
      Sleeping0.740.66–0.80
      Psychological affectation0.760.69–0.82
      ICC: intraclass correlation coefficient. CI: confidence interval

       G-theory findings

      G-theory was used to estimate the amount of variance associated with different sources of error. Variance components were estimated for patients (p; n = 235), time (t; n = 2), domain (d; n = 4: symptoms, daily activities, sleeping, and psychological affection) and item (i; n = 14) nested in domain (i:d). The majority of variance (68%) was attributed to individual differences across patients, indicating that ESPRINT-15 was able to reliably differentiates between patients. Interaction between patient and time accounted for nearly 20% of variability in ESPRIT-15 scores, suggesting that differences between the two time periods varied depending on the patient. A small amount of variance was attributed to patients by domain interaction suggesting that domain scores varied across patients. A small amount of variance was attributed to other facets (<2%) and interactions between facets (<3%), suggesting that ESPRINT-15 scores did not vary substantially across time, or items nested in domains. Overall, only 1.6% of variance was attribute to random error. Table 4 displays the facets and variance components for ESPRINT-15.
      Table 4Generalizability theory.
      FactorsVariance component% Variance
      Patient (p)1.0468.22%
      Time (t)0.010.86%
      Domain (d)0.032.17%
      Item (i); d0.010.60%
      pt0.3019.82%
      pd0.03.16%
      pi:d0.031.84%
      td0.000.70%
      ti:d0.000.00%
      ptd0.031.78%
      pti:d0.021.58%
      Overall Generalizability Coefficient: 0.75
      Variance components were then used to calculate generalizability coefficients (G-coefficients). The overall reliability of ESPRINT-15 was 0.75, which suggests that ESPRINT-15 can generate highly reliable scores that can be used to differentiate between patients based on the AR-HRQL measure. We also calculated G-coefficients that correspond to measures of test-retest reliability (ie, average correlation between time 1 and time 2) and measures of internal consistency (ie, Cronbach's Alpha, or the average correlation between the 14 items: domains) by treating time or item: domain variance components as random facets. The G-coefficient associated with test-rest reliability was 0.61, while the G-coefficient associated with internal consistency of ESPRINT-15 scores was 0.96.

       Validity

      The first 14 items of ESPRINT-15 and their corresponding likely domains were considered for an EFA. Bartlett's sphericity test (χ2 3138.96, p < 0.000) indicated a correlation between the items of the questionnaire. Also, the KMO measurement of sampling adequacy (0.93) showed that data was suitable for the EFA. Initially, the EFA was conducted using principal axis factoring without rotation. Considering the eigenvalues' criterion (eigenvalues above 1), two factors were extracted (eigenvalues of 8.80, and 1.04), and the cumulative variance explained was 70.36%. Hence, an oblique (promax) rotation was performed. All factors were higher than 0.3 for all domains, suggesting that the item distribution and the domain framework would change in contrast to the original structure. Factor 1 included 8 items, ie, from items 1 to 5, and items 9 to 11, and they were entered with item loadings ranging from 0.87 to 0.51. This factor was designated physical function. Six items (items 6 to 8 and items 12 to 14) were entered for factor 2, which was designated as psychological function. Table 5 shows the rotated factor matrix obtained with factor loading of items, and supplement 3 displays the scree plot.
      Table 5Exploratory factor analysis.
      Factor
      Rotation converged in 3 iterations
      Original domains ESPRINT-15
      12
      Item 100.87Sleeping
      Item 110.75
      Item 90.72
      Item 30.70Symptoms
      Item 50.68
      Item 40.65
      Item 10.62
      Item 20.51
      Item 140.98Psychological affectation
      Item 120.98
      Item 130.87
      Item 60.61Daily Activities
      Item 80.370.57
      Item 70.430.49
      Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization.
      a Rotation converged in 3 iterations
      To obtain an adequate model for ESPRINT-15 in Colombia, CFA with MLR estimation was conducted. Two models were tested. The first model was the original ESPRINT-15 model that included 4 factors, and the second model included the 2 factors we obtained from the EFA. Fig. 2 shows the path diagram for the original ESPRINT-15 model and the correlations between observed and latent variables. The path diagram for the second model is included in Supplement 4. MLR was used to calculate the correlations between latent variables. Table 6 presents a summary of the model fit indices of ESPRINT-15. In model 1, the overall goodness-of-fit Chi-square was significant for the model (χ2 201.78, df 71, p < 0.000) with an RMSEA value of 0,09. These results are indicative of acceptable model fit. Likewise, model 1 obtained the lowest SRMR values and a CFI value larger than superior to 0.95 compared to model 2. These findings supported the satisfactory fit of this model.
      Fig. 2
      Fig. 2Factor structure of ESPRINT-15 in Colombian adults with AR. Items are represented as rectangles, latent variables (factors) as ellipses and loading onto factor loadings as arrows. The error terms for the observable variables are shown as a circle plus arrow for each factor. The model showed the original four-factor structure of the ESPRINT-15
      Table 6Confirmatory factor analysis estimates of the ESPRINT-15 (n = 235).
      Model 1 (original ESPRINT-15 model)Model 2 (new model from EFA)
      χ2201.78488.82
      df7176
      pp < 0.00p < 0.00
      CFI0.960.86
      TLI0.950.84
      RMSEA (90% CI)0.09 (0.07–0.10)0.15 (0.14–0.17)
      SRMR0.110.15
      CFI comparative fit index, EFA: exploratory factor analysis, TLI Tucker-Lewis index, RMSEA root-mean square error of approximation, SRMR standardized root-mean square residual
      These results were consistent with prior EFA and CFA and supported the original 4 domains model structure.

       Concurrent criteria validity

      Concurrent criteria validity was assessed through ESPRINT-15 and the overall score (item 15 of the instrument) correlation. These results are shown in Table 7. The correlations ranged between 0.21 and 0.49. The best correlation coefficient was found between the item 14 and the overall score (ρ: 0.49; 95% CI 0.38; 0.59).
      Table 7ESPRINT-15 correlations with overall score (item 15).
      Correlation valuesSpearman's rho (ρ)Significance95% CI
      LowerUpper
      Item 10.41<0.000.290.51
      Item 20.210.000.080.33
      Item 30.34<0.000.210.45
      Item 40.26<0.000.130.38
      Item 50.35<0.000.230.46
      Item 60.38<0.000.250.48
      Item 70.40<0.000.280.50
      Item 80.35<0.000.230.46
      Item 90.35<0.000.230.47
      Item 100.40<0.000.280.51
      Item 110.41<0.000.300.52
      Item 120.40<0.000.280.51
      Item 130.40<0.000.280.50
      Item 140.49<0.000.380.59
      Domain 10.39<0.000.270.50
      Domain 20.41<0.000.290.52
      Domain 30.43<0.000.320.53
      Domain 40.46<0.000.350.56
      CI: confidence interval

       Discriminatory ability

      We conducted a Kruskal-Wallis test to explore the discriminatory capacity of ESPRINT-15 between severities of the disease. We did not find statistical differences in all domains among patients with different levels of AR severity. The median values and p values for each domain were as follows: for the “sleeping” domain a median value of 2.80 (p = 0.28), for the “symptoms” domain a median value of 2.00 (p = 0.16), for the “psychological affectation” domain a median value of 1.66 (p = 0.66), and for the “daily activities” domain a median value of 1.66 (p = 0.41). In terms of the asthma history, there were no statistical differences among the two groups in the domain scores (Mann-Whitney U test; p > 0.005). The median of the ESPRINT-15 domains was similar in patients with asthma compared to patients without asthma (Supplement 5).

      Discussion

      Cross-cultural adaptation of questionnaires for measuring HRQL in a new country, culture, and/or language requires that there is equivalence between the original questionnaire and the new version.
      • Schipper H.
      • Clinch J.
      • Olweny C.
      Quality of life studies: definitions and conceptual issues.
      The ESPRINT-15 questionnaire is considered suitable for use in a variety of contexts, including clinical practice, observational studies, and clinical trials to assess HRQL in adult patients with AR.
      • Group E.S.
      • Investigators
      • Valero A.
      • et al.
      Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
      The aim of this study was to cross-culturally adapt the ESPRINT-15 questionnaire to be used in Colombia in patients with AR. To date, no specific HRQL questionnaire has been adapted and psychometrically validated in Colombian adults with AR. The present study examined the comprehensibility, acceptability, feasibility, reliability, and validity of ESPRINT-15 in Colombia applying 2 methodological theories. The results suggest that ESPRINT-15 has adequate factorial variance, construct validity, internal consistency, and test-retest reliability to be implemented in Colombian patients with AR.
      Overall, the cross-cultural adaptation of ESPRINT-15 was easy except for minor comprehensibility issues between the ESPRINT-15 Spain's questionnaire version and Colombian's version regarding items 3, 4, and 7. We observed comprehensibility problems because respondents did not understand some words that are culturally used in Spain (eg, "picor"). Most ESPRINT-15 items offer 2 alternative options to comprehend each question. Both alternative options are separated by the word “or” (“o” in Spanish) (eg, "el picor en la nariz o estornudos repetidos", "el picor de los ojos o tener que rascarse los ojos"). Therefore, whether respondents did not understand a specific word, they have an alternative to understand the item. We did not modify the original ESPRINT-15 items because the issues reported by the responders did not compromise comprehensibility of any ESPRINT-15 item. The response rate was 100% suggesting acceptability of the instrument. Questionnaire's feasibility was similar to the values reported in the development of ESPRINT questionnaire with 7.1 min of time to administer.
      • Group E.S.
      • Investigators
      • Valero A.
      • et al.
      Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
      Internal consistency, to measure the homogeneity of items on a test, was assessed using the Cronbach's alpha coefficient and the McDonald's omega coefficients. The Cronbach's alpha value and the McDonald's value for the Colombian version of the ESPRINT-15 was 0.95. The overall alpha values for the items of the questionnaire ranged from 0.94 to 0.95, suggesting that ESPRINT15 has a coherent construct to be applied in clinical setting.
      Values of McDonalds's Omega coefficient are considered a more robust measurement of internal consistency than values from Cronbach's Alpha as it corrects the underestimation bias of Cronbach's Alpha when the tau-equivalence assumption is disrupted.
      • Dunn T.J.
      • Baguley T.
      • Brunsden V.
      From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation.
      We obtained an overall McDonalds's Omega value of 0.95 suggesting a high correlation between single items and the overall score item of the Colombian version of the ESPRINT-15.
      The calculated ICC for assessing reliability were adequate in all domains suggesting good test-retest reliability (ICC from 0.70 to 0.76). The internal consistency values were similar to those reported in the original Spanish version for Cronbach's alpha from 0.69 to 0.90, as well as for test re-test reliability with ICC values from 0.63 to 0.83.
      • Valero A.
      • Alonso J.
      • Antepara I.
      • et al.
      Health-related quality of life in allergic rhinitis: comparing the short form ESPRINT-15 and MiniRQLQ questionnaires.
      The small variations in the internal consistency values and reproducibility across both studies can be the consequence of different aspects such as the population sampled, the methods of assessments and the intervals between assessments. According to our findings, ESPRINT-15 is a reliable questionnaire to measure health status.
      G-theory is a comprehensive framework that opened the perspectives of the measurement theories as it allows to estimate score consistency with reference to multiple sources of measurement error.
      • Vispoel W.P.
      • Morris C.A.
      • Kilinc M.
      Applications of generalizability theory and their relations to classical test theory and structural equation modeling.
      However, its application in measurement studies is not popular. Some reasons for its low applicability can its use of technical vocabulary, an overlooked link with CCT, and problems finding a software for doing a G-theory analysis.
      • Vispoel W.P.
      • Morris C.A.
      • Kilinc M.
      Applications of generalizability theory and their relations to classical test theory and structural equation modeling.
      In our study the G-theory reported that 68.2% of the variance was explained for the patients, comparable to the variance obtained in the CTT analysis. This finding reflects that AR patients are the primary source of variability in the score observed when the ESPRINT-15 was applied. It also suggests that there is a substantial variation in how patients self-appraise the effect of the AR disease in their life. We also found that time had an interaction with patients (0.30 of variance component, 19.82% of variance), likely due to the fact that allergy symptoms change over time. Although both measurement theories provided similar variance values, G-theory provides comprehensive multiple sources of variance, including interactions, at the same time.
      • Wan C.
      • Li H.
      • Fan X.
      • et al.
      Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and Generalizability Theory.
      G-Theory estimates how specific changes in a measurement procedure might impact score consistency.
      • Vispoel W.P.
      • Morris C.A.
      • Kilinc M.
      Applications of generalizability theory and their relations to classical test theory and structural equation modeling.
      We performed factor analysis to assess the internal structure of ESPRINT-15 when it was applied to Colombian adults with AR. In the EFA, according to the eigenvalues and scree plot, we identified a new model with 2 factors extracted. We labeled the domains psychological and physical domains based on the components both grouped. This new model suggested modifications in the original domains of ESPRINT-15 moving from 4 original domains to 2 domains. Thus, we conducted a CFA to compare the goodness of fit of the original model and the new model using several statistical tests. According to this analysis, the original model showed better parsimony, stability, and adequacy values than the new model. Thus, it was not necessary to make modifications to the original items. Our findings confirmed the original four domain structure of ESPRINT-15.
      • Group E.S.
      • Investigators
      • Valero A.
      • et al.
      Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
      We evaluated the concurrent validity of ESPRINT-15 measuring correlations between each item and the overall score (item 15). Correlations were positive but low. This finding is explained because the measurement scale for retrieving the score is different from the other 14 items.
      Discriminatory ability was addressed using the Kruskal-Wallis test and Mann Whitney U test. No differences were found between domain values and degrees of severity of the disease. This finding can be explained because the sample size in patients with mild intermittent, moderate/severe intermittent and mild persistent strata was low. Previous studies reported a good discriminatory capacity of ESPRINT-15 in adults' patients with AR,
      • Delgado J.
      • Davila I.D.
      • Dominguez-Ortega J.
      • Quirce S.
      • Marti-Guadano E.
      • Valero A.
      Quality of life in patients with respiratory allergy is influenced by the causative allergen.
      ,
      • Valero A.
      • Ferrer M.
      • Baro E.
      • et al.
      Discrimination between moderate and severe disease may be used in patients with either treated or untreated allergic rhinitis.
      and after 4 weeks of treatment with rupatadine.
      • Valero A.
      • Izquierdo I.
      • Giralt J.
      • Bartra J.
      • del Cuvillo A.
      • Mullol J.
      Rupatadine improves nasal symptoms, quality of life (ESPRINT-15) and severity in a subanalysis of a cohort of Spanish allergic rhinitis patients.
      Finally, the Mann Whitney U test did not show differences between patients with AR and with and without history of asthma. This result is similar to a previous finding reported in the process of developing the ESPRINT questionnaire
      • Group E.S.
      • Investigators
      • Valero A.
      • et al.
      Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
      and confirms that ESPRINT-15 was designed for assessing HRQL focused on AR symptoms but not symptoms related to asthma.
      In conclusion, and according to our findings, ESPRINT-15 can be used as a routine clinical tool to assess HRQL outcomes in Colombian adult patients with AR. The use of this questionnaire in clinical practice would help to understand the impact of AR on quality of life and would improve its management with a patient-centric approach.

       Strengths and limitations

      The strengths of our study are various. First, our sample size for the second stage allowed us to obtain reliable statistic results. Similarly, we addressed the statistical models via EFA and CFA with rigorous statistical methods for extraction and interpretation of data. Second, we found that ESPRINT-15 produced highly reliable scores, suggesting that the questionnaire can discriminate among patients with different AR-HRQL states. Third, our reliability findings using CTT, and G-theories were consistent between both techniques suggesting that these frameworks are useful for conducting cross-cultural adaptation studies, especially in chronic respiratory conditions such as AR. Fourth, we found that an interaction between time and AR patients accounted for significant variability among patients. This finding shows that ESPRINT-15 can be useful for clinicians as it will provide additional documentation regarding the effectiveness of treatments. There are some limitations in our study. First, we conducted one pilot test and it included a relatively small sample size. Further studies using adequate samples and multiple pilot tests for assessing comprehensibility, acceptability, and feasibility would provide useful additional evidence for the measurement performance of ESPRINT-15. Second, most patients in this study reported moderate/severe AR health status; thus, our findings may not be generalized in the context of patients with mild states.

       Future directions

      It is possible that ESPRINT-15 questionnaire requires additional adjustments such as simplifying its findings in 2 factors or reducing the number of items to improve its routine applicability in the clinical context. Likewise, this cross-cultural adaptation will allow us to start adapting other HRQL questionnaires to Colombia as well as in other Spanish speaking countries using CTT or G-theories frameworks. In addition, further studies should be implemented to obtain population reference values and the Minimal Clinically Important Difference (MCID), which will aid to improve its interpretability.

      Conclusion

      ESPRINT-15 was successfully cross-culturally adapted using both CTT and G-theory. Therefore, the ESPRINT-15 questionnaire is suitable for measuring HRQL in Colombian adult patients with AR. The construct validity, internal consistency and test re-test reliability were adequate obtaining an equivalent questionnaire to the original version to be applied in clinical contexts and epidemiological studies in Colombian adult patients with AR. G-theory that are useful for assessing reliability measures in cross-cultural adaptation studies.

      Abbreviations

      AR, allergic rhinitis; ARIA, Allergic Rhinitis and its Impact on Asthma; CFA, confirmatory factor analysis; CFI, confirmatory factor index; CTT, classic test theory; EFA, exploratory factor analysis; ESPRINT-15, Cuestionario ESPañol de Calidad de Vida en RINiTis; G-coefficients, generalizability coefficients; G-theory; generalizability theory; HRQL, health related quality of life; ICC, intra-class correlations; MCID, Minimal Clinically Important Difference; RMSEA, root mean square error of approximation; SD, standard deviation; SRMR, Standardized root mean square residual, TLI, Tucker–Lewis's index.

      Funding

      All authors acknowledge financial provided by the Vice Presidency for Research & Creation publication fund at Universidad de los Andes .

      Author's contributions

      Conceptualization: Juan José Yepes-Nuñez, Edison Morales Cardenas, Carolina Gómez-García, Lina Marcela Martínez, Madelen Manco Sepúlveda, Antonio Valero; Methodology: Juan José Yepes-Nuñez, Antonio Valero, Meghan M. McConnell; Formal analysis and investigation: Juan José Yepes-Nuñez, Meghan M. McConnell; Writing - original draft preparation: Juan José Yepes-Nuñez; Writing - review and editing: Juan José Yepes-Nuñez, Edison Morales Cardenas, Carolina Gómez-García, Antonio Valero, Meghan M. McConnell; Funding acquisition: Juan José Yepes-Nuñez, Edison Morales Cardenas; Resources: Juan José Yepes-Nuñez, Edison Morales Cardenas; Supervision: Juan José Yepes-Nuñez, Meghan M. McConnell.

      Ethics approval and consent to participate

      All procedures performed in this study were in accordance with the ethical standards of 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

      Submission declaration

      The contents of this paper have not been published previously and are not under consideration for publication in any other format.

      Availability of data and materials

      The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

      Authors' consent for publication

      All authors have seen and approved the submission of this version.

      Declaration of competing interest

      The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

      Acknowledgements

      The authors would like to thank Sociedad Española de Alergia e Inmunología Clínica (SEAIC) for facilitating the original version of the ESPRINT-15 questionnaire to this study. We also thank Sergio Moreno for his statistical support with the EFA, concurrent criteria and discriminatory ability analysis.

      Appendix A. Supplementary data

      The following is the Supplementary data to this article:

      References

        • Bousquet J.
        • Van Cauwenberge P.
        • Khaltaev N.
        • Aria Workshop G.
        • World Health O
        Allergic rhinitis and its impact on asthma.
        J Allergy Clin Immunol. 2001; 108: S147-S334
        • Demoly P.
        • Bosse I.
        • Maigret P.
        Perception and control of allergic rhinitis in primary care.
        NPJ Prim Care Respir Med. 2020; 30: 37
        • Dennis R.J.
        • Caraballo L.
        • Garcia E.
        • et al.
        Prevalence of asthma and other allergic conditions in Colombia 2009-2010: a cross-sectional study.
        BMC Pulm Med. 2012; 12: 17
        • Penaranda A.
        • Aristizabal G.
        • Garcia E.
        • Vasquez C.
        • Rodriguez-Martinez C.E.
        • Satizabal C.L.
        Allergic rhinitis and associated factors in schoolchildren from Bogota, Colombia.
        Rhinology. 2012; 50: 122-128
        • Valero A.
        • Izquierdo I.
        • Giralt J.
        • Bartra J.
        • del Cuvillo A.
        • Mullol J.
        Rupatadine improves nasal symptoms, quality of life (ESPRINT-15) and severity in a subanalysis of a cohort of Spanish allergic rhinitis patients.
        J Invest Allergol Clin Immunol. 2011; 21: 229-235
        • Stuck B.A.
        • Czajkowski J.
        • Hagner A.E.
        • et al.
        Changes in daytime sleepiness, quality of life, and objective sleep patterns in seasonal allergic rhinitis: a controlled clinical trial.
        J Allergy Clin Immunol. 2004; 113: 663-668
        • Juniper E.F.
        • Stahl E.
        • Doty R.L.
        • Simons F.E.
        • Allen D.B.
        • Howarth P.H.
        Clinical outcomes and adverse effect monitoring in allergic rhinitis.
        J Allergy Clin Immunol. 2005; 115: S390-S413
        • Shedden A.
        Impact of nasal congestion on quality of life and work productivity in allergic rhinitis: findings from a large online survey.
        Treat Respir Med. 2005; 4: 439-446
        • Bender B.G.
        • Fisher T.L.
        Differential impacts of allergic rhinitis and allergy medication on childhood learning.
        Pediatr Asthma Allergy Immunol. 1998; 12: 1-11
        • Bousquet J.
        • Bullinger M.
        • Fayol C.
        • Marquis P.
        • Valentin B.
        • Burtin B.
        Assessment of quality of life in patients with perennial allergic rhinitis with the French version of the SF-36 Health Status Questionnaire.
        J Allergy Clin Immunol. 1994; 94: 182-188
        • Kirmaz C.
        • Aydemir O.
        • Bayrak P.
        • Yuksel H.
        • Ozenturk O.
        • Degirmenci S.
        Sexual dysfunction in patients with allergic rhinoconjunctivitis.
        Ann Allergy Asthma Immunol. 2005; 95: 525-529
        • Bonini S.
        • Bonini M.
        • Bousquet J.
        • et al.
        Rhinitis and asthma in athletes: an ARIA document in collaboration with GA2LEN.
        Allergy. 2006; 61: 681-692
        • Schipper H.
        • Clinch J.
        • Olweny C.
        Quality of life studies: definitions and conceptual issues.
        in: Spilker B. Quality of Life and Pharmacoeconomics in Clinical Trials. Lippincot-Raven Press, Philadelphia1990: 11-23
        • Ansotegui I.J.
        • Braido F.
        • Molinengo G.
        • et al.
        Cross-cultural adaptation and validation of the RhinAsthma patient perspective (RAPP) in Spanish.
        Allergol Immunopathol (Madr). 2020; 48: 165-169
        • Zhu R.
        • Wang J.
        • Wu Y.
        • et al.
        The allergic rhinitis control test questionnaire is valuable in guiding step-down pharmacotherapy treatment of allergic rhinitis.
        J Allergy Clin Immunol Pract. 2019; 7: 272-278
        • Lim M.C.
        • Baiardini I.
        • Molinengo G.
        • Navarro-Locsin C.G.
        • Canonica G.W.
        • Braido F.
        The cross-cultural validation of the English version of RhinAsthma patient's perspective (RAPP).
        J Asthma. 2020; 57: 680-686
      1. Chronic respiratory diseases.
        • Verheggen B.G.
        • Westerhout K.Y.
        • Schreder C.H.
        • Augustin M.
        Health economic comparison of SLIT allergen and SCIT allergoid immunotherapy in patients with seasonal grass-allergic rhinoconjunctivitis in Germany.
        Clin Transl Allergy. 2015; 5: 1
        • Reed S.D.
        • Lee T.A.
        • McCrory D.C.
        The economic burden of allergic rhinitis: a critical evaluation of the literature.
        Pharmacoeconomics. 2004; 22: 345-361
        • Rodriguez-Martinez C.E.
        • Sossa-Briceno M.P.
        • Vladimir Lemos E.
        Cost-effectiveness analysis of mometasone furoate versus beclomethasone dipropionate for the treatment of pediatric allergic rhinitis in Colombia.
        Adv Ther. 2015; 32: 254-269
        • Canonica G.W.
        • Cox L.
        • Pawankar R.
        • et al.
        Sublingual immunotherapy: world Allergy Organization position paper 2013 update.
        World Allergy Organ J. 2014; 7: 6
        • Brozek J.L.
        • Bousquet J.
        • Agache I.
        • et al.
        Allergic rhinitis and its impact on asthma (ARIA) guidelines-2016 revision.
        J Allergy Clin Immunol. 2017; 140: 950-958
        • Brozek J.L.
        • Bousquet J.
        • Baena-Cagnani C.E.
        • et al.
        Allergic rhinitis and its impact on asthma (ARIA) guidelines: 2010 revision.
        J Allergy Clin Immunol. 2010; 126: 466-476
        • Juniper E.F.
        • Thompson A.K.
        • Ferrie P.J.
        • Roberts J.N.
        Validation of the standardized version of the rhinoconjunctivitis quality of life questionnaire.
        J Allergy Clin Immunol. 1999; 104: 364-369
        • Baiardini I.
        • Pasquali M.
        • Giardini A.
        • et al.
        Rhinasthma: a new specific QoL questionnaire for patients with rhinitis and asthma.
        Allergy. 2003; 58: 289-294
        • Juniper E.F.
        • Thompson A.K.
        • Ferrie P.J.
        • Roberts J.N.
        Development and validation of the mini rhinoconjunctivitis quality of life questionnaire.
        Clin Exp Allergy. 2000; 30: 132-140
        • Guillemin F.
        • Bombardier C.
        • Beaton D.
        Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines.
        J Clin Epidemiol. 1993; 46: 1417-1432
        • Hunt S.M.
        • Alonso J.
        • Bucquet D.
        • Niero M.
        • Wiklund I.
        • McKenna S.
        Cross-cultural adaptation of health measures. European group for health management and quality of life assessment.
        Health Pol. 1991; 19: 33-44
        • de Vet H.C.W.
        • Terwee C.B.
        • Mokkink L.B.
        • Knol D.L.
        Measurement in Medicine: A Practical Guide.
        Cambridge University Press, Cambridge2011
        • Wan C.
        • Li H.
        • Fan X.
        • et al.
        Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and Generalizability Theory.
        Health Qual Life Outcome. 2014; 12: 82
        • Brennan R.L.
        Generalizability theory and classical test theory.
        Appl Meas Educ. 2010; 24: 1-21
        • Henrich J.
        • Heine S.J.
        • Norenzayan A.
        The weirdest people in the world?.
        Behav Brain Sci. 2010; 33 (discussion -135): 61-83
        • Lugo A.L.H.
        • García G.H.I.
        • Gómez R.C.
        Confiabilidad del cuestionario de calidad de vida en salud SF-36 en Medellín, Colombia.
        Revista Facultad Nacional de Salud Pública. 2006; 24: 37-50
        • Quintero C.A.
        • Lugo L.H.
        • García H.I.
        • Sánchez A.
        Validación del cuestionario KIDSCREEN-27 de calidad de vida relacionada con la salud en niños y adolescentes de Medellín, Colombia.
        Rev Colomb Psiquiatr. 2011; 40: 470-487
        • Valero A.
        • Izquierdo I.
        • Sastre J.
        • et al.
        ESPRINT-15 questionnaire (Spanish version): reference values according to disease severity using both the original and the modified ARIA classifications.
        J Invest Allergol Clin Immunol. 2013; 23: 14-19
        • Valero A.
        • Baro E.
        • Sastre J.
        • et al.
        Reference values for facilitating the interpretation of the ESPRINT-15 questionnaire (Spanish version).
        J Invest Allergol Clin Immunol. 2009; 19: 396-403
        • Bousquet J.
        • Khaltaev N.
        • Cruz A.A.
        • et al.
        Allergic rhinitis and its impact on asthma (ARIA) 2008 update (in collaboration with the world health organization, GA(2)LEN and AllerGen).
        Allergy. 2008; 63: 8-160
        • Global Initiativa for Asthma
        Global Strategy for Asthma Management and Prevention.
        2021 (Available from:)
        • Group E.S.
        • Investigators
        • Valero A.
        • et al.
        Development and validation of a new Spanish instrument to measure health-related quality of life in patients with allergic rhinitis: the ESPRINT questionnaire.
        Value Health – J Int Soc Pharm Outcome Res. 2007; 10: 466-477
        • Penagos M.
        • Durham S.R.
        Duration of allergen immunotherapy for inhalant allergy.
        Curr Opin Allergy Clin Immunol. 2019; 19: 594-605
        • Bonett D.G.
        Sample size requirements for estimating intraclass correlations with desired precision.
        Stat Med. 2002; 21: 1331-1335
        • Aaronson N.
        • Alonso J.
        • Burnam A.
        • et al.
        Assessing health status and quality-of-life instruments: attributes and review criteria.
        Qual Life Res – Int J Qual Life aspect Treatm Care Rehabil. 2002; 11: 193-205
        • Trizano-Hermosilla I.
        • Alvarado J.M.
        Best alternatives to Cronbach's alpha reliability in realistic conditions: congeneric and asymmetrical measurements.
        Front Psychol. 2016; 7: 769
        • Revelle W.
        • Zinbarg R.E.
        Coefficients alpha, beta, omega, and the glb: comments on sijtsma.
        Psychometrika. 2008; 74: 145
        • Tangarife J.
        • Arias J.
        Construcción y validación de escalas de medición en salud: revisión de propiedades psicométricas.
        Arch Med. 2015; 11: 1-10
        • Brennan R.L.
        Generalizability Theory.
        Springer-Verlag, New York2001
        • Chronbach L.
        • Gleser G.
        • Nanda H.
        • Rajaratnam N.
        The Dependability of Behavioral Measurements: Theory of Generalizability of Scores and Profiles.
        John Wiley & Sons, New York1972
        • Fabrigar L.R.
        • Wegener D.T.
        • MacCallum R.C.
        • Strahan E.J.
        Evaluating the use of exploratory factor analysis in psychological research.
        Psychol Methods. 1999; 4: 272-299
        • Bentler P.M.
        Comparative fit indexes in structural models.
        Psychol Bull. 1990; 107: 238-246
        • Stevens J.P.
        Applied Multivariate Statistics for the Social Sciences.
        5ed. Routledge Taylor & Francis Group, New York2009
        • Dunn T.J.
        • Baguley T.
        • Brunsden V.
        From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation.
        Br J Psychol. 2014; 105: 399-412
        • Valero A.
        • Alonso J.
        • Antepara I.
        • et al.
        Health-related quality of life in allergic rhinitis: comparing the short form ESPRINT-15 and MiniRQLQ questionnaires.
        Allergy. 2007; 62: 1372-1378
        • Vispoel W.P.
        • Morris C.A.
        • Kilinc M.
        Applications of generalizability theory and their relations to classical test theory and structural equation modeling.
        Psychol Methods. 2018; 23: 1-26
        • Delgado J.
        • Davila I.D.
        • Dominguez-Ortega J.
        • Quirce S.
        • Marti-Guadano E.
        • Valero A.
        Quality of life in patients with respiratory allergy is influenced by the causative allergen.
        J Invest Allergol Clin Immunol. 2013; 23: 309-314
        • Valero A.
        • Ferrer M.
        • Baro E.
        • et al.
        Discrimination between moderate and severe disease may be used in patients with either treated or untreated allergic rhinitis.
        Allergy. 2010; 65: 1609-1613