Psychiatry Research 237 (2016) 196200

Contents lists available at ScienceDirect

Psychiatry Research
journal homepage: www.elsevier.com/locate/psychres

Validation of the French version of the Acceptability E-scale (AES) for

mental E-health systems
Jean-Arthur Micoulaud-Franchi a,b, Alain Sauteraud a,b, Jrme Olive b, Patricia Sagaspe a,b,
Stphanie Bioulac a,b, Pierre Philip a,b,n
a
Clinique du Sommeil, Service d'Explorations Fonctionnelles du Systme Nerveux, CHU de Bordeaux, Place Amlie Raba-Lon, 33076 Bordeaux, France
b
USR CNRS 3413 SANPSY, Universit de Bordeaux, 33076 Bordeaux, France

art ic l e i nf o a b s t r a c t

Article history: Despite the increasing use of E-health systems for mental-health organizations, there is a lack of psy-
Received 9 November 2015 chometric tools to evaluate their acceptability by patients with mental disorders. Thus, this study aimed
Received in revised form to translate and validate a French version of the Acceptability E-scale (AES), a 6-item self-reported
9 January 2016
questionnaire that evaluates the extent to which patients nd E-health systems acceptable. A forward
Accepted 16 January 2016
Available online 19 January 2016
backward translation of the AES was performed. The psychometric properties of the French AES version,
with construct validity, internal structural validity and external validity (Pearson's coefcient between
Keywords: AES scores and depression symptoms on the Beck Depression Inventory II) were analyzed. In a sample of
Satisfaction 178 patients (mean age 46.51 years, SD 12.91 years), the validation process revealed satisfactory
E-health
psychometric properties: factor analysis revealed two factors: Satisfaction (3 items) and Usability (3
Questionnaire
items) and Cronbach's alpha was 0.7. No signicant relation was found between AES scores and de-
Validity
Psychometrics pression symptoms. The French version of the AES revealed a two-factor scale that differs from the
original version. In line with the importance of acceptability in mental health and with a view to E-health
systems for patients with mental disorders, the use of the AES in psychiatry may provide important
information on acceptability (i.e., satisfaction and usability).
& 2016 Elsevier Ireland Ltd. All rights reserved.

1. Introduction decisions for diagnosis, prognosis and therapeutic follow-up

Patient-reported outcomes (PROs), including measures of sa-
tisfaction, have gained increasing attention in the mental-health
services (Boyer et al., 2009; Crow et al., 2002; Fitzpatrick, 1991).
Indeed, patient satisfaction has become one of the important PROs
and a contributing outcome in the assessment of health care
quality (Zendjidjian et al., 2014b). Patient satisfaction with car-
egiving is a strong predictor of future behaviors, in particular
treatment adherence (Ware and Davies, 1983; Zendjidjian et al.,
2014a; 2014b), and provides important information for improving
the quality of health care organizations (Cleary and McNeil, 1988).
Given the increased prevalence of chronic disease, in particular
of mental and brain disorders (Wykes et al., 2015), health care
organizations now have to deal with the increased use of E-health
systems (Chouvarda et al., 2015). Such systems facilitate: i) com-
munication and coordination between patient and healthcare
professionals (in primary care and hospital), and ii) medical
n
Corresponding author at: Clinique du Sommeil, Service d'Explorations Fonc-
tionnelles du Systme Nerveux, CHU de Bordeaux, Place Amlie Raba-Lon, 33076
Bordeaux, France.
E-mail address: pr.philip@free.fr (P. Philip).
(Chouvarda et al., 2015). While the extent to which patients are
satised with E-health systems and nd them acceptable should
be evaluated, there is a lack of psychometric tools to evaluate it.
The Acceptability E-scale (AES) presents some advantages in
the eld of patient-reported outcomes in E-health systems (Tari-
man et al., 2011). It is a generic and validated questionnaire that
can accurately evaluate satisfaction with a broad range of E-health
systems (Carlson et al., 2001; Mullen et al., 2004; Taenzer et al.,
2000), a point that differentiates it from other questionnaires that
may have no clear validation process and which focus only on
satisfaction with one precise E-health solution. It is also based on
6 items (with a 5-point Likert scale for each item), which is time-
efcient and increases its usability in clinical practice. Further-
more, it is a self-reported questionnaire, which is recognized as
the most reliable method to measure satisfaction (Crow et al.,
2002).
Despite French being the 6th most widely spoken world lan-
guage with 220 million speakers (Organization International de la
francophonie, 2009), the AES has not previously been translated
and validated in French. Thus, in order to better evaluate sa-
tisfaction with E-health solution in native French speakers, this
study sought to design and validate a French version. Translating

http://dx.doi.org/10.1016/j.psychres.2016.01.043
0165-1781/& 2016 Elsevier Ireland Ltd. All rights reserved.
 J.-A. Micoulaud-Franchi et al. / Psychiatry Research 237 (2016) 196200 197

questionnaires may be dependent on cultural background (Brislin,
1970) so before using any translated questionnaire, a transcultural
validation should be undertaken according to specic rules and
methods. In the present study we analyzed the psychometric
properties of the French AES version in a sample of French patients
were notied to the French psychiatrist (AS) for a second round of translation and
back-translation (KR). After this second round, the back-translated version and the
original version was declared equivalent. The translated version of the AES was
administered to 7 patients and showed good clarity and cultural acceptability. No
further adaptations were required. The nal version of the French AES is shown in
Table 1.

that was interviewed with a virtual agent, which are the most
innovative E-health solution to sustain a clinical evaluation (Philip
et al., 2014). In this study, the virtual agents made a clinical eva-
luation of depressive symptoms.
2. Methods and materials
2.1. Participants
Outpatients were recruited by psychiatrists in Bordeaux University Hospital
from November 2014 to June 2015 in a consecutive sample design. Inclusion criteria
were: native French-speaking adult patients ( 418 years) with any type of psy-
chiatric or sleep complaints. Exclusion criteria were: insufcient capacity to con-
sent to and understand and answer the self-report questionnaires.
Gender, age, year of education and antidepressant treatment were noted. Pa-
tients were given a clinical interview with a virtual agent to evaluate Major De-
pressive Disorder (MDD) according to DSM-5 criteria, which was the E-health so-
lution that they had to evaluate with the AES. This virtual agent was adapted from
previously developed software able to self-conduct interactive face-to-face clinical
interviews (Philip et al., 2014). This software was based on four modules (Philip
et al., 2014): i) an interview manager module that conducts the whole interview
and manage the other modules, ii) a 3D rendering module that display the virtual
agent and play animations, iii) a speech synthesizer module that create speech of
the virtual agent and iv) a speech recognizer module that recognize the responses
of the patients. The software suite was installed on a computer (Windows 8-i7
3770@3.4 GHz8 GB-NVidia 670 GTX) connected to a 40-in. display and to a Mi-
crosoft Kinect sensor. In the present study the virtual agent was able to self-con-
duct interactive face-to-face clinical interviews to investigate each MDD DSM-5
criteria (American Psychiatric Association, 2013).
Patients were invited to participate in the study during their routine clinical
evaluation. After receiving a detailed description of the study, participants gave
their informed consent. The study was conducted in accordance with the De-
claration of Helsinki and French Good Clinical Practices. The study was classied as
a clinical trial by the US National Institutes of Health (ClinicalTrials.gov identier:
NCT02544295). The present article of validation of the AES is part of a more general
study on the validation of based-virtual reality diagnosis for neuropsychiatric dis-
orders and sleep/wake disorders (PHENOVIRTPSY).
2.2. Procedure
2.2.1. Translation of the AES
A forwardbackward translation was performed through a multistep method
(Brislin, 1970; Epstein et al., 2015). The original version was translated into French
by a French native psychiatrist (AS). The back-translation into English was under-
taken by a professional native English translator (KR) blind to the original version.
This back-translated version was submitted to a second professional native English
translator (HK) who noted the differences with the original version. The differences
2.2.2. Assessments
2.2.2.1. Self-rated assessment. The Acceptability E-scale (AES) (Tariman et al., 2011)
and the Beck Depression Inventory II (BDI-II) (Beck et al., 1996) were completed as
part of the self-rated psychiatric assessment. The AES consists of 6 items rated by
the patients on a balanced ve-point Likert scale ranging from 1 to 5. The rating
was determined according to patients' experience concerning the evaluation with
the virtual agent. The score was obtained by computing the sum of the scores
obtained by items associated with it, from 1 to 5. The scale ranges from 6 to 30. The
BDI-II consists of 21 items rated by the patients according to their experience in the
preceding month. The BDI-II has previously been translated and validated in French
(Beck, 1996; Bourque and Beaudette, 1982). The scale ranges from 0 to 63.
2.2.2.2. Psychiatric assessment. Patients were medically examined by an experi-
enced psychiatrist, who judged whether MDD was present according to DSM-5
criteria. However, no standardized interview was applied. The consulting psy-
chiatrist judged whether MDD was present and treatment was required. Therefore,
the diagnostic standard for the present study was expert opinion. Only MDD was
evaluated in the present study. The psychiatrists were blind to the results of the
virtual agent.
2.3. Statistical analyses and hypotheses
Descriptive statistics of the obtained data included frequencies and percentages
of categorical variables together with means and standard deviations of continuous
variables. For the validation process, we analyzed the psychometric properties of
the French AES version including construct validity, internal structural validity and
external validity. Data analysis was performed using SPSS software (Version 18 for
Mac, PASW Statistics), LISREL software (Scientic Software International, Inc.) and
WINSTEP Software (Wright et al., 2001). For all the tests, the accepted signicance
level was 5%.
2.3.1. Construct validity
A conrmatory factor analysis (CFA) was performed using the LISREL model to
analyze the construct validity and to test the one factor structure of the original
scale (Tariman et al., 2011). If the Root Mean Square Error of Approximation
(RMSEA) was not satisfactory (4 0.05), a principal component factor analysis (PCA)
with Varimax rotation was performed to explore the structure of the French ver-
sion of the AES (Kaiser, 1958). This analysis determined the nal structure and the
number of independent factor of the French version of the AES. Items were in-
cluded in a factor if they revealed a loading greater than 0.4. The undimensionality
of each factor was assessed using Rasch analyses by computing the pattern of item
goodness-of-t statistics (INFIT) for each factor (Wright et al., 2001). A value of
INFIT between 0.5 and 1.5 ensures that all items of the scale tend to measure the
same concept (Wright et al., 2001).
2.3.2. Internal structural validity
Item-internal consistency (IIC) was assessed by correlating each item with its
related factor; correlations of at least 0.4 are recommended for supporting IIC

Table 1
English version/French version and factor analysis of the French version of the AES.

No. Factor Items French version Mean SDa Floor (%) Ceiling (%) Alphab INFITc

1 Usability How easy was this computer program for quel point avez-vous trouv ce programme in- 4.62 1.06 6.2 85.4 0.762 1.64
you to use? formatique facile d'utilization?
2 Usability How understandable were the questions? quel point les questions taient-elles 4.77 0.52 15.7 80.9 0.713 0.95
comprhensibles?
3 Satisfaction How much did you enjoy using this com- quel point avez-vous apprci l'utilization de ce 4.07 0.91 34.8 38.8 0.595 0.47
puter program? programme informatique?
4 Satisfaction How helpful was this computer program in A quel point ce programme informatique vous a-t-il 3.7 1.07 31.5 27.5 0.626 0.64
describing your symptoms and quality of t utile pour dcrire vos symptmes et votre qua-
life? lit de vie?
5 Usability Was the amount of time it took to com- Le temps consacr rpondre ce programme in- 4.57 0.89 14 75.3 0.644 0.80
plete this computer program acceptable? formatique tait-il acceptable?
6 Satisfaction How would you rate your overall satisfac- Comment valuer-vous votre satisfaction gnrale 4.07 0.93 38.8 38.2 0.577 0.69
tion with this computer program? de cet outil informatique?

a
Standard deviation.
b
Cronbach's alpha if item is deleted.
c
Rasch statistics.
198 J.-A. Micoulaud-Franchi et al. / Psychiatry Research 237 (2016) 196200
(Carey and Seibert, 1993). Item discriminant validity (IDV) was assessed by de-
termining whether items correlated better with the factor they were hypothesized
to represent compared with the other factor (Campbell and Fiske, 1959).
Internal consistency reliability was assessed by Cronbach's alpha coefcient
and was recalculated after items were removed. To conrm consistency, a coef-
cient of at least 0.7 was expected (Cronbach, 1951). It is recommended that the
delete of any of the 6 items did not increase the Cronbach's alpha coefcient.
Floor and ceiling effects were reported to assess the distribution of responses.
The rate of oor and ceiling effects was calculated as the proportion of individuals
who obtained the lowest (1) and the highest (5) scores for any of the items.
2.3.3. External validity
To explore external validity, the relation between BDI-II and AES scores was
investigated by computing Pearson's coefcients. No relations between symptoms
of depression and the AES score were hypothesized. Difference in AES scores be-
tween men and women was also investigated by t-test. No signicant difference
between men and women was hypothesized.
3. Results
3.1. Sample characteristics
A total of 178 native French speakers were included. None of
the patients reported any difculties in understanding the items of
the AES.
The mean age was 46.51 years old (SD 12.91, range: [1964]
years old), 57.3% (102) were women, the mean number of years of
education was 13.36 (SD 3.07), 14.6% (26) were being treated
with antidepressant treatment, and 19.7% (35) had a diagnosis of
current MDD according to the psychiatrist's evaluation. Age was
not signicantly different between patients with and without
MDD (p 0.30). However, there were signicantly more women
with MDD than men with MDD (p 0.002). The mean BDI-II score
was 10.73 (SD 12.52) and the mean AES score was 25.81
(SD 3.48).
3.2. Validity
3.2.1. Construct validity
The conrmatory factor analysis using LISREL model revealed
that the RMSEA was unsatisfactory (RMSEA 0.089). The ex-
ploratory factor analysis using principal component factoring re-
vealed two factors: the rst with items 3, 4, and 6, and the second
with items 1, 2, and 5. For each factor the value of item loads was
greater than 0.4. The overall pattern of item goodness-of-t sta-
tistics (INFIT) for each factor was satisfactory, except for item 1
(Table 1).
3.2.2. Internal structural validity
The item-internal consistency (IIC) was satisfactory for the two
factors: each item achieved the 0.40 standard threshold value. The
item discriminant validity (IDV) was satisfactory as the correlation
between items with their contributive factor was higher than
items with the other factor. Cronbach's alpha coefcient was 0.70
and ranged from 0.58 to 0.76 after items were deleted. The overall
Table 2
Validation of the French version of the AES.
Factor (number of items) Mean (SD)a IICb minmax IDVc minmax
Factor 1: satisfaction (3) 11.84 (2.53) 0.860.88 0.210.32
Factor 2: usability (3) 13.96 (1.72) 0.520.75 0.090.42
Global score 25.81 (3.48)
a
Standard deviation.
b
Item internal consistency (item-to-own dimension correlations).
c
Item discriminant validity (item-to-other dimensions correlations).
d
Cronbach's alpha.
e
Rasch statistics.
internal consistency reliability was satisfactory, except for item 1
(as the deletion of this item increase the Cronbach's alpha coef-
cient). Floor effects ranged from 27.5% to 85.4% and ceiling effects
ranged from 0% to 6.7% (Table 2).
3.2.3. External validity
The correlation between AES and BDI-II scores was not sig-
nicant (r (178) 0.12, p 0.11). The mean AES score in women
was 25.75 (SD 3.31) and in men was 25.88 (SD 3.71). No sta-
tistical association was related to gender concerning the AES
scores (p 0.81).
4. Discussion
The aim of this study was to translate and validate the French
version of the AES in order to make it available for evaluating
satisfaction with E-health systems in French-speaking patients.
Transcultural validation supported the structural validity of the
instrument but revealed some differences with the original ver-
sion (Tariman et al., 2011).
In the original version, principal component factor analysis
found only one factor with all 6 items with a value of item load
equal to or greater than 0.4 (Tariman et al., 2011). With the French
version, two factors were found: the rst factor is related to the
question that evaluates the enjoyability, the helpfulness to de-
scribe his/her own feeling and the satisfaction of the patient with
the E-health system, while the second factor is related to the
question that evaluates the usability, the understandability and the
acceptability of E-health systems. These two factors can be thus
labeled according to their constitutive items as Satisfaction for
factor 1 (3 items: 3, 4 and 6) and Usability for factor 2 (3 items: 1,
2 and 5). Interestingly, these two factors refer to the classical
Technology Acceptance Model (TAM). The TAM is an information
systems theory that models the acceptability for a new technology
(Davis et al., 1989). The TAM posits that acceptability is concerned
with the user's subjective evaluation of technological tools in-
cluding two factors: perceived usefulness (related to satisfaction)
and perceived ease of use (related to usability) (Davis et al.,
1989).
The difference between the construct of the original version
and that of the French version may be due to differences with the
E-health systems used to validate the AES. In the original version,
the AES was used to evaluate a web-based questionnaire (Tariman
et al., 2011). In the French version, it was used to evaluate a virtual
agent (Philip et al., 2014). Two important kinds of parameters are
controlled to build a virtual agent: a) the quality of each question
and more generally of the understandability of the verbal con-
tents; and b) the quality of the interaction involving non-verbal
parameters. The latter are not involved in classical web-based
questionnaires and are specic to virtual agents. Thus, the use of a
virtual agent to evaluate the AES probably favored the responses of
participants in two directions: the understandability of each
Floor (%) Ceiling (%) Alphad INFITeminmax
20.8 1.1 0.835 0.470.69
61.2 1.1 0.643 0.81.64
13.5 1.1
 J.-A. Micoulaud-Franchi et al. / Psychiatry Research 237 (2016) 196200
question asked by the virtual agent, and the satisfaction with the
clinical interaction of the interview conducted by the virtual agent.
Moreover, the understandability of the virtual agent used in this
study was very high (with a proportion of individuals who ob-
tained the highest scores for the items of factor usability higher
than 75%) and may explain the poor internal consistency of these
items. Further studies are needed to evaluate the specicity of the
satisfaction for a broad range of E-health systems and in particular
for the use of a virtual agent. In line with the Proximal Similarity
Model (Campbell, 1986), the difference between the construct of
the original version and that of the French version may also be due
to differences in the medical conditions of patients who were
evaluated with the AES and the difference in the symptoms eval-
uated by the E-health tool. In the original version, patients were
suffering from cancer and quality of life was evaluated with the
E-health tool (Tariman et al., 2011). In the French version, patients
were consulting for psychiatric or sleep problems, and depressive
symptoms were evaluated with the E-health tool. Further studies
are needed to evaluate the specicity of the satisfaction with
E-health systems in different medical conditions and different
kind of symptoms.
Some limitations in the current study have to be considered.
Firstly, the diagnostic standard for MDD in the present study was
not a structured clinical interview. Instead, MDD diagnosis was
based on expert opinion. This could prove to be a disadvantage
considering the accuracy of the assessment. Secondly, the present
study with virtual agent was focused on MDD and the psychiatrist
evaluated only MDD criteria. Around 20% of the patients tested in
this sample had a diagnosis of MDD, which is also within the range
previously described in France (Bromet et al., 2011). Interestingly,
their symptoms of depression were not related to their AES score.
These results suggest that the AES can be used with a large range
of patients, including those with a mental disorder. However, the
generalizability of the AES French version could be limited. Indeed,
the validity of the AES in other mental disorders could not be
assured. Thus, the psychometric validities of the AES in patients
with MDD and with other mental disorders should be evaluated
specically in further studies. This is essential in view of the im-
portance of satisfaction in mental health (Boyer et al., 2009) and
the perspective of using E-health systems with patients with
mental disorders (Kimhy et al., 2012; Serre et al., 2015). Thirdly, in
the context of E-health services, there was no gold-standard
questionnaire to examine the external validity of the AES. How-
ever, there are some generic and validated questionnaires, like the
Client Satisfaction Questionnaire CSQ-8, which is a self-report
questionnaire constructed to measure satisfaction with health,
mental health, and human services received by individuals (Att-
kisson and Zwick, 1982). The external validity of the AES with the
CSQ-8 should be investigated in further studies. Fourthly, the use
of PCA to explore the structure of the AES can be criticized. A CFA
is also a relevant statistical method when a theoretical model of
relationships is available. The choice between PCA and CFA can
thus be debatable. In this work, as the TAM model was not the
explicitly the theoretical model used by Tariman et al. (2011), we
considered that our knowledge was insufcient to provide a robust
underlying structure. Moreover, an exploratory approach was
justied to propose an adaptive structure in the French context.
Finally, the internal consistency of factor 2 was limited. Indeed,
item 1, which explores how easy was the use of the computer
program, showed ceiling effect, an increase of the internal con-
sistency when deleted, and an INFIT statistic above the satisfactory
range. Thus, the internal consistency of factor 2 should be in-
vestigated in further studies.
In conclusion, the French version of the AES is a psychome-
trically acceptable self-reported questionnaire for evaluating sa-
tisfaction with and the usability of E-health systems. The French
199
version of the AES revealed a two-factor scale that differs from the
original version. Its inclusion in research on E-health systems
would provide critical information regarding patient acceptability
(i.e., satisfaction and usability) of these new ways of giving and
organizing care for chronic disease. Since use of the French lan-
guage is increasing (Organization International de la francophonie,
2009) and given the increasing prevalence of mental and brain
disorders in developed countries (Wykes et al., 2015), the French
version of the AES may prove to be of particular interest in this
future.
Conict of interest
We report no conicts of interest.
Acknowledgments
Special thanks to Muriel Bacarisse, Cdric Valtat, and Aurlien
Boiseau for selecting participants, collecting and monitoring data
and for administrative, technical, and logistic support. We ac-
knowledge the ANR-PHENOVIRT.
References
American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental
Disorder, 5th ed. American Psychiatric Association, Washington, DC.
Attkisson, C.C., Zwick, R., 1982. The client satisfaction questionnaire. Psychometric
properties and correlations with service utilization and psychotherapy out-
come. Eval. Program. Plan. 5, 233237.
Beck, A.T., Steer, R.A., Ball, R., Ranieri, W., 1996. Comparison of beck depression
inventories -IA and -II in psychiatric outpatients. J. Personal. Assess. 67,
588597.
Beck A.T., 1996. Inventaire de Dpression de Beck (2me dition): Manuel du BDI-II.
les d. du Centre de psychologie applique (Ed.), Paris.
Bourque, P., Beaudette, D., 1982. Etude psychomtrique du questionnaire de d-
pression de Beck auprs d'un chantillon d'tudiants universitaires franco-
phones. Revue can. des sci. du comportement 14, 211218.
Boyer, L., Baumstarck-Barrau, K., Cano, N., Zendjidjian, X., Belzeaux, R., Limousin, S.,
Magalon, D., Samuelian, J.C., Lancon, C., Auquier, P., 2009. Assessment of psy-
chiatric inpatient satisfaction: a systematic review of self-reported instruments.
Eur. Psychiatry. 24, 540549.
Brislin, R., 1970. Back-translation for cross-cultural research. J. Cross Cult. Psychol. 1,
185216.
Bromet, E., Andrade, L.H., Hwang, I., Sampson, N.A., Alonso, J., de Girolamo, G., de
Graaf, R., Demyttenaere, K., Hu, C., Iwata, N., Karam, A.N., Kaur, J., Kos-
tyuchenko, S., Lepine, J.P., Levinson, D., Matschinger, H., Mora, M.E., Browne, M.
O., Posada-Villa, J., Viana, M.C., Williams, D.R., Kessler, R.C., 2011. Cross-national
epidemiology of DSM-IV major depressive episode. BMC Med. 9, 90.
Campbell, D., 1986. Relabeling internal and external validity for applied social sci-
entists. New Dir. Program Eval. 31, 6777.
Campbell, D.T., Fiske, D.W., 1959. Convergent and discriminant validation by the
multitrait-multimethod matrix. Psychol. Bull. 56, 81105.
Carey, R.G., Seibert, J.H., 1993. A patient survey system to measure quality im-
provement: questionnaire reliability and validity. Med. Care 31, 834845.
Carlson, L.E., Speca, M., Hagen, N., Taenzer, P., 2001. Computerized quality-of-life
screening in a cancer pain clinic. J. Palliat. Care 17, 4652.
Chouvarda, I.G., Goulis, D.G., Lambrinoudaki, I., Maglaveras, N., 2015. Connected
health and integrated care: Toward new models for chronic disease manage-
ment. Maturitas 82, 2227.
Cleary, P.D., McNeil, B.J., 1988. Patient satisfaction as an indicator of quality care.
Inquiry 25, 2536.
Cronbach, L.J., 1951. Coefcient alpha and the internal consistency of tests. Psy-
chometrika 16, 297334.
Crow, R., Gage, H., Hampson, S., Hart, J., Kimber, A., Storey, L., Thomas, H., 2002. The
measurement of satisfaction with healthcare: implications for practice from a
systematic review of the literature. Health Technol. Assess. 6, 1244.
Davis, F., Bagozzi, R., Warshaw, P., 1989. User acceptance of computer technology: a
comparison of two theoretical models. Manag. Sci. 35, 9821003.
Epstein, J., Santo, R.M., Guillemin, F., 2015. A review of guidelines for cross-cultural
adaptation of questionnaires could not bring out a consensus. J. Clin. Epidemiol.
68, 435441.
Fitzpatrick, R., 1991. Surveys of patients satisfaction: I Important general con-
siderations. BMJ 302, 887889.
Kaiser, H., 1958. The varimax criterion for analytic rotation in factor analysis.
200 J.-A. Micoulaud-Franchi et al. / Psychiatry Research 237 (2016) 196200
Psychometrika 23, 187200.
Kimhy, D., Myin-Germeys, I., Palmier-Claus, J., Swendsen, J., 2012. Mobile assess-
ment guide for research in schizophrenia and severe mental disorders. Schi-
zophr. Bull. 38, 386395.
Mullen, K.H., Berry, D.L., Zierler, B.K., 2004. Computerized symptom and quality-of-
life assessment for patients with cancer part II: acceptability and usability.
Oncol. Nurs. Forum 31, E84E89.
Organisation International de la francophonie, 2009. Le franais, enjeux du XXIe
sicle.
Philip, P., Bioulac, S., Sauteraud, A., Chaufton, C., Olive, J., 2014. Could a virtual
human be used to explore excessive daytime sleepiness in patients? Presence:
Teleoperators Virtual Environ. 23, 369376.
Serre, F., Fatseas, M., Swendsen, J., Auriacombe, M., 2015. Ecological momentary
assessment in the investigation of craving and substance use in daily life: a
systematic review. Drug Alcohol Depend 148, 120.
Taenzer, P., Bultz, B.D., Carlson, L.E., Speca, M., DeGagne, T., Olson, K., Doll, R.,
Rosberger, Z., 2000. Impact of computerized quality of life screening on phy-
sician behaviour and patient satisfaction in lung cancer outpatients. Psy-
chooncology 9, 203213.
Tariman, J.D., Berry, D.L., Halpenny, B., Wolpin, S., Schepp, K., 2011. Validation and
testing of the Acceptability E-scale for web-based patient-reported outcomes in
cancer care. Appl. Nurs. Res. 24, 5358.
Ware Jr., J.E., Davies, A.R., 1983. Behavioral consequences of consumer dissatisfac-
tion with medical care. Eval. Program Plan. 6, 291297.
Wright, B., Linacre, J., 2001. Winsteps: A Rasch Model Computer Program. MESA
Press, Chicago.
Wykes, T., Haro, J.M., Belli, S.R., Obradors-Tarrago, C., Arango, C., Ayuso-Mateos, J.L.,
Bitter, I., Brunn, M., Chevreul, K., Demotes-Mainard, J., Elfeddali, I., Evans-Lacko,
S., Fiorillo, A., Forsman, A.K., Hazo, J.B., Kuepper, R., Knappe, S., Leboyer, M.,
Lewis, S.W., Linszen, D., Luciano, M., Maj, M., McDaid, D., Miret, M., Papp, S.,
Park, A.L., Schumann, G., Thornicroft, G., van der Feltz-Cornelis, C., van Os, J.,
Wahlbeck, K., Walker-Tilley, T., Wittchen, H.U., 2015. Mental health research
priorities for Europe. Lancet Psychiatry.
Zendjidjian, X.Y., Baumstarck, K., Auquier, P., Loundou, A., Lancon, C., Boyer, L.,
2014b. Satisfaction of hospitalized psychiatry patients: why should clinicians
care? Patient Preference Adherence 8, 575583.
Zendjidjian, X.Y., Auquier, P., Lancon, C., Loundou, A., Parola, N., Faugere, M., Boyer,
L., 2014a. Determinants of patient satisfaction with hospital health care in
psychiatry: results based on the SATISPSY-22 questionnaire. Patient Preference
Adherence 8, 14571464.