2022 11th International Conference on Modern Circuits and Systems Technologies (MOCAST)

Autism detection in High-Functioning Adults with

the application of Eye-Tracking technology and
Machine Learning
2022 11th International Conference on Modern Circuits and Systems Technologies (MOCAST) | 978-1-6654-6717-9/22/$31.00 ©2022 IEEE | DOI: 10.1109/MOCAST54814.2022.9837653

Konstantinos-Filippos Kollias Christine K. Syriopoulou-Delli Panagiotis Sarigiannidis

Laboratory of Robotics, Embedded and Department of Educational and Social Department of Electrical and Computer
Integrated Systems Policy, University of Macedonia Engineering, University of Western
Department of Electrical and Computer Thessaloniki, Greece Macedonia
Engineering, University of Western [email protected] Kozani, Greece
Macedonia [email protected]
Kozani, Greece
[email protected]

George F. Fragulis
Laboratory of Robotics, Embedded and
Integrated Systems
Department of Electrical and Computer
Engineering, University of Western
Macedonia
Kozani, Greece
[email protected]

Abstract— High-Functioning Autism Detection in Adults is High-functioning ASD individuals face significant
significantly difficult compared with early Autism Spectrum difficulties to get autism diagnosis in adulthood as their
Disorder (ASD) diagnosis with severe symptoms. ASD diagnosis symptoms are not that evident compared to early ASD
is usually achieved by behavioural instruments relying on diagnosis when symptoms are severe [6], [7]. More
subjective rather on objective criteria, whereas advances in
specifically, ASD individuals develop specific strategies
research indicate cutting - edge methods for early assessment,
such as eye-tracking technology, machine learning, Internet of throughout their life, which are likely to mask otherwise
Things (IoT), and other assessment tools. This study suggests the apparent relevant symptoms. Thus, the development of a
detection of ASD in high-functioning adults with the screening method for high-functioning ASD identification
contribution of Transfer Learning. Decision Τrees, Logistic which does not depend on subjective measures is of essential
Regression and Transfer Learning were applied on a dataset importance.
consisting of high-functioning ASD adults and controls, who ASD is defined as a neurodevelopmental disorder
looked for information within web pages. A high classification prevalent in 1% of the world’s population [8] and
accuracy was achieved regarding a Browse (80.50%) and a characterised by social communication/interaction
Search (81%) task showing that our method could be considered
difficulties and repetitive behaviours/interests [9].
a promising tool regarding automatic ASD detection.
Limitations and suggestions for future research are also Apart from reduced social interaction and
included. communication, restricted, repetitive, and stereotyped
behaviour, people with ASD have a tendency to show a
Keywords—High-Functioning Autism detection, eye-tracking, deficit in eye gaze. This characteristic cannot cause autism
machine learning, transfer learning, IoT, web. [9] but constitutes an important item in several diagnostic
tests [3]. Eye gaze deficits of people with ASD are associated
I. INTRODUCTION
both with social [10] and non-social stimuli [11].
Early Autism Spectrum Disorder (ASD) assessment and Unlike conventional assessment approaches to ASD
intervention are thought to have major long-term benefits for research, eye-tracking technology is considered more
ASD children and adults as well as their families. However, beneficial, as it can contribute to early, objective and reliable
the diagnostic process of neurodevelopmental disorders and detection of autism and features of it [10], [12], [13]. Eye-
more specifically ASD assessment is challenging for tracking ASD studies have increased steeply during the last
professionals, as there are no well-established decade, either due to easier access to eye-tracking technology
biophysiological diagnostic tests [1], [2]. ASD diagnosis is [14], or because a great variety of special devices and
generally based on behavioural assessment, utilising software playing an important role in easier and less
standardised tools characterised by high validity and expensive eye-tracking data recording have been developed
reliability, such as the Autism Diagnostic Observation [12].
Schedule (ADOS) [3] and the Autism Diagnostic Interview- Eye-tracking instruments are frequently combined with
Revised (ADI-R) [4]. Nevertheless, their time-consuming, contemporary artificial intelligence techniques, such as
costly and difficult administration combined with the need of Machine Learning (ML), a data driven technique based on
experienced and trained interviewers can often result in a advanced learning of mathematics, theories of information
delayed diagnosis and consequently a retard regarding the and statistical estimation [15]. ML can be applied in autism
onset of early intervention [1], [2], [5]. research by offering a more objective and reproducible
second opinion [16], i.e., diagnosis enhancement [15], early

978-1-6654-6717-9/22/$31.00 © 2022 IEEE

Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on June 25,2024 at 07:20:29 UTC from IEEE Xplore. Restrictions apply.
autism screening [17], as well as brain activity [18] and from videos and divided the features into angle and length.
different behaviours observation [19]. Furthermore, ML can They also made accumulative and non-accumulative
be a valid biomarker-based technique contributing to histograms for single and combined features and fed them to
objective ASD diagnosis [20]–[22]. six three-layer Long Short-Term Memory (LSTM) networks
Machine Learning has been applied in Internet of Things for classification. The authors computed Kernel Principal
(IoT) systems as well, aiming at ASD assessment [23], [24] Component Analysis (KPCA) to reduce the data and fed six
and early detection [25]. More specifically, an ML method SVMs. LSTM networks accuracy was 6.2% higher than the
was proposed by [25] with an aim to facilitate doctors detect SVM one. The best ASD classification performance was
the type of ASD in children from a preliminary symptom. IoT attained when LSTM was combined with accumulative
was utilised for symptoms collection attaining a high histograms with an accuracy of 92.60%.
accuracy of 83%.
Traditional ML algorithms, such as Random Forests, II. MATERIALS AND METHODS
Logistic Regression and Decision Trees, can be improved by A. Dataset
Transfer Learning (TL) which transfers the knowledge Eye-tracking data were collected by [27] and comprised
acquired in a single or more tasks and utilises it for improving high-functioning ASD adults and controls, who looked for
learning in a related target task [26]. information within web pages. The final data included for
A. Our contribution analysis were attained from 15 high-functioning ASD adults
(9 male and 6 female) and 15 controls (8 male and 7 female).
This study aims at high-functioning ASD classification
The participants were involved in two web-browsing tasks
based on eye-tracking data that were acquired from web-
looking for information on some web pages: Browse and
related tasks. Decision Trees, Logistic Regression and Search tasks.
Transfer-learning are applied. Eye-tracking data were On the browsing task, the participants were free to spend
collected by [27] and consisted of high-functioning ASD
two minutes maximum on each web page while looking for
adults and controls, who looked for information within web
any kind of information that could be interesting to them.
pages. The significance of this study lies on the fact that we
During the search task, the participants had 30 seconds to find
propose a transfer-learning-based ASD classification certain information on each web page which they would
framework which can achieve high accuracy. This is a less exploit to answer two questions verbally asked by the
explored area of research in which a screening method not
researcher.
depending on subjective measures is of essential importance.
A 60Hz Gazepoint GP3 video-based eye tracker was
The utilisation of eye-tracking data obtained from web-
employed with its accuracy ranging from 0.5 to 1 degree of
related tasks for the classification of ASD and non-ASD
visual angle. Gaze features such as Time to First View, Time
people is also prominent. In other words, the acquisition of Viewed (sec), Time Viewed %, Fixations and Revisits as well
such kind of data is less-expensive, easier, and less obtrusive as non-gaze features such as Media ID, Area of Interest (AOI)
than conventional ASD assessment methods such as ADOS,
ID, Correct Answer ID, Participant Gender, and Level of
ADI-R and Functional Magnetic Resonance Imaging (fMRI).
Visual Complexity were applied to train the classifiers. There
Therefore, the current study applies Transfer Learning on
was a definition of Page-specific and Generic AOIs, as well.
eye-tracking data collected by [27] devising the hypothesis
A Logistic Regression algorithm was implemented. A best
that a better classification accuracy can be achieved. classification performance of 0.75 was achieved in Search
B. Background task and of 0.71 in Browse task when training on selected
There have been several ASD classification eye-tracking media took place. The results obtained showed that Generic
studies in the past which used data collected in previous AOIs were more suitable for tasks such as the Browse task,
studies and tried to improve their classification performance. whereas page specific AOIs were required for tasks like the
For example, [28] used the eye-tracking data of high- Search task. Some other non-gaze features did not play a
functioning ASD and Typically Developing (TD) significant role in the classification performance. For more
participants who passively watched natural scene images in information about the dataset, the tasks, eye-tracking and
[29]. Specifically, there was a feature selection which was analysis see [27]. In a more recent study by the same authors
based on the Fischer score method [30] in [28]. This feature [12], containing similar tasks to the earlier one, the data
selection aimed at finding the images that could best collected in [27] were utilised in addition to some more data.
contribute to ASD classification. Deep Neural Networks B. Classifying Autism based on Transfer-Learning
(DNNs) were used to automatically acquire image features In this study, Transfer-learning and various classification
from natural scenes and consequently these features were fed
algorithms were tested in Matlab to improve the accuracy
to a linear Support Vector Machine (SVM) that classified
achieved in [27] by a Logistic Regression algorithm.
ASD and TD participants providing an increased
Decision Trees and Logistic Regression attained the highest
classification accuracy (92%). In another study [31], the
accuracy. Both Browse and Search tasks data were used for
authors employed a dataset from a prior study in which ASD classification. Gaze as well as non-gaze features were utilised
and TD children were videotaped while watching their in the training of classifiers.
mothers’ pictures presented on a computer screen [32]. They
added one more dataset of ASD children to balance the III. RESULTS AND DISCUSSION
amount of ASD and TD participants which increased the
This study aimed at high-functioning ASD classification
validity of the ML model. Additionally, they employed the
employing eye-tracking data acquired from web-related tasks
tracking learning detection algorithm for feature extraction
[27]. Transfer-learning was applied, and various

Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on June 25,2024 at 07:20:29 UTC from IEEE Xplore. Restrictions apply.
classification algorithms were tested in Matlab to improve the addressed in future research. For example, Neural Networks,
accuracy achieved by the Logistic Regression algorithm Principal Component Analysis (PCA) and possibly Random
applied in the prior study [27]. Decision Trees and Logistic Forests (RF) can be implemented in a future study improving
Regression achieved the highest accuracy concerning Browse classification accuracy. Another important issue concerns the
and Search task, respectively. According to our results, the dataset size, i.e., by employing some additional participants
classification accuracy of both Browse and Search tasks was the model’s validity could be increased. Additionally, only
higher compared to the results of the prior study [27]. two web-browsing tasks were analysed in this study, i.e.,
Τhe highest classification accuracy for the Browse task Browse and Search tasks. Future studies could use datasets
(80.50%) was achieved when all Gaze features, AOI ID, from other studies comprising more tasks for instance [12]
Media ID, Participant Age and Participant Gender were used which contained three web-browsing tasks. Finally, future
and Decision trees were applied. As it is depicted in Table I, studies could utilise ML and IoT not only for ASD
ASD participants were classified with a higher accuracy assessment and early detection, but also for intervention,
(83.3%) than Control ones (77.8%). assistance, monitoring and inclusion in future smart cities
[33].
TABLE I: CONFUSION MATRIX COMPARING OUR BROWSE TASK BEST
RESULT WITH THE BROWSE TASK BEST RESULT OF [27]. V. CONCLUSION
The current study employed Decision Trees, Logistic
Regression and Transfer-Learning aiming at high-
KOLLIAS 2022 YANEVA 2018 functioning ASD classification by eye-tracking data acquired
from web-related tasks and collected by [27]. Eye-tracking
ASD CONTROL ASD CONTROL data consisted of high-functioning ASD adults and controls,
who looked for information within web pages. Our proposed
method provided increased high-functioning ASD
ASD 83.3% 16.7% 70.6% 29.4%
classification accuracy concerning both Browse (80.50%)
and Search (81%) task compared to the results attained by
CONTROL 22.2% 77.8% 27.8% 72.2% [27]. Thus, the accuracy of a screening method not depending
on subjective measures is of essential importance, can be
Τhe best classification accuracy regarding the Search task improved by transfer-learning and is less-expensive, easier,
(81%) was achieved when all Gaze features, AOI ID, Media and less obtrusive than conventional ASD assessment
ID, Participant Age and Participant Gender were used and methods such as ADOS, ADI-R and fMRI which are less
Logistic Regression was applied. As it is shown in Table II, objective, costly, and time-consuming.
ASD participants were again classified with a higher ACKNOWLEDGEMENT
accuracy (82.3%) than Control ones (79.5%).
This project has received funding from the European
TABLE II: CONFUSION MATRIX COMPARING OUR SEARCH TASK BEST Union’s Horizon 2020 research and innovation programme
RESULT WITH THE BROWSE TASK BEST RESULT OF [27]. under grant agreement No 957406.
REFERENCES
KOLLIAS 2022 YANEVA 2018 [1] K. J. Tsuchiya et al., “Diagnosing autism spectrum
disorder without expertise: a pilot study of 5-to 17-year-old
ASD CONTROL ASD CONTROL
individuals using Gazefinder,” Frontiers in Neurology, vol.
11, p. 1963, 2021.
[2] T. Vu et al., “Effective and efficient visual stimuli
ASD 82.3% 17.7% 80.8% 19.2% design for quantitative autism screening: An exploratory
study,” in 2017 IEEE EMBS International Conference on
CONTROL 20.5% 79.5% 30.2% 69.8% Biomedical & Health Informatics (BHI), Orlando, FL, USA,
16–19 February, 2017, pp. 297–300.
[3] Lord, C., Rutter, M., DiLavore, P. C., and Risi, S.,
Earlier ASD classification eye-tracking studies which ADOS. Autism diagnostic observation schedule. Manual. Los
employed data collected in previous studies and tried to Angeles, CA: Western Psychological Services, 2001.
improve their classification performance dealt with datasets [4] C. Lord, M. Rutter, and A. Le Couteur, “Autism
including images as stimuli. For example, [28] employed Diagnostic Interview-Revised: a revised version of a
natural scene images, whereas [31] used images of children’s diagnostic interview for caregivers of individuals with
mothers who took part in the study. To our knowledge, our possible pervasive developmental disorders,” Journal of
study is the first which included eye-tracking data from web- autism and developmental disorders, vol. 24, no. 5, pp. 659–
related tasks collected in a previous study. 685, 1994.
IV. LIMITATIONS [5] Q. He, Q. Wang, Y. Wu, L. Yi, and K. Wei,
“Automatic classification of children with autism spectrum
Although the results of this study show that Machine disorder by using a computerized visual-orienting task,”
Learning and Transfer-Learning can improve the PsyCh Journal, vol. 10, no. 4, pp. 550–565, 2021.
classification accuracy concerning a dataset used in a [6] L. D. Wiggins, J. Baio, and C. Rice, “Examination
previous study, there are some limitations that can be of the Time Between First Evaluation and First Autism

Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on June 25,2024 at 07:20:29 UTC from IEEE Xplore. Restrictions apply.
Spectrum Diagnosis in a Population-based Sample,” Journal Systematic Review,” Journal of Autism and Developmental
of Developmental & Behavioral Pediatrics, vol. 27, no. 2, p. Disorders, pp. 1–16, 2021, doi:
S79, Apr. 2006. https://doi.org/10.1007/s10803-021-05106-5.
[7] C. M. Murphy et al., “Autism spectrum disorder in [21] K.-F. Kollias, C. K. Syriopoulou-Delli, P.
adults: Diagnosis, management, and health services Sarigiannidis, and G. F. Fragulis, “The Contribution of
development,” Neuropsychiatric Disease and Treatment, vol. Machine Learning and Eye-Tracking Technology in Autism
12, 2016, doi: 10.2147/NDT.S65455. Spectrum Disorder Research: A Systematic Review,”
[8] R. Carette, F. Cilia, G. Dequen, J. Bosche, J.-L. Electronics, vol. 10, no. 23, p. 2982, 2021.
Guerin, and L. Vandromme, “Automatic autism spectrum [22] K.-F. Kollias, C. K. Syriopoulou-Delli, P.
disorder detection thanks to eye-tracking and neural network- Sarigiannidis, and G. F. Fragulis, “The contribution of
based approach,” in International conference on IoT Machine Learning and Eye-tracking technology in Autism
technologies for healthcare, Angers, France, 24–25 October, Spectrum Disorder research: A Review Study,” in 2021 10th
2017, pp. 75–81. International Conference on Modern Circuits and Systems
[9] American Psychiatric Association, Diagnostic and Technologies (MOCAST), 2021, pp. 1–4.
statistical manual of mental disorders, 5th ed. Washington, [23] M. E. Alam, M. S. Kaiser, M. S. Hossain, and K.
DC: Author, 2013. Andersson, “An IoT-belief rule base smart system to assess
[10] A. Klin, S. Shultz, and W. Jones, “Social visual autism,” in 2018 4th International Conference on Electrical
engagement in infants and toddlers with autism: early Engineering and Information & Communication Technology
developmental transitions and a model of pathogenesis,” (iCEEiCT), 2018, pp. 672–676.
Neuroscience & Biobehavioral Reviews, vol. 50, pp. 189– [24] M. Hosseinzadeh et al., “A review on diagnostic
203, 2015. autism spectrum disorder approaches based on the Internet of
[11] K. Nayar, A. C. Voyles, L. Kiorpes, and A. Di Things and Machine Learning,” The Journal of
Martino, “Global and local visual processing in autism: An Supercomputing, vol. 77, pp. 2590–2608, 2020.
objective assessment approach,” Autism Research, vol. 10, [25] S. R. Dutta, M. Roy, S. Datta, and R. Datta, “IoT in
no. 8, pp. 1392–1404, 2017. Autism Detection in Its Early Stages,” in Internet of Things:
[12] V. Yaneva, S. Eraslan, Y. Yesilada, and R. Mitkov, Enabling Technologies, Security and Social Implications, S.
“Detecting high-functioning autism in adults using eye Kumar Pani and M. Pandey, Eds. Singapore: Springer, 2021,
tracking and machine learning,” IEEE Transactions on pp. 47–58. doi: 10.1007/978-981-15-8621-7_5.
Neural Systems and Rehabilitation Engineering, vol. 28, no. [26] L. Torrey and J. Shavlik, “Transfer Learning,”
6, pp. 1254–1261, 2020. Handbook of Research on Machine Learning Applications
[13] M. C. Frank, E. Vul, and R. Saxe, “Measuring the and Trends: Algorithms, Methods, and Techniques, 2010.
development of social attention using free-viewing,” Infancy, https://www.igi-global.com/chapter/transfer-
vol. 17, no. 4, pp. 355–375, 2012. learning/www.igi-global.com/chapter/transfer-
[14] N. J. Sasson and J. T. Elison, “Eye tracking young learning/36988 (accessed Dec. 27, 2021).
children with autism,” Journal of visualized experiments: [27] V. Yaneva, L. A. Ha, S. Eraslan, Y. Yesilada, and
JoVE, no. 61, 2012. R. Mitkov, “Detecting autism based on eye-tracking data
[15] D. Bone, M. S. Goodwin, M. P. Black, C.-C. Lee, K. from web searching tasks,” in Proceedings of the 15th
Audhkhasi, and S. Narayanan, “Applying machine learning International Web for All Conference, Lyon, France, 23–25
to facilitate autism diagnostics: pitfalls and promises,” April, 2018, pp. 1–10.
Journal of autism and developmental disorders, vol. 45, no. [28] M. Jiang and Q. Zhao, “Learning visual attention to
5, pp. 1121–1136, 2015. identify people with autism spectrum disorder,” in
[16] R. Carette, M. Elbattah, F. Cilia, G. Dequen, J.-L. Proceedings of the ieee international conference on computer
Guérin, and J. Bosche, “Learning to Predict Autism Spectrum vision, Venice, Italy, 22–29 October, 2017, pp. 3267–3276.
Disorder based on the Visual Patterns of Eye-tracking [29] S. Wang et al., “Atypical visual saliency in autism
Scanpaths.,” in HEALTHINF, 2019, pp. 103–112. spectrum disorder quantified through model-based eye
[17] J. Peral, D. Gil, S. Rotbei, S. Amador, M. Guerrero, tracking,” Neuron, vol. 88, no. 3, pp. 604–616, 2015.
and H. Moradi, “A Machine Learning and Integration Based [30] P. E. Hart, D. G. Stork, and R. O. Duda, Pattern
Architecture for Cognitive Disorder Detection Used for Early classification. Wiley Hoboken: Hoboken, NJ, USA, 2000.
Autism Screening,” Electronics, vol. 9, no. 3, Art. no. 3, Mar. [31] J. Li, Y. Zhong, J. Han, G. Ouyang, X. Li, and H.
2020, doi: 10.3390/electronics9030516. Liu, “Classifying ASD children with LSTM based on raw
[18] Y. Zhou, F. Yu, and T. Duong, “Multiparametric videos,” Neurocomputing, vol. 390, pp. 226–238, 2020.
MRI characterization and prediction in autism spectrum [32] J. Li, Y. Zhong, and G. Ouyang, “Identification of
disorder using graph theory and machine learning,” PloS one, ASD children based on video data,” in 2018 24th
vol. 9, no. 6, p. e90405, 2014. International conference on pattern recognition (ICPR),
[19] A. Crippa et al., “Use of machine learning to Beijing, China, 20–24 August, 2018, pp. 367–372.
identify children with autism and their motor abnormalities,” [33] T. Ghosh et al., “Artificial intelligence and internet
Journal of autism and developmental disorders, vol. 45, no. of things in screening and management of autism spectrum
7, pp. 2146–2156, 2015. disorder,” Sustainable Cities and Society, vol. 74, p. 103189,
[20] M. E. Minissi, I. A. C. Giglioli, F. Mantovani, and Nov. 2021, doi: 10.1016/j.scs.2021.103189.
M. A. Raya, “Assessment of the Autism Spectrum Disorder
Based on Machine Learning and Social Visual Attention: A

Authorized licensed use limited to: SRM Institute of Science and Technology. Downloaded on June 25,2024 at 07:20:29 UTC from IEEE Xplore. Restrictions apply.