Review
Acta Cytologica 2021;65:330–334 Received: August 13, 2020
Accepted: November 4, 2020
DOI: 10.1159/000513286 Published online: February 23, 2021
Applications of Computational
Pathology in Head and Neck
Cytopathology
Trang K. Lollie Jeffrey F. Krane
Department of Pathology and Laboratory Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
Keywords therapeutic decision-making. Integration of machine
Thyroid · Head and neck · Salivary gland · Squamous cell learning processes such as artificial neural networks
carcinoma · Fine-needle aspiration · Computational analysis (ANNs), deep learning, genetic algorithms, discriminant
analysis, and classification and regression trees has been
successfully implemented in various sectors of medicine
Abstract [1]. While light microscopic evaluation remains the gold
Background: The application of computational technology standard for most pathological diagnosis, the precise
to head and neck cytology material has been explored ex- measurements and quantification provided by computa-
perimentally in several areas with a variety of potential ap- tional analysis promise a degree of objectivity that may
plications. Summary: This review summarizes the applica- help to resolve diagnostic uncertainty in ambiguous cases
tion of these techniques to the diagnosis of thyroid, salivary that are prone to subjective interpretation. Computation-
gland, and other head and neck fine-needle aspiration spec- al methodology also offers the ability to perform and an-
imens. Current limitations and potential future applications alyze complex iterative tasks in an objective quantifiable
in diagnosis are discussed along with the possibilities for manner not possible with routine examination.
therapeutic applications of computational methodology. The ease with which many head and neck lesions can
Key Message: Particularly promising applications include re- be accessed via palpation or ultrasound guidance coupled
solving diagnostic uncertainty in indeterminate thyroid as- with the capability of rapidly and accurately diagnosing
pirates and assessing the tumor microenvironment in re- such lesions via minimally invasive fine-needle aspiration
sponse to immunotherapy for squamous cell carcinoma. (FNA) biopsy have increased cytopathology’s footprint in
© 2021 S. Karger AG, Basel head and neck pathology. Diagnoses are rendered by
evaluating individual cells instead of larger tissue biopsies
which improves patient safety through reduced proce-
Introduction dural complications while also decreasing overall costs.
The ability to analyze specific diagnostic cytologic cellular
Technological advancements in artificial intelligence criteria makes cytopathology ideal for computational ap-
and machine learning have enabled groundbreaking plications by objectively quantifying morphologic find-
computational models to support clinical diagnosis and ings. This review will address the efforts and advances
130.209.6.61 - 8/15/2021 10:01:15 PM
[email protected] © 2021 S. Karger AG, Basel Jeffrey F. Krane
www.karger.com/acy Department of Pathology and Laboratory Medicine
David Geffen School of Medicine at UCLA
10833 Le Conte Ave., CHS 13-145F, Los Angeles, CA 90095 (USA)
Glasgow Univ.Lib.
Downloaded by:
jkrane @ mednet.ucla.edu
that have been made in applying digital and computa- or malignant. At a single patient level, 98% of cases were
tional techniques to head and neck cytology, specifically classified correctly. In a subsequent larger study, the
with regard to thyroid FNA, salivary gland FNA, and same group verified that a variety of ANN approaches
head and neck squamous cell carcinoma. Efforts to apply were superior to a linear classifier approach (87–89%
computational cytology for clinical diagnosis and man- overall accuracy vs. 61%) on an individual case basis. In
agement are summarized, shortcomings of current mod- those cases with an indeterminate cytologic diagnosis,
els are highlighted, and potential future applications are overall accuracy of the ANN was diminished (74–81%)
discussed. [5].
ANNs have been developed to recognize specific path-
Thyroid Lesions ological thyroid entities. In 2007, Shapiro et al. [6] applied
Thyroid FNA is the gold standard diagnostic test for ANNs to analyze nuclear morphometric parameters
thyroid nodule evaluation. Most thyroid nodules prove to (area, perimeter, and shape) and chromatin texture (mean
be benign via FNA. For follicular neoplasms, thyroid value and standard deviation of gray levels) to differenti-
FNA is primarily a screening test with the majority of cy- ate follicular tumors and reported an ability to distinguish
tologically concerning nodules ultimately proving to be follicular adenomas from follicular carcinomas in 87%
benign. In contrast, thyroid FNA can serve as a diagnostic (73/84) of cases. Convoluted neural networks, a form of
test for papillary thyroid carcinoma (PTC), particularly ANN that propagates feedforward neural networks of
for classic and tall cell variants in which the cytologic fea- whole image inputs and accordingly categorizes images
tures are usually definitive. Follicular variants of PTC and into specific categories, were tested by Sanyal et al. [7] to
the indolent noninvasive follicular thyroid neoplasm separate PTC from non-PTC. Whole images at both ×10
with papillary-like nuclear features (NIFTP) are more and ×40 magnification were passed through multiple
challenging and often result in an indeterminate diagno- feedforward layers and eventually converted to a binary
sis (Bethesda categories III–V). Indeterminate aspirates choice of PTC or non-PTC with a reported sensitivity of
encompass benign lesions for which surgery could be 90%, specificity of 83%, and overall accuracy of 85%.
avoided and neoplasms where lobectomy or near total However, limitations attributed to potential “overfitting”
thyroidectomy would be more appropriate. Accordingly, of training data resulted in findings such as vague papil-
there is intense interest in resolving diagnostic uncertain- lary formations by normal follicular cells or thick colloid
ty for thyroid nodules with indeterminate cytologic fea- occasionally being misclassified as PTC.
tures. Molecular testing has largely filled this role, but an A number of studies have attempted to transform sub-
alternative, potentially cost-effective approach is to use jective cytologic criteria used in thyroid FNA diagnosis
computational assistance in an attempt to resolve diag- into quantitatively classifiable parameters. Most such
nostic uncertainty. studies focus on nuclear parameters that distinguish pap-
Computational technology has been applied to thy- illary or follicular carcinomas from benign thyroid nod-
roid FNA material in numerous studies dating back as far ules. For example, Murata et al. [8] used image analysis
as 1980 [2] (reviewed by Chain et al. [3]). Comparison of and morphological abstraction by factor analysis with
these studies is challenging as they vary in the substrate combined morphometry and textural parameters to show
being assessed (smear preparation types and static vs. that PTCs had larger, more irregular nuclei with higher
whole slide images), diagnostic material input (benign, contrast chromatin distribution while follicular carcino-
PTC, and follicular carcinoma), parameters being as- mas had larger nuclei with a more monotonous chroma-
sessed, diagnoses attempting to be distinguished, and the tin distribution when compared to their benign counter-
underlying technology implemented to make the assess- parts. Overall, nuclear area measurements appear to most
ment. An early proof of concept of computational cytol- reliably distinguish PTC from benign nodules, although
ogy’s diagnostic capabilities was provided in 1996 by other factors related to nuclear shape and chromatin
Karakitsos et al. [4] using backpropagation in ANNs to characteristics are predictive in some studies [3, 9–14].
improve distinction between benign versus malignant For follicular carcinomas, nuclear size variation may be a
thyroid nuclei. Using image analysis to assess 26 param- more important distinguishing feature [15, 16].
eters of nuclear morphometry in May-Grünwald-Giem- While distinction of clearly benign versus malignant
sa-stained smears from 51 specimens (including 12 PTCs lesions holds academic interest, the ultimate value of
and 4 follicular/Hürthle cell carcinomas), correct classi- quantitative techniques for diagnosis likely rests in how
fication was achieved for 90.6% of nuclei as either benign useful they might be in resolving diagnostic uncertain-
130.209.6.61 - 8/15/2021 10:01:15 PM
Computational Pathology in Head and Acta Cytologica 2021;65:330–334 331
Neck Cytopathology DOI: 10.1159/000513286
Glasgow Univ.Lib.
Downloaded by:
ty in challenging cases. Relatively few studies have fo- scoring system for making this distinction in clinical
cused on this issue, particularly since the advent of the practice.
use of Bethesda system terminology [3, 17–19]. In one To date, most morphometric-based studies in thyroid
such study, computerized nuclear morphometry was FNA have been limited by small sample sizes, the need for
assessed in 9 cytohistologically discordant cases. Two manual identification and extraction of nuclei for analy-
cases originally diagnosed as AUS/FLUS on cytology sis, and long computational times. The more recent use
that proved to be PTC on histopathology exhibited nu- of whole slide imaging promises to facilitate and poten-
clear parameters (such as mean nuclear measurements tially automate future applications, but a current limita-
of diameter, perimeter, and area) closer to those ob- tion is the massive storage requirements required which
served in other neoplastic cases. Conversely, 7 false- has limited studies thus far to the use of a single slide from
positive cases originally diagnosed as Bethesda category each case [3, 19]. Nevertheless, these recent studies sug-
V or VI that proved to be goiter or lymphocytic thyroid- gest that there is significant potential in the near future
itis histologically exhibited nuclear features compara- for using computational technology with machine learn-
ble to other nonneoplastic cases [17]. In a similar dem- ing algorithms in refining diagnostic uncertainty in inde-
onstration of the potential value of nuclear morphom- terminate cases.
etry in equivocal cases, Chain et al. [3] analyzed nuclear
area and elongation of follicular cells in thyroid FNAs Salivary Gland Lesions
in whole slide images. PTC showed the largest mean Cytologic diagnosis of salivary gland tumors is espe-
and median nuclear area and elongation, whereas be- cially challenging due to the wide variety of benign and
nign lesions had the smallest and roundest morpholo- malignant neoplasms described, many of which exhibit
gy. Statistically significant differences were identified extensive morphologic overlap. This complexity in clas-
among indeterminant cases that proved to be benign sification suggests that broad cytomorphometric analy-
versus those proving to be PTC. Cases that proved to be sis of salivary gland tumors may lack the nuance needed
NIFTP also differed from PTC cases but exhibited more for specific differential diagnostic considerations. Never-
overlap with benign cases. theless, one such attempt exists in the literature in which
In the largest study to date, a machine learning algo- Obad-Kovacevic et al. [21] applied 13 morphometric pa-
rithm was used to predict malignancy on whole slide im- rameters to a study of 64 parotid gland tumors. Area,
ages from 908 cases [19]. The authors used 2 convoluted perimeter, convex area, convexity, maximal and mini-
neural networks, one developed to identify follicular cells mum radius, length, breadth, form factor, elongation
and the second to classify aspirates as benign or malig- factor, area symmetry factor, and perimeter symmetry
nant and to recapitulate Bethesda system classification. factor significantly differed between benign and malig-
The algorithm achieved comparable overall performance nant tumors. Receiver operative characteristic curves of
to an expert cytopathologist in distinguishing benign and area, perimeter, convex area, maximum radius, length,
malignant nodules with a sensitivity of 92.0% and a spec- and form factor were useful in distinguishing benign
ificity of 90.5%. Since the cytopathologist performed bet- from malignant tumors. Minimal and maximum radius,
ter at the extremes (benign or malignant diagnoses), a convexity, breadth, form factor, and elongation signifi-
combined approach in which the machine learning algo- cantly differed in whole cell morphometry parameters.
rithm was only applied in cytologically indeterminate The authors concluded that digital cytomorphometry
cases was evaluated and yielded the greatest overall per- can assist as an objective tool in distinguishing malignant
formance. The authors propose that their algorithm could from benign salivary gland tumors, especially in cases
be used in risk stratifying indeterminate aspirates with with hypocellular material. Further study is clearly need-
potential improvements made via modeling that would ed, particularly in the context of specific diagnostically
integrate clinical and ultrasonographic data. challenging scenarios (e.g., basaloid aspirates) to estab-
Rather than directly assessing morphologic features, a lish whether there is utility for this technology in clinical
recent study used a machine learning algorithm to distin- practice.
guish classic PTC from NIFTP/encapsulated follicular
variants of PTC based on microscopic descriptions con- Squamous Cell Carcinoma
tained within FNA reports [20]. This method achieved a Some of the most exciting potential applications of
76% success rate overall. The authors propose that their computational cytology are in the diagnosis and manage-
approach could be valuable in developing a data-driven ment of head and neck squamous cell carcinomas. Cyto-
130.209.6.61 - 8/15/2021 10:01:15 PM
332 Acta Cytologica 2021;65:330–334 Lollie/Krane
DOI: 10.1159/000513286
Glasgow Univ.Lib.
Downloaded by:
logic diagnosis of squamous cell carcinoma is typically allows for frequent serial sampling compared to tissue bi-
straightforward; however, cystic lesions are often a nota- opsy. A novel, technologically innovative rapid chemis-
ble exception where there can be difficulty in distinguish- try-enabled FNA (“FAST-FNA”) is currently being evalu-
ing cystic metastasis of squamous cell carcinoma from a ated to enable rapid multiplex biomarker analysis using
benign developmental cyst due to sample hypocellularity, DNA barcode-modified antibodies, ultrafast destaining
extensive inflammation, reactive changes, and cellular and “click chemistry” reactions, and automated image cy-
debris [22]. Image cytometry DNA analysis has been used tometry readers [26]. Analogous to the combined positive
to assess aneuploidy (defined as DNA values exceeding score used in tissue biopsies, 2 newly developed scores
5c) to make this distinction both retrospectively [23] and (FAST PD-L1 and FAST cold/hot score) could potential-
prospectively[24]. In the prospective study, 50 patients ly offer a quicker (approximately 2 h from cell harvest-
with cystic lesions in the neck had aneuploidy identified ing), more cost effective, and more nuanced alternative to
by DNA analysis in 8/9 cases of cytologically confirmed standard PD-L1 assessment that is amenable to serial
squamous cell carcinoma metastasis and in all 4 cytolog- monitoring. Use of a deep learning-enabled fluorescence
ically inconclusive but histologically confirmed cystic cytometer, a stand-alone unit weighing 6 pounds, is cur-
metastatic lesions. Esophageal carcinoma accounted for rently under clinical investigation.
the one discordant diploid case. Conversely, all benign
lesions were diploid.
Furthermore, point-of-care technology is being devel- Conclusion
oped to provide immediate feedback of malignant oral
lesions that require urgent referral for treatment. In a cy- Despite great promise, significant hurdles remain to
tology-on-a-chip approach, brush cytology samples were the widespread adoption of computational cytology in
applied to microfluidic-based cell capture systems, and head and neck cytology specimens. Nuclear morphomet-
numeric indices were assigned based on cell phenotypes ric studies typically require tedious and time-consuming
to determine probability of malignancy [25]. Additional manual selection of target nuclei (typically at least 100
features such as clinical history were also integrated into nuclei per case). Initial startup requires extensive testing,
the program’s interface and could be coupled with mo- potentially expensive specialized equipment, and fine-
lecular marker results. These innovative studies demon- tuning of multiple parametric measurements to deter-
strate computational cytology’s potential in quickly triag- mine suitable algorithms. The training process mandates
ing cases and improving referral efficiency, particularly in substantial computational power and a large training set
settings where cytopathology expertise may not be imme- to provide enough data for machine learning, which may
diately available. be prone to data overfitting.
A recent, potentially groundbreaking application of To date, the use of computational cytology in head and
computational cytology is in the field of cancer immuno- neck cytology has largely been limited to the research set-
therapy surveillance. PD-1/PD-L1 inhibitor immuno- ting. Extensive literature exists within the area of thyroid
therapy has revolutionized the treatment landscape across cytology, but the technology has yet to establish its ability
a multitude of tumor types. Response to PD-1/PD-L1 in- to provide significant added value in routine practice.
hibitors is predicted by monitoring PD-L1 level expressed Nevertheless, advances in computing power, storage, im-
by tumor cells. However, tumor cells often adapt under aging, and machine learning algorithms point to real po-
treatment pressures and fluidly reshape the tumor micro- tential in the near future for this technology to play a role
environment (TME). The complexity of defining changes in resolving diagnostic uncertainty in indeterminate thy-
in the TME that correlate with treatment responsiveness roid FNA specimens as an adjunct to routine microscopy
exceeds the capabilities of routine pathologic evaluation and as a potential alternative to molecular testing. Con-
due to the need to evaluate a complex array of parameters versely, the complexity of salivary gland tumor diagnosis
and cell types both spatially and temporally. Mapping the suggests that computational techniques are unlikely to
TME using computational techniques at the cellular and have a widespread role in the diagnostic realm for these
molecular levels may help identify the next generation of specimens in the foreseeable future. With regard to head
predictive biomarkers and further guide clinical manage- and neck squamous cell carcinomas, computational cy-
ment as the TME evolves over the course of treatment. tology may have potential as a tool in point-of-care test-
Cytologic assessment of FNAs is ideal for posttreatment ing. The potential application of complex quantitative
surveillance as it is minimally invasive, cost effective, and analysis to evaluate the TME in head and neck squamous
130.209.6.61 - 8/15/2021 10:01:15 PM
Computational Pathology in Head and Acta Cytologica 2021;65:330–334 333
Neck Cytopathology DOI: 10.1159/000513286
Glasgow Univ.Lib.
Downloaded by:
cell carcinomas via rapid multiplex biomarker analysis Conflict of Interest Statement
for immunotherapy surveillance holds great theoretical
The authors have no conflicts of interest to declare.
promise. Should this promise be fulfilled, one can readily
envision the application of a similar approach to other
advanced head and neck malignancies, such as thyroid
Funding Sources
and salivary gland carcinomas, as well as tumors from
other primary sites. No funding was received.
Statement of Ethics Author Contributions
The authors have no ethical conflicts to disclose. T.K.L. and J.F.K.: conception, analysis, drafting, revising, and
final approval of work.
References
1 Pouliakis A, Karakitsou E, Margari N, Bountris 10 Słowińska-Klencka D, Klencki M, Sporny S, 20 Maleki S, Zandvakili A, Gera S, Khutti S, Ger-
P, Haritou M, Panayiotides J, et al. Artificial Lewiński A. Karyometric analysis in the cyto- sten A, Khader S. Differentiating noninvasive
neural networks as decision support tools in cy- logic diagnosis of thyroid lesions. Anal Quant follicular thyroid neoplasm with papillary-like
topathology: past, present, and future. Biomed Cytol Histol. 1997;19(6):507–13. nuclear features from classic papillary thyroid
Eng Comput Biol. 2016 Feb 18;7:1–18. 11 Kirillov VA, Yuschenko YP, Paplevka AA, carcinoma: analysis of cytomorphologic de-
2 Boon ME, Löwhagen T, Willems JS. Planimet- Demidchik EP. Thyroid carcinoma diagnosis scriptions using a novel machine-learning ap-
ric studies on fine needle aspirates from follicu- based on a set of karyometric parameters of fol- proach. J Pathol Inform. 2019;10(1):29.
lar adenoma and follicular carcinoma of the licular cells. Cancer. 2001;92(7):1818–27. 21 Obad-Kovačević D, Kardum-Skelin I, Jelić-
thyroid. Acta Cytol. 1980;24(2):145–8. 12 Harms H, Hofmann M, Ruschenburg I. Fine Puškarić B, Vidjak V, Blašković D. Parotid
3 Chain K, Legesse T, Heath JE, Staats PN. Digital needle aspiration of the thyroid: Can an image gland tumors: correlation between routine cy-
image-assisted quantitative nuclear analysis processing system improve differentiation? tology and cytomorphometry by digital image
improves diagnostic accuracy of thyroid fine- Anal Quant Cytol Histol. 2002;24(3):147–53. analysis using conventional and newly intro-
needle aspiration cytology. Cancer Cytopathol. 13 Khatri P, Choudhury M, Jain M, Thomas S. duced cytomorphometric parameters. Diagn
2019;127(8):501–13. Role of morphometry in the cytological differ- Cytopathol. 2013;41(9):776–84.
4 Karakitsos P, Cochand-Priollet B, Guillausseau entiation of benign and malignant thyroid le- 22 Krane JF. Role of cytology in the diagnosis and
PJ, Pouliakis A. Potential of the back propaga- sions. J Cytol. 2017;34(1):1–4. management of HPV-associated head and neck
tion neural network in the morphologic exam- 14 Deka L, Gupta S, Gupta R, Gupta K, Kaur CJ, carcinoma. Acta Cytol. 2013;57(2):117–26.
ination of thyroid lesions. Anal Quant Cytol Singh S. Nuclear morphometry and texture 23 Nordemar S, Högmo A, Lindholm J, Tani E,
Histol. 1996;18(6):494–500. analysis on cytological smears of thyroid neo- Sjöström B, Auer G, et al. The clinical value of
5 Cochand-Priollet B, Koutroumbas K, Megalo- plasms: a study of 50 cases. Malays J Pathol. image cytometry DNA analysis in distinguish-
poulou TM, Pouliakis A, Sivolapenko G, Kara- 2017;39(1):33–7. ing branchial cleft cysts from cystic metastases
kitsos P. Discriminating benign from malig- 15 Nagashima T, Suzuki M, Oshida M, Hashimo- of head and neck cancer. Laryngoscope. 2002;
nant thyroid lesions using artificial intelligence to H, Yagata H, Shishikura T, et al. Morphom- 112(11):1983–7.
and statistical selection of morphometric fea- etry in the cytologic evaluation of thyroid fol- 24 Nordemar S, Tani E, Högmo A, Jangard M,
tures. Oncol Rep. 2006;15(4):1023–6. licular lesions. Cancer. 1998;84(2):115–8. Auer G, Munck-Wikland E. Image cytometry
6 Shapiro NA, Poloz TL, Shkurupij VA, Tarkov 16 Savala R, Dey P, Gupta N. Artificial neural net- DNA-analysis of fine needle aspiration cytolo-
MS, Poloz VV, Demin AV. Application of arti- work model to distinguish follicular adenoma gy to aid cytomorphology in the distinction of
ficial neural network for classification of thy- from follicular carcinoma on fine needle aspira- branchial cleft cyst from cystic metastasis of
roid follicular tumors. Anal Quant Cytol Histol. tion of thyroid. Diagn Cytopathol. 2018;46(3): squamous cell carcinoma: a prospective study.
2007;29(2):87–94. 244–9. Laryngoscope. 2004;114(11):1997–2000.
7 Sanyal P, Mukherjee T, Barui S, Das A, Gango- 17 Yashaswini R, Suresh TN, Sagayaraj A. Cyto- 25 McRae MP, Modak SS, Simmons GW, Tro-
padhyay P. Artificial intelligence in cytopathol- logical evaluation of thyroid lesions by nuclear chesset DA, Kerr AR, Thornhill MH, et al.
ogy: a neural network to identify papillary car- morphology and nuclear morphometry. J Cy- Point-of-care oral cytology tool for the screen-
cinoma on thyroid fine-needle aspiration cytol- tol. 2017;34(4):197–202. ing and assessment of potentially malignant
ogy smears. J Pathol Inform. 2018;9(1):43. 18 Celik ZE, Altinay S, Kilinc F, Arslan N, Yilmaz oral lesions. Cancer Cytopathol. 2020; 128(3):
8 Murata S, Mochizuki K, Nakazawa T, Kondo T, BS, Karabagli P, et al. Using computerized cy- 207–20.
Nakamura N, Yamashita H, et al. Morphologi- tomorphometry to distinguish between benign 26 Pai SI, Faquin WC, Sadow PM, Pittet MJ,
cal abstraction of thyroid tumor cell nuclei us- and malignant cases in thyroid fine-needle as- Weissleder R. New technology on the horizon:
ing morphometry with factor analysis. Microsc piration cytology. Diagn Cytopathol. 2016; fast analytical screening technique FNA (FAST-
Res Tech. 2003;61(5):457–62. 44(11):902–11. FNA) enables rapid, multiplex biomarker anal-
9 Sassi I, Mangili F, Sironi M, Freschi M, Canta- 19 Elliott Range DD, Dov D, Kovalsky SZ, Henao ysis in head and neck cancers. Cancer Cytopa-
boni A. Morphometric evaluation of fine nee- R, Carin L, Cohen J. Application of a machine thol. 2020 Aug 25;128(11):782–791.
dle biopsy of single thyroid nodules. Pathol Res learning algorithm to predict malignancy in
Pract. 1989;185(5):722–5. thyroid cytopathology. Cancer Cytopathol.
2020;128(4):287–95.
130.209.6.61 - 8/15/2021 10:01:15 PM
334 Acta Cytologica 2021;65:330–334 Lollie/Krane
DOI: 10.1159/000513286
Glasgow Univ.Lib.
Downloaded by:
