Oller, John W., Jr.

APRIL GINTHER

The work of John Oller (1943– ) in the 1970s and 1980s contributed greatly to the emergence
and recognition of language testing as a discipline within the field of applied linguistics.
A prolific writer, author of six books, more than 230 published articles, and coauthor/
editor of 10 collections, his work includes examination of a wide range of topics related
to language testing, intelligence testing, communication disorders, and other subjects.
Characteristic of his work is a theoretical orientation emphasizing semiotics, the study of
signs and symbols, and the pragmatics of knowledge, beliefs, and expectations of language
users in the contexts of ordinary experience. In addition to his contributions to theoretical
discussions of testing methods and the nature of language proficiency, Oller is recognized
as a proponent of innovative teaching methods. Methods That Work: A Smorgasbord of Ideas
for Language Teachers, edited with Richard-Amato in 1983, remains a classic attesting to the
advantages of distinctly pragmatic approaches, followed by the 1993 version generalizing
the application of pragmatic approaches to first language acquisition and the teaching of
literacy.
Oller’s explication of pragmatic naturalness criteria addresses the quality of the instruments
and methods used to measure language proficiency, defining a pragmatic test as “any
procedure or task that causes the learner to process sequences of elements in a language
that conform to the normal contextual constraints of that language and which requires the
learner to relate sequences of linguistic elements via pragmatic mappings to extralinguistic
contexts” (1979, p. 38). Based on the pragmatic naturalness criteria, a good test, or any
valid testing, assessment, or teaching method, invokes the learner’s pragmatic expectancy
grammar, described as the psychologically real system that links, relates, or maps the
characteristics of language to knowledge about the context in which an utterance occurs.
The idea of expectancy, a critical notion in Oller’s arguments, is related to the gestalt notion
of closure through which human beings solve problems by supplying missing elements.
As Oller points out, whether completing a pattern, solving an equation, or communicating
with another person, it is always possible to predict partially what will come next in any
given sequence of elements: the more accurate the expectations, the more efficient the
problem solving.
Oller’s work contributed to the shift away from discrete-point language testing to assess-
ment methods he identified as integrative. Drawing on and developing Carroll’s (1961)
original distinction, Oller described discrete-point and integrative language tests as oppo-
site ends of a continuum. He explained, “The concept of an integrative test was born in
contrast with the definition of a discrete point test. If discrete items take language skill
apart, integrative tests put it back together” (1979, p. 37). While discrete-point tests purport
to focus attention on one isolated element of grammar at a time, integrative tests require
access to meaning through deeper linguistic resources. By focusing the examinee’s or
learner’s attention on surface elements of grammar, discrete-point items were less effective
in engaging the deeper grammatical system and thereby failed to assess how the learner
would use language in ordinary contexts of experience.
While discourse-based testing procedures Oller identified as integrative included cloze
procedures, dictation, oral interviews, and composition, it is important to keep in mind

The Encyclopedia of Applied Linguistics, Edited by Carol A. Chapelle.

© 2013 Blackwell Publishing Ltd. Published 2013 by Blackwell Publishing Ltd.
DOI: 10.1002/9781405198431.wbeal0882
2 oller, john w., jr.

that the distinction that Oller was making was not one that relied on the surface charac-
teristics of any item format. He recognized that in practice, the most discrete items require
some integrative skills and the most integrative procedures always demand use of surface
forms that can be scored as if they were more or less discrete items. He also noted that
multiple-choice items do not necessarily constitute discrete-point tests. For example, the
abilities required to identify the best alternative answer on a reading comprehension test
nearly always lean toward the integrative rather than the discrete end of the continuum.
Nor is it necessarily the case that an integrative test requires that sources of information
be integrated across different modalities or skills (e.g., listening and reading), although
particular integrative tasks may do so. The desire for clarification of these points lead to
one of the more interesting and protracted debates that has taken place in language testing
to date.
Oller’s argument that discourse-based cloze tests (Oller & Conrad, 1971; Oller, 1972;
Chihara, Oller, Weaver, & Chavez-Oller, 1977; Oller, 1979) assess abilities for examinees
to use deep grammatical constraints was challenged by researchers (Alderson, 1979) who
argued that the selection of the words needed to fill deletions could be successful based
solely on information available within five to seven words on either side of a deletion. If
cloze tests measured only such short-range grammatical constraints, the method was flawed
as a measure of deep and general proficiency. Bachman (1982) also argued that cloze tasks
based on random deletions may not systematically target long-range discourse constraints,
and others rejected the method on the basis of their acceptance of null results comparing
sequential versus scrambled cloze passages (Shanahan, Kamil, & Tobin, 1982). Given that
null results are never a satisfactory basis for the rejection of any empirically testable
hypothesis, the perceived importance of these findings was challenged (Cziko, 1983) and
many ensuing studies investigated deletion procedures and scoring procedures, with respect
to short- and long-range discourse constraints, as well as individual and group effects
using a variety of statistical methods and fundamentally different experimental designs.
Studies showing the impact of long-range constraints on random-deletion discourse-based
cloze tasks were actually replicated in all cases where appropriately sensitive (powerful)
designs were used. Reported null contrasts, on the other hand, comparing scrambled
versus sequential texts across different texts and groups of examinees, failed to incorporate
the necessary counterbalancing of within- and between-subjects variance, order effects,
and differences across the selected texts themselves. All those variables and interactions
between them can be controlled in an appropriately counterbalanced repeated-measures
design, but where such counterbalancing was omitted, the contrasts of interest between
sequential versus scrambled texts were commonly (almost always) swamped, rendering
the effects of long-range discourse constraints on specific items statistically insignificant.
However, with more subtle repeated-measures designs, it was demonstrated that both
short-range and long-range discourse constraints (certainly ones ranging beyond 50 words)
are tapped by standard, random-deletion approaches to cloze test construction (Oller &
Jonz, 1994; Jonz, 1987). Later, it would be shown on a strictly logical basis, confirmed by
many empirical studies, that a substantial number of random deletions could not fail to
tap long-range constraints (Oller, Chen, Oller, & Pan, 2005; Oller & Chen, 2007).
Statistical investigations of integrative tests, following a principal components analysis
suggested by Nunnally (1967), the long-standing argument for a general factor of intelli-
gence first noted by Spearman (1904), and the growing evidence for the profound con-
nectedness of syntax, semantics, and pragmatics within grammatical systems (Oller, 1970,
1975), led to the simple but ultimately incorrect hypothesis that language abilities might
best be represented by a single underlying trait (Oller, 1976). That unitary trait hypothesis
was challenged and rejected with more sophisticated factor analytic procedures (Bachman
 oller, john w., jr. 3

& Palmer, 1981, 1982; Oller, 1983; Carroll, 1993; Bachman, Davidson, Ryan, & Choi, 1995;
Kunnan, 1995). The whole controversy that ensued, however, as Stansfield (2008) has noted,
focused attention on more sophisticated statistical procedures. Bachman (1990) wrote,
“Oller’s work, as well as the research it stimulated, firmly established construct validation
as a central concern of language testing research and generated renewed interest in factor
analytic procedure” (p. 7).
The unitary trait hypothesis was wrong, but an abstracted general factor accounting
for the lion’s share of the variance in language tests, intelligence measures, achievement
tests, and even personality inventories would remain as noted by Oller, Scott, and Chesarek
(1991) and Carroll (1993). As with the debate surrounding cloze procedure, even within
the present decade, the research and discussions generated by the unitary trait hypothesis
still resonate. Current large-scale factor analytic studies seldom fail to carry out model
comparisons to include (or exclude) a higher-order general factor (e.g., Sawaki, Stricker,
& Oranje, 2009). The models that include a higher-order factor typically provide better fits
to the data (just as Carroll, 1993, predicted).
Over the last two decades, Oller has continued to study “language acquisition and its
relation to intelligence, cognition, social development and their interactions” (Oller, personal
communication). His most recent work with the general theory of signs (GTS), elaborated
in Oller (2005), Oller et al. (2005), Oller, Oller, and Badon (2006, 2010), and Oller (2010),
have led into a more general investigation of sign systems not only in linguistics, but also
in genetics, biochemistry, and especially, the amazingly complex and layered immune
systems (Oller, 2010).
Oller (2005) explains:

Building on the foundational trichotomy of icons, indexes, and symbols, GTS addresses
three interrelated questions in its three component theories: (a) The theory of TNRs [true
narrative representations] is about how any signs can attain any meaning, (b) the theory
of abstraction concerns the unpacking and deciphering of signs by infants, and (c) the
theory of systems grammar examines the properties of the sign systems generated by the
prior theories. (p. 119)

Central to GTS is the idea that “in linguistic and genetic terms, successful discovery of
language conventions, i.e., language acquisition, and ordinary successful communications
in all domains of experience and biochemistry—depend ultimately (and unsurprisingly)
on valid representations that are faithfully mapped onto whatever facts they may purport
to be about” (Oller, personal communication).
Pragmatic mapping remains central and affords a compelling alternative to strictly
semantic and formal (autonomous syntactic) theories of language acquisition, grammar,
and language testing. In language testing, the shift Oller encouraged continues away from
discrete-point approaches toward integrated performances in contexts broader than those
circumscribed by the surface forms or component domains of any particular language-
processing task. However, Oller’s emphases on processing, episodic organization (discur-
sive constraints), and cognition have been less fully elaborated in current language testing
research and theory. It may be the case that in the next decade, processing, cognition, and
pragmatics will receive greater attention. In any case, Oller’s legacy will remain one that
will continue to be influential while providing rich opportunities for further development.

SEE ALSO: Assessment of Pragmatics; Language Assessment Methods; Validation of

Language Assessments
4 oller, john w., jr.

References

Alderson, J. C. (1979). The cloze procedure and proficiency in English as a foreign language.
TESOL Quarterly, 13(2), 219–23.
Bachman, L. F. (1982). The trait structure of cloze test scores. TESOL Quarterly, 16(1), 61–70.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford, England: Oxford
University Press.
Bachman, L. F., Davidson, F., Ryan, K., & Choi, I. C. (1995). An investigation into the comparability
of two tests of English as a foreign language: The Cambridge-TOEFL comparability study. Cambridge,
England: Cambridge University Press.
Bachman, L. F., & Palmer, A. (1981). The construct validation of the FSI oral interview. Language
Learning, 31(1), 67–86.
Bachman, L. F., & Palmer, A. (1982). The construct validation of some components of commu-
nicative proficiency. TESOL Quarterly, 16, 449–65.
Carroll, J. B. (1961). Fundamental considerations in testing for English language proficiency of foreign
language students. Washington, DC: Center for Applied Linguistics.
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor analytic studies. Cambridge, England:
Cambridge University Press.
Chihara, T., Oller, J. W., Jr., Weaver, K., & Chavez-Oller, M. A. (1977). Are cloze items sensitive
to discourse constraints? Language Learning, 27, 63–73.
Cziko, G. A. (1983). Commentary: Another response to Shanahan, Kamil, and Tobin: Further
reasons to keep the cloze case open. Reading Research Quarterly, 18(3), 361–5.
Jonz, J. (1987). Textual cohesion and second language comprehension. Language Learning, 37,
409–38.
Kunnan, A. J. (1995). Test taker characteristics and test performance: A structural modeling approach.
Cambridge, England: Cambridge University Press.
Nunnally, J. C. (1967). Psychometric theory. New York, NY: McGraw-Hill.
Oller, J. W., Jr. (1970). Transformational theory and pragmatics. The Modern Language Journal,
54, 504–7.
Oller, J. W., Jr. (1972). Scoring methods and difficulty levels for cloze tests of proficiency in ESL.
The Modern Language Journal, 56(3), 151–8.
Oller, J. W., Jr. (1975). Pragmatic mappings. Lingua, 35, 333–44.
Oller, J. W., Jr. (1976). Evidence for a general language proficiency factor. Die Neuren Sprachen,
76, 165–74.
Oller, J. W., Jr. (1979). Language tests at school: A pragmatic approach. London, England: Longman.
Oller, J. W., Jr. (Ed.). (1983). Issues in language testing research. Rowley, MA: Newbury House.
Oller, J. W., Jr. (Ed.). (1993). Methods that work: Ideas for literacy and language teachers. Boston,
MA: Heinle.
Oller, J. W., Jr. (2005). Common ground between form and content: The pragmatic solution to
the bootstrapping problem. The Modern Language Journal, 89, 92–114.
Oller, J. W., Jr. (2010). The antithesis of entropy: Biosemiotic communication from genetics to
human language with special emphasis on the immune systems. Entropy, 12, 631–705.
Oller, J. W., Jr., & Chen, L. (2007). Episodic organization in discourse and valid measurement
in the sciences. Journal of Quantitative Linguistics, 14, 127–44.
Oller, J. W., Jr., Chen, L., Oller, S. D., & Pan, N. (2005). Empirical predictions from a general
theory of signs. Discourse Processes, 40(2), 115–44.
Oller, J. W., Jr., & Conrad, C. (1971). The cloze technique and ESL proficiency. Language Learning,
21, 183–95.
Oller, J. W., Jr., & Jonz, J. (Eds.). (1994). Cloze and coherence. Cranbury, NJ: Bucknell University
Press.
Oller, J. W., Jr., Oller, S. D., & Badon, L. C. (2006). Milestones: Normal speech and language develop-
ment across the life span. San Diego, CA: Plural Publishing.
 oller, john w., jr. 5

Oller, J. W., Jr., Oller, S. D., & Badon, L. C. (2010). Cases: Introducing communication disorders
across the life span. San Diego, CA: Plural Publishing.
Oller, J. W., Jr., & Richard-Amato, P. (Eds.). (1983). Methods that work: A smorgasbord of ideas for
language teachers. Rowley, MA: Newbury House.
Oller, J. W., Jr., Robert Scott, J., & Chesarek, S. (1991). Language and bilingualism: More tests of
tests. Cranbury, NJ: Bucknell University Press.
Sawaki, Y., Stricker, L. J., & Oranje, A. H. (2009). Factor structure of the TOEFL Internet-based
test. Language Testing, 26(1), 5–30.
Shanahan, T., Kamil, M. L., & Tobin, A. W. (1982). Cloze as a measure of intersentential com-
prehension. Reading Research Quarterly, 17, 229–55.
Spearman, C. (1904). “General intelligence” objectively determined and measured. American
Journal of Psychology, 15, 201–93.
Stansfield, C. W. (2008). Where we have been and where we should go. Language Testing, 25(3),
311–26.

Suggested Readings

Oller, J. W., Jr. (Ed.). (1989). Language and experience: Classic pragmatism. Lanham, MD: University
Press of America.
Oller, J. W., Jr., & Perkins, K. (Eds.). (1978). Language in education: Testing the tests. Rowley, MA:
Newbury House.
Oller, J. W., Jr., & Perkins, K. (Eds.). (1980). Research in language testing. Rowley, MA: Newbury
House.
Oller, J. W., Jr., & Richards, J. (Eds.). (1973). Focus on the learner: Pragmatic perspectives for the
language teacher. Rowley, MA: Newbury House.