A New Interpretation of the Turing Test

Diane Proudfoot

 

The Turing Test and the Orthodox Interpretation

In ‘Computing Machinery and Intelligence’ (1950) Turing described an ‘imitation game’ played by three people, an interrogator and two interviewees, one male (A) and one female (B). The interrogator communicates with A and B from a separate room (by means of printed text and nowadays probably using a keyboard and screen); apart from this the three players have no contact with each other. The interrogator’s task is to find out, by asking questions, which of A and B is the man. A’s aim is that the interrogator decide wrongly. (Turing said, ‘The object of the game for the third player (B) is to help the interrogator. The best strategy for her is probably to give truthful answers.’ (1950, p. 434).)

In a second version of the game, a computer takes the part of A and a male or female the part of B.¹ Now the interrogator’s task is to discover which of A and B is the computer; to do so he or she is permitted to ask any question, on any topic. The computer is allowed to do everything possible so that the interrogator makes the wrong identification. Turing said that the question ‘Can machines think?’ is ‘too meaningless to deserve discussion’ and proposed replacing it by the question ‘Are there imaginable digital computers which would do well in the [computer-imitates-human] game?’ (ibid., p. 442). The upshot of his famous proposal is: if some computer does well in the imitation game, the answer to (an appropriate substitute for) the question ‘Can machines think?’ is ‘yes’. Hence Turing offers a test of intelligence in machines.

A Definition of Intelligence?

On the orthodox interpretation of (what is now known as) the Turing test, the computer-imitates-human game yields an operational definition of thinking: a machine is intelligent (or thinks) if and only if its behaviour in the game matches that of a human being. For example, French states that Turing’s ‘philosophical claim translates elegantly into an operational definition of intelligence: what acts sufficiently intelligent is intelligent’ (1996, p. 12; see also French 2000). Hodges, Turing’s biographer, states that Turing’s ‘operational definition of “thinking” … was nothing new—it was an entirely standard line of rationalist thought’ (1992, p. 266).

The orthodox interpretation is fundamentally flawed. First, as Moor (1976, 1987) and Copeland (1993, 2003) have pointed out, Turing did not present the imitation game as part of any definition of (i.e. logically necessary and sufficient conditions for) intelligence or thinking. Copeland (2003) reveals that, on the contrary, in his 1952 BBC radio broadcast ‘Can Automatic Calculating Machines Be Said to Think?’ Turing said ‘I don’t want to give a definition of thinking, but if I had to I should probably be unable to say anything more about it than that it was a sort of buzzing that went on inside my head. But I don’t really see that we need to agree on a definition at all.’ (Turing et al. 1952, p. 494).

If Turing was not offering a definition of intelligence in machines, how are we to take his test? Moor claims that

[a] plausible interpretation of the imitation game is to regard it as an inductive test … If a machine passed a rigorous Turing test, then we … would have sufficient good evidence to infer that the machine was intelligent. (2003a, p. 202; emphasis added)

This interpretation has the advantage that outcomes of the test are defeasible: for example, we would not be forced by a machine’s impressive performance in one game to ignore its poor performances in later games.

Should we accept Moor’s interpretation? He points out this attraction: on the inductive reading we need not conclude that a device such as Block’s Jukebox (Block 1981), which does well in the game but only by utilizing a complex look-up table, is intelligent (Moor 2003a, p. 202). However, any interpretation that does not make a machine’s satisfactory performance in an imitation game logically sufficient for intelligence has this advantage. In addition, it is notable that Turing himself provided a very different reply to the challenge (to his test) posed by hypothetical devices such as Block’s. Copeland shows that Turing’s remarks in the 1952 broadcast suggest the following reply: given the practical limitations on storage capacity, we simply could not build a machine to imitate human intellectual behaviour that works by means of a look-up table—and even if we could, such a machine would take many years rather than minutes to answer the interrogator’s questions (Turing et al. 1952; Copeland 2003, p. 15).

Moor also claims that, on his interpretation, Turing’s test offers a scientific method of gathering evidence for the existence of intelligence in machines (2003a, p. 203). However, Turing was concerned, not only with the evidence for intelligence, but with the nature of intelligence. The inductive interpretation fails to do justice to Turing’s discussions of the latter. On Moor’s reading, all the test tells us about intelligence itself is that it is some property the possession of which is indicated by success in the imitation game.

The Philosophical Motivation for the Test

Why did Turing link performance in an imitation game to intelligence? The second fundamental mistake in the orthodox interpretation of the Turing test is that it takes Turing’s philosophical motivation to be a version of behaviourism or operationalism—i.e., some variant on the thesis that an entity’s psychological properties are determined by the entity’s behaviour. There is no textual basis in Turing’s 1950 paper for this interpretation. Moreover, Turing’s remarks in an earlier presentation of the test falsify a behaviourist reading.

There are two, rarely discussed, versions of the imitation game outside Turing’s famous 1950 paper. The latest version is in ‘Can Automatic Calculating Machines Be Said To Think?’ (Turing et al. 1952; see Copeland 2003). The first and—if we are to understand Turing’s motivation—most significant version is the chess-playing experiment presented in ‘Intelligent Machinery’, Turing’s 1948 report for the National Physical Laboratory. (This report was AI’s first manifesto; see Copeland 2004, Copeland and Proudfoot 2005.) In ‘Intelligent Machinery’ Turing said:

The extent to which we regard something as behaving in an intelligent manner is determined as much by our own state of mind and training as by the properties of the object under consideration. If we are able to explain and predict its behaviour or if there seems to be little underlying plan, we have little temptation to imagine intelligence. With the same object therefore it is possible that one man would consider it as intelligent and another would not; the second man would have found out the rules of its behaviour.

It is possible to do a little experiment on these lines, even at the present state of knowledge. It is not difficult to devise a paper machine which will play a not very bad game of chess. Now get three men as subjects for the experiment A, B, C. A and C are to be rather poor chess players, B is the operator who works the paper machine. (In order that he should be able to work it fairly fast, it is advisable that he be both mathematician and chess player.) Two rooms are used with some arrangement for communicating moves, and a game is played between C and either A or the paper machine. C may find it quite difficult to tell which he is playing.

(This is a rather idealized form of an experiment I have actually done.) (1948, p. 431)

A ‘paper machine’ is a simulation of machine behaviour by a human being using paper and pencil—the only sort of computer freely available in 1948.

These paragraphs from ‘Intelligent Machinery’ are almost invariably overlooked in the extensive literature on the motivation and justification of the Turing test.² Yet here is the first (restricted) version of the computer-imitates-human game. Turing’s famous 1950 presentation develops the experiment in his 1948 report. Both papers begin by proposing to investigate the question whether it is possible for machines to exhibit ‘intelligent behaviour’ (in the 1948 paper) or to ‘think’ (in the 1950 paper). In each case Turing constructed and answered objections to this possibility, the objections in the 1948 paper forming a subset of those in the 1950 paper. The computer-imitates-human game of the 1950 article is simply a version of the 1948 experiment that is unrestricted with respect to which questions may be put to the contestants.

Turing’s discussion of the first version of the game makes clear the philosophical idea behind the imitation game. In ‘Intelligent Machinery’ Turing said that intelligence is an ‘emotional concept’ (1948, p. 431), i.e. one the application of which is ‘determined as much by our own state of mind and training as by the properties of the object under consideration’. This is an externalist stance. Externalism in the philosophy of mind is the thesis that an entity’s psychological states (or their representational contents) are individuated at least partly in terms of the entity’s history or (social or physical) environment. In Turing’s view, whether an entity (real or artificial) is intelligent is determined, at least in part, by our responses to the entity’s behaviour.

A New Interpretation

Turing’s test is externalist; whether or not machines think is in part determined by social environment, in the form of the interrogator’s responses. (This is not the trivial truth that a machine contestant cannot win an imitation game unless judged to win.) First, the goal for the (programmer of the) machine contestant in an imitation game is that the interrogator respond by misclassifying the machine as the human contestant—the interrogator ‘must be taken in by the [machine’s] pretence’ (Turing et al. 1952, p. 495). Second, how well the machine does depends upon responses outside the (computer-imitates-human) game: the machine’s performance is satisfactory if the interrogator in the computer-imitates-human game is fooled no less often than the interrogator in the man-imitates-woman game (Turing 1950, p. 434; Copeland 2003).

The orthodox interpretation’s claim that Turing’s motivation is behaviourist is wrong-headed. Turing’s discussion of intelligence in the 1948 report makes plain his externalist ‘criterion for “thinking”’ (1950, p. 443) in the case of machines: an interrogator in an unrestricted computer-imitates-human game ‘imagines’ (the term in the 1948 report) that the machine is an intelligent human being. Externalism resembles operationalism and behaviourism only insofar as all three deny internalism (the thesis that an entity’s psychological states, or their representational contents, are individuated purely in terms of the entity’s current internal states). Otherwise it is very different.

Moor’s alternative interpretation also misconstrues the motivation for the test. On his reading, the test is internalist, providing good inductive evidence for a hypothesis concerning the machine’s ‘inner information processes’ (1976, p. 251). This interpretation is incompatible with Turing’s explicit assertions in the 1948 report. If intelligence is an emotional concept, then there is no (observer-independent) fact of the matter that an entity is intelligent, with respect to which we can gather weak or strong inductive evidence. And if thinking does not consist in ‘properties of the object under consideration’, then it does not consist in the machine’s inner processes.

The philosophical motivation for Turing’s test has been consistently misunderstood. Attention to the 1948 restricted version of the imitation game and to Turing’s claim that intelligence is an emotional concept rectifies this misunderstanding.

 

References

Block, N. (1981) Psychologism and Behaviorism. Philosophical Review 90: 5-43.

Copeland, B.J. (1993) Artificial Intelligence: a Philosophical Introduction. Oxford: Blackwell.

Copeland, B.J. (2003) The Turing Test. In: Moor 2003b.

Copeland, B.J. (ed.) (2004) The Essential Turing. Oxford University Press.

Copeland, B.J. and Proudfoot, D. (2005) Turing and the Computer. In: Alan Turing’s Automatic Computing Engine, ed. B.J. Copeland. Oxford University Press.

French, R.M. (1996) Subcognition and the Limits of the Turing Test. In: Machines and Thought: The Legacy of Alan Turing, Volume 1, eds P. Millican and A. Clark. Oxford University Press.

French, R.M. (2000) The Turing Test: the First 50 Years. Trends in Cognitive Science 4(3): 115-122.

Hodges, A. (1992) Alan Turing: The Enigma. London: Vintage.

Moor, J.H. (1976) An Analysis of the Turing Test. Philosophical Studies 30: 249-257.

Moor, J.H. (1987) Turing Test. In: Encyclopedia of Artificial Intelligence, Vol. 2, ed. S.C. Shapiro. New York: John Wiley and Sons.

Moor, J.H. (2003a) The Status and Future of the Turing Test. In: Moor 2003b.

Moor, J.H. (ed.) (2003b) The Turing Test: The Elusive Standard of Artificial Intelligence. Dordrecht: Kluwer.

Piccinini, G. (2003) Turing’s Rules for the Imitation Game. In: Moor 2003b.

Sterrett, S.G. (2003) Turing’s Two Tests for Intelligence. In: Moor 2003b.

Traiger, S. (2003) Making the Right Identification in the Turing Test. In: Moor 2003b.

Turing, A.M. (1948) Intelligent Machinery. National Physical Laboratory Report. Reproduced in Copeland 2004; page references are to Copeland 2004.

Turing, A.M. (1950) Computing Machinery and Intelligence. Mind 59: 433-460.

Turing, A.M., Braithwaite, R., Jefferson, G., and Newman, M. (1952) Can Automatic Calculating Machines Be Said to Think? A BBC radio broadcast. Reproduced in Copeland 2004; page references are to Copeland 2004.

Notes