CONNECTIONISM

"Connectionism" is an approach within cognitive science that employs neural networks, rather than computer programs, as the basis for modeling mentality. A connectionist system, or neural network, is a structure of simple neuronlike processors called nodes or units. Each node has directed connections to other nodes, so that the nodes send and receive excitatory and inhibitory signals to and from one another. The total input to a node determines its state of activation. When a node is on, it sends out signals to the nodes to which it has output connections, with the intensity of a signal depending upon both (1) the activation level of the sending node and (2) the strength or "weight" of the connection between it and the receiving node. Typically at each moment during processing, many nodes are simultaneously sending signals to others.

When neural networks are employed for information processing, certain nodes are designated "input" units and others as "output" units, and potential patterns of activation across them are assigned interpretations. (The remaining nodes are called "hidden units.") Typically a "problem" is posed to a network by activating a pattern in the input nodes; then the various nodes in the system simultaneously send and receive signals repeatedly until the system settles into a stable configuration; the semantic interpretation of the resulting pattern in the output nodes is what the system currently represents, hence its "answer" to the problem. Connectionist systems are capable of "learning" from "experience" by having their weights changed systematically in a way that depends upon how well the network has performed in generating solutions to problems posed to it as a training regimen. (Typically the device employed is not an actual neural network but a simulation of one on a standard digital computer.)

The most striking difference between such networks and conventional computers is the lack of an executive component. In a conventional computer the behavior of the whole system is controlled at the central processing unit (CPU) by a stored program. A connectionist system lacks both a CPU and a stored program. Nevertheless, often in a connectionist system certain activation patterns over sets of hidden units can be interpreted as internal representations with interesting content, and often the system also can be interpreted as embodying, in its weighted connections, information that gets automatically accommodated during processing without getting explicitly represented via activation patterns.

Connectionist models have yielded particularly encouraging results for cognitive processes such as learning, pattern recognition, and so-called multiple-soft-constraint satisfaction (i.e., solving a problem governed by several constraints, where an optimal solution may require violating some constraints in order to satisfy others). For example, Terry Sejnowski and Charles Rosenberg trained a network they called NETtalk to convert inputs that represent a sequence of letters, spaces, and punctuation constituting written English into outputs that represent the audible sounds constituting the corresponding spoken English. (The phonetic output code then can be fed into a speech synthesizer, a device that actually produces the sounds.)

Philosophical discussion of connectionism has largely centered on whether connectionism yields or suggests a conception of mentality that is importantly different from the conception of mind-as-computer at the core of classical cognitive science. Several different nonclassical alternatives have been suggested; each has been alleged to fit well with connectionism, and each has been a locus of debate between fans and foes of connectionism. Three proposed interpretations of connectionism deserve specific mention.

On one view, the key difference between classical models of mental processes and connectionist models is that the former assume the existence of languagelike mental representations that constitute a so-called language of thought (LOT), whereas the latter supposedly favor representations that are alleged to be inherently non-languagelike in structure: namely, activation patterns distributed over several nodes of a network, so-called activation vectors. On this interpretation connectionism shares with classicism the assumption that cognition is computation over mental representations—that cognitive transitions conform to rules for transforming representations on the basis of their formal structure, rules that could be formulated as an explicit computer program. (In connectionist systems the rules are wired into the weights and connections rather than being explicitly represented. In classical systems some rules must be hard wired; and there may be—but need not be—other rules that are explicitly represented as stored data structures.) The key difference allegedly turns on the languagelike or non-languagelike structure of mental representations.

This construal of connectionism fits naturally with the idea that human cognition involves state transitions that are all essentially associative—in the sense that they reflect statistical correlations among items the system can represent and can be analyzed as the drawing of statistical inferences. Many fans of connectionism, including Patricia Churchland and Paul Churchland, evidently see things this way and tend to regard connectionism as breathing new life into associationism. Prominent foes of connectionism, notably Jerry Fodor and Zenon Pylyshyn, also see things this way; but they regard the link with associationism as grounds for maintaining that connectionism is bound to founder on the same general problem that plagued traditional associationism in psychology: namely, inability to account for the rich semantic coherence of much human thought. To overcome this problem, Fodor and Pylyshyn maintain, cognitive science must continue to posit both (1) mental representations that encode propositional information via languagelike syntactic structure and (2) modes of processing that are suitably sensitive to syntactic structure and are thereby sensitive to propositional content.

A second interpretation of connectionism claims that connectionist models do not really employ internal representations at all in their hidden units (and, a fortiori, do not employ internal representations with languagelike structure). This view has been defended—by Rodney Brooks, for example—on the grounds that putative representations in connectionist systems play no genuine explanatory role. It has also been defended—for instance, by Hubert Dreyfus and Stuart Dreyfus—on the basis of a Heideggerian critique of the notion of mental representation itself. The approach goes contrary to the views of most (but not all) practicing connectionists, who typically posit internal representations in connectionist models and assign them a central explanatory role.

A third interpretation assumes the existence of internal mental representations; and it does not deny—indeed, the version defended by Terence Horgan and John Tienson resolutely affirms—that mental representations often have languagelike structure. It focuses instead on the classical assumption that cognition is computation (see above). This third approach maintains (1) that much of human cognition is too rich and too subtle to conform to programmable rules and (2) that connectionism has theoretical resources for potentially explaining such nonalgorithmic cognitive processing. The approach stresses that there is a powerful branch of mathematics that applies naturally to neural networks: dynamical systems theory. According to this anticomputational construal of connectionism, there can be cognitive systems—subservable mathematically by dynamical systems, which in turn are subservable physically by neural networks—whose cognitive state transitions are not tractably computable. In other words, mental activity in these systems is too refined and too supple to conform to programmable rules. Humans are alleged to be such cognitive systems, and connectionism (so interpreted) is held to yield a more adequate picture of the mind than the classical computational picture.

One objection to this third interpretation of connectionism alleges that cognitive state transitions in a connectionist system must inevitably conform to programmable rules, especially since neural networks are simulable on standard computers. Another objection, directed specifically at the version that retains languagelike representations, alleges that the LOT hypothesis is intelligible only on the assumption that cognition is computation.

In much of the early philosophical debate between proponents and opponents of connectionism, the first interpretation was largely taken for granted. But as competing interpretations get articulated, defended, and acknowledged, philosophical discussion of connectionism and its potential implications becomes richer.

See also Cognitive Science; Fodor, Jerry A.; Language of Thought; Philosophy of Mind.

Bibliography

Aizawa, K. "Explaining Systematicity." Mind and Language 12 (1997): 115–136.

Bechtel, W., and A. Abrahamsen. Connectionism and the Mind: Parallel Processing, Dynamics, and Evolution in Networks. 2nd ed. Oxford: Blackwell, 2001.

Churchland, P. M. "Conceptual Similarity across Sensory and Neural Diversity: The Fodor/Lepore Challenge Answered." Journal of Philosophy 95 (1998): 5–32.

Clark, A., and P. Millican, eds. Connectionism, Concepts, and Folk Psychology. Oxford: Oxford University Press, 1996.

Cummins, R. "Systematicity." Journal of Philosophy 93 (1996): 561–614

Dawson, M. Minds and Machines: Connectionism and Psychological Modeling. Oxford: Blackwell, 2004.

Horgan, T., and J. Tienson. Connectionism and the Philosophy of Psychology. Cambridge, MA.: MIT Press, 1996.

Macdonald, C., ed. Connectionism: Debates on Psychological Explanation. Oxford: Blackwell, 1995.

Marcus, G. The Algebraic Mind. Cambridge, MA: MIT Press, 2001.

McLeod, P., K. Plunkett, and E. T. Rolls. Introduction to Connectionist Modelling of Cognitive Processes. Oxford: Oxford University Press, 1998.

Tomberlin, J., ed. Philosophical Perspectives 9: AI, Connectionism and Philosophical Psychology. Atascadero: Ridgeview Press, 1995.

Tuescher, C. Turing's Connectionism. An Investigation of Neural Network Architectures. London: Springer-Verlag, 2002.

Terence E. Horgan (1996)

Bibliography updated by Alyssa Ney (2005)