CONDITIONING, CELLULAR AND NETWORK SCHEMES FOR HIGHER-ORDER FEATURES OF CLASSICAL

Experimental work on invertebrates and vertebrates has begun to allow the analysis of the neural mechanisms underlying second-order conditioning, blocking, and contingency. Here we look at the experimental and theoretical data that provide a cellular-and network-level description of higher-order forms of classical conditioning.

Experimental Models of Higher-Order Forms of Classical Conditioning

The development of neural analogs of habituation, sensitization, and classical conditioning have greatly enhanced the understanding of the neural mechanisms responsible for these forms of learning. A logical extension of these studies is to develop neural analogs of higher-order forms of classical conditioning. Research has shown that because many higher-order forms of conditioning are present in invertebrates, these preparations may lead to the establishment of neural analogs of higher-order conditioning. Moreover, recent research in vertebrates has also begun to examine the neural mechanisms of higher-order conditioning.

Second-Order Conditioning

In second-order conditioning, a CS produces a conditioned response not by direct pairing with the US but by pairing with another CS that has previously been paired with the US. The training protocol for second-order conditioning proceeds in two phases. During Phase I, CS1 is paired with the US (resulting in CS1⁺). During Phase II, a novel CS2 is paired with CS1⁺. The defining feature of second-order conditioning is that a conditioned stimulus (CS1⁺) functions as a reinforcing stimulus for the conditioning of a second conditioned stimulus (CS2).

Second-order conditioning has been demonstrated in various invertebrate (e.g., Menzel, 1981; Sahley et al., 1982) and vertebrate preparations (Rescorla, 1980). It has also been observed in the gill-withdrawal reflex in a reduced preparation in Aplysia (Hawkins et al., 1998). In this preparation the mantle organs are isolated together with the abdominal ganglia. Tactile and electrical stimulation of the siphon and gill can be used as CSs and US. In this study extinction of CS1 resulted in extinction of CS2, suggesting the formation of a direct association between both conditioned stimuli. Indeed an important question regarding the underlying mechanisms of second-order conditioning is whether CS1⁺ essentially takes over the system generally activated by the US or whether qualitatively different associations are formed between the CSs. The notion that the same associative forms of synaptic plasticity are involved in both the formation of the CS1-US and CS2-CS1 associations has been corroborated by studies in rat fear conditioning that have shown that the same manipulation that blocks first-order conditioning also blocks second-order conditioning without blocking the conditioned response (Gewirtz and Davis, 1997). It was shown that infusion of an NMDA antagonist (which blocks associative synaptic plasticity) into the amygdala prevented second-order conditioning. This finding supports the notion that second-order conditioning relies on the same forms of synaptic plasticity as classical conditioning.

Blocking

Blocking, along with contingency, establishes that pairing of a CS with a US is not sufficient to produce conditioning. Blocking shows that it is important that the CS contains novel information about the US. A typical blocking protocol (Kamin, 1968) consists of two phases. First CS1 is paired with the US. In Phase II a compound stimulus CS1⁺/CS2 is paired with the US. In the case of complete blocking, CS2 does not undergo any conditioning. The argument is that even though it was explicitly paired with the US, CS2 does not provide any information that was not already predicted by CS1⁺.

Blocking has been described in many vertebrates, in the snail Limax (Sahley et al., 1982) and in bees (Smith and Cobey, 1994). A critical question regarding the mechanisms underlying blocking is, How does preconditioning of CS1 prevent or "block" the conditioning of CS2, even though CS2 is paired with the US? One study on eyeblink conditioning in rabbits has suggested that blocking relies on active inhibition of the US pathway (Kim et al., 1998). Eyeblink conditioning relies on the cerebellum. It was shown that complex spikes in Purkinje cells of the cerebellum (which represent activity in the inferior olive) are elicited by the CS-US pairing in naïve animals but not by CS-US presentations in trained animals. Since inferior olive activity seems to mediate the US, these studies suggest that the US pathway is actively inhibited during Phase II of a blocking protocol. The active suppression of the US response seems to be mediated by the inhibition in the inferior olive produced by the cerebellar output (representing CS1⁺). Indeed, blocking of this inhibition during Phase II of training prevented blocking—that is, it allowed CS2 to undergo conditioning. This manipulation did not prevent first-order conditioning to the CS1 in a separate group of control animals. These experiments suggest that first-order conditioning not only results in the production of a CR but also causes active inhibition of the US pathway; they also suggest that this negative-feedback loop is involved in blocking (see below).

Contingency

Rescorla (1968) suggested that it is not just the number of CS-US pairings but also the correlation between the CS and the US, or the ability of the CS to predict the US, that determines the occurrence of conditioning. He demonstrated that if one group of animals was presented with ten CS-US pairings, and one group was trained with ten extra USs in addition to the ten CS-US pairings (i.e., the probability of a US given a CS equals .5), the latter group displayed less conditioning. Despite both groups having received an equal number of CS-US pairings, the group that received the additional US presentations exhibited less conditioning.

In most invertebrate (Abramson and Bitterman, 1986; Farley, 1987) and vertebrate preparations in which it has been examined, contingency has been observed. However, it is important to distinguish between the different protocols used to study contingency. The extra US presentations can take place before (US preexposure), interspersed with, or after (US postexposure) the CS-US pairings. In Aplysia it has been shown that extra US presentations interspersed with training decreases conditioning (Hawkins et al., 1986). Understanding the mechanisms underlying contingency requires a determination of whether both US pre- and postexposures decrease conditioning. This is because both US pre- and postexposure effects are difficult to explain with the same mechanism. For example, Hawkins and Kandel (1984) suggested that contingency could result from habituation of the US pathway; this phenomenon, however, would not account for postexposure effects. Thus, it is possible that what is often thought of as a single higher-order form of conditioning may actually reflect multiple mechanisms.

Theoretical Models of Higher-Order Forms of Conditioning

Some researchers have proposed numerous mechanistic models that address both the cellular and network mechanisms underlying higher-order forms of conditioning (Hawkins and Kandel, 1984; Gluck and Thompson, 1987; Buonomano et al., 1990). Many of these models share common conceptual features that we will examine with respect to a model of second-order conditioning and blocking originally developed in the context of the Aplysia siphon-withdrawal reflex (Buonomano et al., 1990).

Consider two CS pathways, CS1 and CS2, each represented by a sensory neuron (SN1 and SN2), a US pathway represented modulatory or facilitatory neuron (FN), and a single output unit that generates both the CR and UR, represented by a motor neuron (MN). An important aspect of the network is that the CS neurons not only connect to the MN but also to the FN (see Figure 1). During first-order conditioning in which the SN1 and FN are paired (for example with a 100-ms interval), the facilitatory neuron will produce associative plasticity at both the SN1→MN and SN1→FN synapse.

Second-Order Conditioning

It is easy to see how this circuit is intrinsically able to account for second-order conditioning. Since plasticity is also occurring at the SN1→FN synapse, CS1 can take over the FN and become a US. When CS2 and CS1 are paired, CS1 will activate the FN and result in conditioning of CS2. In this model, second-order conditioning results directly from associative synaptic plasticity occurring at two synapses. Plasticity at the SN1→MN synapse is responsible for the generation of the conditioned response, whereas plasticity at the SN1→FN synapses accounts for second-order conditioning. One important prediction of this type of model is that it should be possible to block second-order conditioning without interfering with first-order conditioning by specifically blocking plasticity at the SN1→FN synapse.

Blocking

Like second-order conditioning, blocking could arise from the ability of a previously conditioned CS1⁺ (SN1⁺) to "take control" of the FN. As in second-order conditioning, during Phase I of training, the synaptic strength of SN1 would increase and become strong enough to activate the FN. There are two consequences of this CS1-induced activity in the FN that would contribute to blocking. Note that if SN1⁺ effectively activates the FN (and activity in the FN undergoes rapid depression), during the Phase II of blocking in which CS1⁺/CS2 are paired with the US, two important things happen: First, the FN will be activated approximately simultaneously with the onset of CS1⁺/CS2, essentially resulting in a 0-ms interstimulus interval that should not result in conditioning of CS2. Second, since the FN undergoes depression, the output of the FN in response to a US that followed CS1/CS2 would be small or zero and would not support associative conditioning of SN2. One subtle aspect of blocking pertains to the question of extinction of CS1⁺: if CS2 does not undergo conditioning because the US in ineffective, why doesn't CS1⁺ undergo extinction? Plasticity in the Aplysia sensory neurons predicts that the effective interstimulus interval function of plasticity will be state-dependent. That is, while a nonconditioned SN may not exhibit plasticity with a 0-ms ISI, a conditioned SN can exhibit plasticity at 0 ms and thus prevent extinction of CS1⁺ during Phase II of blocking.

Therefore, in this model, although CS2 would be paired with the US during Phase II of training, prior conditioning of CS1 would block associative enhancement of SN2. Thus, blocking could be supported by activity-dependent neuromodulation (the mechanism of classical conditioning) in combination with accommodation or synaptic depression.

Conclusion

The results of both empirical and theoretical studies indicate that at least two factors determine the ability of neural circuits to support various features of learning: the learning rules or forms of synaptic plasticity and the connectivity of the circuitry in which those forms of plasticity are embedded. Therefore, complex forms of learning may differ from simple forms of learning in either the form(s) of cellular plasticity involved and/or in the circuitry. However, the studies described above indicate that when the same forms of synaptic plasticity responsible for simpler forms of associative learning are embedded in more complex networks, more complex forms of learning may emerge, including blocking and second-order conditioning.

See also:APLYSIA: CLASSICAL CONDITIONING AND OPERANT CONDITIONING; INVERTEBRATE LEARNING: ASSOCIATIVE LEARNING AND MEMORY PROCESSING IN BEES; INVERTEBRATE LEARNING: ASSOCIATIVE LEARNING IN LIMAX

Bibliography

Buonomano, D. V., Baxter, D. A., and Byrne, J. H. (1990). Small networks of empirically derived adaptive elements simulate higher-order features of classical conditioning. Neural Networks 3, 507-523.

Gewirtz, J. C., and Davis, M., (1997) Second-order fear conditioning prevented by blocking NMDA receptors in amygdala. Nature 368, 471-474.

Gluck, M. A., and Thompson, R. F. (1987). Modeling the neural substrates of associative learning and memory, A computational approach. Psychological Review 94, 176-191.

Hawkins, R. D., Carew, T. J., and Kandel, E. R. (1986). Effects of interstimulus interval and contingency on classical conditioning of the Aplysia siphon withdrawal reflex. Journal of Neuroscience 6, 1,695-1,701.

Hawkins, R. D., Cohen, T. E., Greene, W., and Kandel, E. R.(1998). Classical conditioning, differential conditioning, and second-order conditioning in the Aplysia gill-withdrawal reflex in a simplified mangle organ preparation. Behavioral Neuroscience 112, 636-645.

Hawkins, R. D., and Kandel, E. R. (1984). Is there a cell-biological alphabet for simple forms of learning? Psychological Review 91, 376-391.

Kim, J. J., Krupa, D. J., and Thompson, R. F. (1988). Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science 279, 570-573.

Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology 66, 1-5.

—— (1980). Pavlovian second-order conditioning, studies in associative learning. New York: Erlbaum.

Smith, B. H., and Boey, S. (1994). The olfactory memory of the honeybee Apis mellifera II Blocking between odorants in binary mixtures. Journal of Experimental Biology 195, 91-108.

Jennifer L.Raymond

John H.Byrne

Revised byDean V.Buonomano