A parallel distributed processing model of Wason s selection task

Transcription

1 Journl of Cognitive Systems Reserch 2 (2001) locte/ cogsys A prllel distributed processing model of Wson s selection tsk Action editor: Steve J. Hnson Jcqueline P. Leighton *, Michel R.W. Dwson, b Centre for Reserch in Applied Mesurement nd Evlution, Fculty of Eduction, Eduction North Building, University of Albert, Edmonton, Albert, Cnd T6G 2G5 b Biologicl Computtion Project, Deprtment of Psychology, University of Albert, Edmonton, Albert, Cnd T6G 2E9 Received 1 September 2000; received in revised form 10 Mrch 2001; ccepted 17 June 2001 Abstrct Architecturl ccounts of cognitive performnce re importnt to explore becuse they provide the infrstructure for lgorithmic theories of cognition [Dwson, M.R.W. (1998). Understnding cognitive science. Mlden, MA: Blckwell]. Three prllel distributed processing (PDP) networks were trined to generte the p, the p nd not-q nd the p nd q responses, respectively, to the conditionl rule used in Wson s selection tsk [Wson, P.C. (1966). Resoning. In: Foss, B.M. (Ed.), New Horizons in Psychology, London, Penguin]. Afterwrd, ech trined network ws nlyzed for the lgorithm it developed to lern the desired response to the tsk. Anlyses of ech network s solution to the tsk suggested specilized lgorithm tht focused on crd loction. For exmple, if the desired response to the tsk ws found t crd 1, then specific set of hidden units detected the response. In ddition, we did not find support tht selecting the p nd q response is less difficult thn selecting the p nd not-q response. Humn studies of the selection tsk usully find tht prticipnts fil to generte the ltter response, wheres most esily generte the former. We discuss how our findings cn be used to () extend our understnding of selection tsk performnce, (b) understnd existing lgorithmic theories of selection tsk performnce, nd (c) generte new venues of study of the selection tsk Elsevier Science B.V. All rights reserved. 1. Introduction then Q nd four crds displying instnces of p, not-p, q, nd not-q. In the ctul tsk, ech crd No other tsk hs spurred s much reserch into contins letter on one side nd number on the humn resoning s hs Wson s (1966) selection other side. Prticipnts re instructed to test the truth tsk (Evns, Newsted, & Byrne, 1993). Fig. 1 or flsity of the rule by selecting the fewest crds illustrtes the tsk, which involves presenting possible from the set of four. Although prticipnts prticipnt with conditionl rule in the form of If P cn see only side of ech crd, they re told tht ech crd s flip side contins informtion tht might be useful in testing the rule. *Corresponding uthor. Tel.: ; fx: According to propositionl logic, only the piring E-mil ddress: jcqueline.leighton@ulbert.c (J.P. Leighton). of the p (i.e., the ntecedent) with the not-q (i.e., the negtion of the consequent) flsifies nd, there / 01/ $ see front mtter 2001 Elsevier Science B.V. All rights reserved. PII: S (01)

2 208 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) models theory (1983; Johnson-Lird & Byrne, 1991) proposes tht ny context tht impirs prticipnts bility to consider counter-exmples to the selection tsk s rule will hinder logicl performnce. Finlly, Rips (1994) syntctic theory proposes tht prticipnts fil to respond correctly to the selection tsk becuse the tsk clls for sophisticted ppliction of mentl rules tht mny prticipnts either do not hve or hve not yet mstered. One similrity mong Fig. 1. Wson s crd selection tsk. the first three theories prgmtic resoning theory, socil contrct theory, nd mentl models is tht fore, conclusively tests the rule (Grnhm & Okhill, ech provides n lgorithmic ccount of perform- 1994). Typiclly only 10% of prticipnts select nce; tht is, ech theory specifies procedurl crds corresponding to p nd not-q, however. description of how prticipnts solve the selection Most prticipnts select either the p lone or the p tsk. Rips (1994) syntctic theory lso provides n nd the q (for complete review of the selection lgorithmic ccount but, in ddition, the theory tsk, the reder is referred to Evns et l., 1993). specifies n rchitecturl ccount of prticipnts Why do so mny prticipnts fil to respond performnce system of productions tht implelogiclly to the selection tsk? This is question tht ments logicl routines. hs spurred numerous studies nd host of theories Rips (1994) theory suggests tht system of (e.g., Cheng & Holyok, 1985, 1989; Cosmides, productions underlies resoning, but some critics 1989; Johnson-Lird & Byrne, 1991; Rips, 1994). hve rgued tht his theory is unconvincing becuse Unfortuntely, mny of these theories re lgorith- it fils to reflect the inductive qulity of humn mic nd do not ddress the question of the kind of resoning (e.g., Oksford & Chter, 1993). For rchitecture tht underlies performnce. Specifying exmple, unlike prgmtic resoning theory, socil the rchitecture of cognitive performnce is useful contrct theory, nd mentl models theory, syntctic becuse it nchors lgorithmic theories to more theory in generl ignores the role of context in concrete descriptions of performnce; blck box resoning, nd the non-monotonicity of resoning descriptions re precluded (Dwson, 1998). (Byrne, 1989; Evns et l., 1993). According to Cheng nd Holyok (1985, 1989) ccount for some theorists, other rchitectures might offer more prticipnts poor performnce with prgmtic credible ccounts of resoning (e.g., Oksford & resoning theory. According to the theory, prticip- Chter, 1993). In prticulr, connectionist rchitecnts possess domin-specific schemt tht, invoked ture might be more representtive of humn resonin meningful contexts, fcilitte resoning re- ing, s Shstri (1991) explins: sponses. Cheng nd Holyok (1985, 1989) suggest tht prticipnts perform poorly on the selection tsk Connectionism offers n extremely efficient becuse it lcks meningful context. In support, metphor for resoning where inference is rethey hve empiriclly shown tht logicl responses to duced to spreding ctivtion in prllel the selection tsk increse when the tsk is frmed in network...the connectionist pproch suggests meningful context, such s when prticipnts must lternte formultions of informtion processing. decide which person, mong four, stnds in violtion Thus insted of viewing knowledge-bsed sysof permission rule (see Libermn nd Klr, 1996, tem s theorem prover or production system, for discussion of how the permission context my one my view it s system tht performs chnge the nture of the tsk). Cosmides (1989) constrint stisfction, energy minimiztion, or socil contrct theory proposes similr ccount of evidentil nd probbilistic resoning ( p. 263). prticipnts poor performnce, lthough the dominspecific schemt in this theory re invoked spe- Specifying the rchitecture of n lgorithmic cificlly in response to situtions involving costs nd ccount of cognitive performnce is importnt bebenefits. Alterntively, Johnson-Lird s mentl cuse it provides n ccount of the mentl progrm-

3 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) ming lnguge in which cognitive lgorithms re PDP networks to extend our understnding of selecwritten (Dwson, 1998, p. 159). Such n ccount tion tsk performnce. First, PDP networks lern to serves to clrify theories nd to extend our under- solve tsks by mens of pttern clssifiction or stnding of the cognitive performnce under study. mpping input ptterns to output responses. This For instnce, we cn pursue questions such s how method of lerning to solve tsks is comptible with might domin-specific schemt be instntited in mny of the lgorithmic theories of selection tsk the brin? or re rules or some other opertion performnce, such s prgmtic resoning theory underlying performnce? To begin nswering such (Cheng & Holyok, 1985). In fct, pttern clssificquestions, we need to explore rchitecturl ccounts tion ccounts of resoning in generl hve been of resoning longside lgorithmic ccounts. Explor- proposed (e.g., Bechtel & Abrhmsen, 1991; Gobet ing functionl rchitecturl ccounts of cognitive & Simon, 1998; Goldstone & Brslou, 1998). performnce cn bring us closer to more complete Pttern clssifiction ccounts of resoning ssume theories of cognition. tht people mke sense of their environment by In this pper we explore whether specific ctegorizing objects nd events not only to mke connectionist or prllel distributed processing predictions bout their (unseen) chrcteristics, but (PDP) rchitecture, termed the vlue unit rchitec- lso to decide upon ctions in light of the ctegorizture, cn extend our understnding of prticipnts tion. For exmple, Bechtel nd Abrhmsen (1991), performnce on the selection tsk. Hence, we ttempt reiterting n ide proposed by Mrgolis (1987), to resolve the critique we levy ginst theories of suggest the following view of how resoning might selection tsk performnce; nmely, tht they re not proceed ccording to the pttern clssifiction view: linked to functionl rchitecture. This reserch is explortory since to our knowledge no one hs The recognition of one pttern constitutes n simulted performnce on the selection tsk using internl cue which, together with the externl connectionist rchitecture. Other investigtors hve cues vilble from outside the system, fcilittes used connectionist rchitectures to model other kinds yet nother recognition. Thus, we work our wy of (inferentil) problems but not specificlly the through complex problem by recognizing someselection tsk (e.g., Derthick, 1991; Shstri, 1991). thing, nd with the help of tht result, recognizing As generl overview of the pper, we first something further. (p. 141) discuss why PDP networks cn be used to explore n rchitecturl ccount of selection tsk performnce. Viewing resoning from the perspective of pttern Second, we provide generl introduction to PDP clssifiction complements theories such s Cheng networks nd in prticulr the vlue unit rchitecture, nd Holyok s prgmtic resoning theory (1985) which is the specific rchitecture we use in the nd Cosmides socil contrct theory (1989), which present studies. Third, we illustrte how three differ- emphsize the inductive qulity of resoning. ent vlue unit networks were trined to generte Another importnt reson for employing PDP different responses to the selection tsk nd how networks is tht they chrcterize, lbeit roughly, the ech network cn help us understnd prticipnts kind of processing tht occurs in the brin (Dwson, responses to this tsk. Finlly, we discuss the results 1998). PDP networks re considered brin-like from ll three networks nd their reltion to existing systems in tht they re built from inter-connected, lgorithmic ccounts of performnce on the selection simple processing units tht cn be used to clssify tsk. ptterns. We turn now to description of PDP networks. 2. Why use PDP networks to explore n rchitecturl ccount of performnce on the 2.2. A PDP network of vlue units selection tsk? A PDP network is system of inter-connected, 2.1. Pttern clssifiction simple processing units tht cn be used to clssify ptterns presented to it. A PDP network is usully There re two resons why we would wnt to use mde up of three kinds of processing units: () Input

4 210 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) units encode the stimulus or ctivity pttern tht the units, nd either mplifies or ttenutes the signl network will eventully clssify; (b) hidden units being sent from one processing unit to nother. detect fetures or regulrities in the input ptterns A network is not given step by step procedure tht cn be used to determine clssifiction decisions; for solving desired tsk, but, is insted trined to nd (c) output units represent the network s response solve the tsk. Consider populr supervised lernto the input pttern; tht is, the ctegory to which the ing procedure clled the generlized delt rule pttern is to be ssigned on the bsis of the fetures (Rumelhrt, Hinton & Willims, 1986). To trin or regulrities tht hve been detected by the hidden network with this rule, one strts with network (of units. Processing units communicte by mens of pre-specified number of hidden units) tht hs weighted connections. Fig. 2 provides n illustrtion smll, rndomly ssigned connection weights. The of typicl network. network is then developed by presenting it set of In most cses, processing unit crries out three trining ptterns, ech of which is ssocited with centrl functions: First, processing unit computes desired response. To trin network to clssify the net input or the totl signl tht it receives from pttern correctly, pttern is presented to the netother units. A net input function is used to crry out work s input units, nd the network genertes this clcultion. After the processing unit determines response to this stimulus using its existing conits net input, it trnsforms it into n internl level of nection weights. The network s response is then ctivity, which typiclly rnges between 0 nd 1. The compred ginst the desired output (i.e., the correct internl ctivity level is clculted by mens of n response) nd n error vlue is computed for ech of ctivtion function. Finlly, the processing unit the network s output units. This error vlue is then determines the signl tht needs to be sent to other fed bckwrds through the network, nd it is used to units. This signl is creted by pplying n output modify connection weights in such wy tht the function to the unit s internl ctivity. The most next time the pttern is presented to the network, the common output function is the identity function, network s output errors will be smller. By repeting suggesting tht the signl sent out from unit equls this procedure lrge number of times for ech the unit s internl ctivity. (The reder is referred to pttern in the trining set, the network s response Dwson (1998) for more complete expliction of errors for ech pttern cn be reduced to ner zero. the different functions.) A weighted connection cts At the end of this procedure, the network will hve s communiction chnnel between two processing very specific pttern of connectivity (in comprison Fig. 2. Illustrtion of typicl PDP network, including lyer of input units, hidden units, nd output units.

5 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) to its rndom strt) nd will hve lerned to perform the decision regions nd led to the network s correct the desired stimulus/ response piring (if it is pos- responses? sible for such piring to be lerned). At first glnce it might pper uninteresting to A number of different versions of the generlized investigte whether PDP network lerns the desired delt rule exist, ech designed to trin networks input/ output response to the selection tsk since we whose processors hve specific properties. For in- know tht PDP networks, s universl Turing mstnce, one form of the generlized delt rule is chines, should hve little difficulty lerning the tsk pplied when logistic eqution is used s n (Dwson, 1998). Our purpose, however, is not ctivtion function (Rumelhrt et l., 1986). A simply to see whether we cn trin network to modified version of the generlized delt rule cn be solve the selection tsk but, more importntly, to used to trin networks of vlue units (Dwson & explore the lgorithm or how trined network Schopflocher, 1992). A vlue unit is processor tht solves the tsk. The explortion is not trivil since uses Gussin eqution for its ctivtion function. we cnnot predict the lgorithm connectionist network will develop to solve tsk. Exploring how 2.3. Problem difficulty nd PDP network s trined network solves the selection tsk cn lgorithm for problem solving extend our understnding of the tsk nd its existing theories. It is becuse of our interest in exploring It is possible to evlute problem s difficulty by how trined network solves the tsk tht we use the the extent of network s requirements for lerning to vlue unit rchitecture, whose hidden units often solve the problem. For instnce, the number of exhibit properties which render them interpretble hidden units tht network requires to solve (e.g., Berkeley, Dwson, Medler, Schopflocher & problem is indictive of the problem s difficulty Hornsby, 1995; Dwson, 1998; Dwson, Medler & (Dwson, 1998). Hidden units llow connectionist Berkeley, 1997). Next, we trin vlue unit network networks to solve linerly nonseprble problems. to generte response tht humn prticipnts Linerly nonseprble problems re difficult to solve, commonly mke to the selection tsk, but is incomcompred to linerly seprble problems, becuse plete. they require the network to divide the pttern spce into multiple decision regions. In contrst, linerly seprble problems require the network to mke 3. Network 1: selection of the p crd single division in the pttern spce so s to crete two decision regions. Linerly nonseprble prob- The gol of the first study ws to trin vlue unit lems cn be solved by networks tht hve lyer of PDP network to generte the p crd in response to hidden units. Ech hidden unit cn be viewed s the selection tsk. We trined this first network to creting cut in the pttern spce. Hence, the greter generte the p crd for two resons: First, prticipthe number of hidden units required by network to nts typiclly select the p crd lone in response to solve problem, the greter the number of decision the selection tsk (see Evns et l., 1993). Second, regions or cuts in the pttern spce demnded by we wnted to trin network to generte comprthe tsk. tively simple response, the selection of only one Lter in the pper, we will show how the ctivity crd, before trining network to generte more within hidden units cn be nlyzed nd interpreted difficult response. The purpose of trining this to uncover the lgorithm by which network lerns network (nd others which will be described lter in to solve prticulr tsk. Interpreting hidden unit studies 2 nd 3) is to determine the lgorithm by ctivity informs us of the specific fetures tht define which the network lerns to generte simple, but the decision regions creted by the network to solve incomplete response, to the tsk. Ultimtely, we the tsk. Although nlyzing hidden unit ctivity wnt to exmine the trined network s lgorithm to does not necessrily inform the question of problem see wht it cn tech us bout performnce on the or tsk difficulty, hidden unit ctivity does inform selection tsk nd existing lgorithmic theories of the the question of how the network solved the tsk; tht tsk. In the next section we discuss how we trined is, wht re the importnt pttern fetures tht define the network.

6 212 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Method Tble 1 Binry coding of conditionl rule types nd crds used to trin PDP version of Wson s selection tsk: networks nd generte input ptterns network rchitecture Text form Binry form When humn prticipnts re presented with the Rules: selection tsk, they lredy hve gret del of If vowel then even number knowledge bout its components. For instnce, pr- If vowel then odd number If consonnt then even number ticipnts come to the tsk with knowledge of () the If consonnt then odd number connective if then, (b) different kinds of numbers If even number then vowel (i.e., odd versus even), nd (c) different letters (i.e., If even number then consonnt vowels versus consonnts). In contrst, PDP net- If odd number then vowel works do not strt with this kind of knowledge. PDP If odd number then consonnt networks must first be given this preliminry in- Crds: formtion vi some formt tht the network cn A process. Furthermore, the responses or behvior the E network genertes t the end of trining should be J brod or generl enough (i.e., pplicble cross K lrge set of distinct ptterns) to be of interest to psychologists. Hence, we needed to first find wy to encode the tsk for the network nd, second, to crete sufficient number of input stimuli so tht the network could be trined on lrge number of ptterns. their ctegories in form tht is esily vilble to To solve these issues, binry code ws de- the network. As illustrted in Tble 1, ech crd is veloped tht llowed representtion of both the represented with three input units in such wy tht tsk s conditionl rule nd the four crds using 16 the first two input units represent the crd s ctegory, input units. Four inputs units were used to represent nd the third input represents the crd s exemplr of the rule. The first two input units reflected the tht ctegory. For instnce, the first two zeros in the ntecedent of the rule, wheres the lst two units code 000 indicte tht this string is vowel while reflected the consequent of the rule (see Tble 1). the third zero indictes tht, specificlly, this string is Four sets of three input units were then used to 1 letter A. represent the crd ctegories. The first two input units of ech set were used to represent the crd s 1 Although the encoding of vowel (i.e., 00) is on the surfce ctegory membership, wheres the lst unit of ech more similr to the encoding of even number (i.e., 10) thn to set ws used to represent its specific instnce. Using odd number (i.e., 11), this surfce similrity should not bis the network s solution. The reson for this is tht our trining set this encoding scheme, trining set ws developed included ll possible permuttions of rules nd ssocited solutht included eight different conditionl rules nd tions; hence, tht the rules nd crds shre some surfce eight different instnces of crd ctegories (two similrity mong portion of the ptterns is offset by the lck of vowels, two consonnts, two even numbers, nd two similrity shred mong the remining portion of ptterns. For odd numbers). These crd instnces were crossed exmple, suppose the network s tsk is to lern to select the p (ntecedent) nd the q (consequent) in response to the rules it is with 24 unique orders generted from ssembling presented (see study 3 in this pper). In response to one form of four crds in ll possible combintions. This ltter the rule, if vowel then even (0010), the network would lern to step led to 384 unique orders of crd vlues, which select the vowel crd (00) nd the even crd (10). In response to were then crossed with ech of the eight rules to nother rule, if vowel then odd (0011), the network would produce finl trining set of 3072 input ptterns. lern to select the vowel crd (00) nd the odd crd (11). Hence, ny surfce similrity between some rules nd some crds should We believe tht our encoding scheme cptures the not bis the network s solution of the tsk becuse the network underlying structure of the selection tsk. First, our lerns to solve the tsk by responding to the full set of rules nd binry coding scheme distinguishes exemplrs from crds in the trining set.

7 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Second, becuse the network ws exmined fter units, one corresponding to ech crd. The network it ws presented lrge number of trining ptterns, ws trined to respond by turning on one of its four it hd the opportunity to determine for itself the output units. For exmple, output unit 1 turned on underlying nture of the tsk. For exmple, ll of the (i.e., ws ctivted to vlue of 1) if crd 1 held the networks tht we trined lerned to ignore the lst desired response (i.e., ffirmed the ntecedent of the bit of the encoded crds, nd insted lerned to rule), but turned off if crd 1 did not hold this select crds bsed on the first two bits (i.e., cte- response. Only one output unit ws ctivted in gory). This is exctly wht humn prticipnts re response to n input pttern becuse, for network 1, expected to distinguish when they re presented with the desired response involved only the selection of the selection tsk; they utomticlly focus on crd the p crd. After the network lerned the solution, ctegory s opposed to crd instnce. we exmined this mture network for how it solved As shown in Fig. 3, network 1 required three the tsk. hidden units to lern the tsk. This ws becuse pilot A mture network responds ccurtely nd relibly simultions reveled tht three hidden units ws the to complete set of trining ptterns. In this study, minimum number of hidden units required for the relibility of response required tht () the network network to converge (i.e., to lern the desired be ble to identify the presented rule, (b) the network mpping between input pttern nd output response). hve some representtion tht ssigned ech input If fewer thn three hidden units were used, then the symbol to more bstrct ctegory (e.g., differentitnetwork ws unble to generte the desired response ing between vowel nd odd number ), nd (c) the to every pttern in the trining set. We used the network hve some representtion of the content of minimum number of hidden units to study the the presented rule, such tht its output would indicte network s solution of the problem (nd did not use wht could be done to test the vlidity of the rule. more hidden units) becuse it hs been rgued tht We believe tht network performnce consistent with this kind of network is most likely to produce n these three requirements for such lrge number of internl structure tht cn be trctbly interpreted different ptterns reflecting the selection tsk hs (e.g., Berkeley et l., 1995). developed sufficient bilities to be of psychologicl Fig. 3 lso shows tht the network hd four output interest. In short, such network could be used to Fig. 3. Illustrtion of the PDP network trined to generte the crd ffirming the ntecedent of the conditionl rule (i.e., the p crd).

8 214 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) extend our understnding of selection tsk perform- solved the tsk involved wire-tpping ech indince nd existing lgorithmic theories of the selec- vidul hidden unit for the input fetures it detected. tion tsk. Wire-tpping is one procedure tht cn be used to determine how PDP networks, in prticulr vlue Trining unit networks, solve problems (e.g., Dwson, 1998; Network 1 ws trined using Dwson nd Schop- Moorehed, Hig, & Clement, 1989). Wiretpping flocher s (1992) modifiction of Rumelhrt et l. s involves recording the responses of the hidden units (1986) generlized delt rule. The network ws to the ptterns in the trining set. After the network trined with lerning rte of nd momen- is trined on set of input ptterns, the ptterns re tum of zero. Connection weights nd unit bises (i.e., presented gin to the network while their ctivities the men of the Gussin) were rndomly selected in individul hidden units re recorded. The recorded from the rnge of to The weights nd ctivities re plotted nd exmined for meningful bises were updted fter the presenttion of ech configurtions. The configurtions provide clues s pttern. The order of pttern presenttion ws rn- to how the network is solving the tsk; tht is, the domized during ech epoch (n epoch is defined s configurtions indicte which ptterns fll into ech the presenttion of ll ptterns in the trining set). of the decision regions creted by the network to This ws done to ensure tht the network s lerning solve the tsk. of the tsk ws contingent on the specific input ptterns nd not on their specific sequence of pre Jittered density plots senttion. Jittered density plots of ech hidden unit were The network ws trined until it generted hit constructed subsequent to wiretpping, s shown in in response to every pttern. A desired response or Fig. 4. A jittered density plot illustrtes the dis- hit consisted of n ctivtion of 0.9 or higher in the tribution of ctivtion vlues produced in single output unit corresponding to the p crd long with hidden unit of mture network following prectivtions of 0.1 or lower in output units corre- senttion of full set of input ptterns. A single dot sponding to crds not ffirming the ntecedent of the in the plot represents the ctivtion tht one input rule. The network converged to solution to this pttern produces in hidden unit. Hence, one plot problem fter 83 epochs. Following trining, network illustrtes s mny dots s there re input ptterns. performnce on the selection tsk ws considered The x-xis on the jittered density plot rnges from 0 comprble to humn performnce in so fr s the to 1 nd shows the rnge of ctivtion vlues network generted relible response to specific generted by the input pttern set. Dots re lso input. Tht is, the network generted the p crd in rndomly jittered long the y-xis to mke them s response to the rule in the tsk, which is consistent discernible s possible. with pttern typiclly shown by humn prticip- Jittered density plots of vlue unit networks re nts. frequently highly structured or bnded. The distinct bnds represent groups of input ptterns tht shre 3.2. Results: definite fetures of hidden units similr fetures nd produce similr ctivtions in hidden unit. By exmining the fetures tht fll into The purpose of this first study ws to interpret the ech bnd, it is possible to identify the fetures tht method by which network 1 lerned to solve the the network used to solve problem. From the tsk; tht is, lerned to select the p crd in response presence of the bnds, we know tht hidden units re to the conditionl rule. Interpreting network 1 in- relibly detecting specific input fetures in solving volved exmining ech of the network s hidden units the tsk (Berkeley et l., 1995). Although bnds for the input fetures it detected. Once the reltion- provide informtion bout the fetures used in ship between hidden units nd input fetures ws solving problem, the bnds do not necessrily known, it ws then possible to determine how the provide informtion bout the problem s complexity. network solved the tsk. The number of hidden units is better indictor of The first step to understnding how the network tsk complexity. While the number of hidden units

9 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Fig. 4. Jittered density plots for ech of the three hidden vlue units in network 1. required by network to solve tsk indictes the exmple, perfectly positive correltion between number of decision regions in the pttern spce, input units 6 nd 7 indictes tht these units lwys bnds reflect the fetures tht chrcterize the deci- ssume the sme vlue; if input unit 6 hs vlue of sion regions. As shown in Fig. 4, ll three hidden 1.0, then so does input unit 7. In contrst, perfectly units in network 1 exhibited high degree of negtive correltion between pir of input units bnding. suggests tht whenever one input unit is 1.0, the With the id of descriptive sttistics, it is possible other is 0.0 nd vice vers. Network 1 s hidden units to identify the specific fetures tht cluster into ech detected only definite binry fetures. A description bnd nd, moreover, how the network uses this of the definite binry fetures detected by ech configurtion or crving of the input spce to solve hidden unit in network 1 is presented in Tble 2. tsk. By clculting the Person product-moment correltion mong the ptterns in bnd, it is Interprettion of definite binry fetures possible to lern if hidden unit is detecting definite An inspection of Tble 2 suggests tht network 1 binry fetures. A definite binry feture indictes solved the tsk by detecting binry fetures repreperfectly relible correltion between input units. For senting rules nd specific crds. First, lthough ech hidden unit detected ll eight rules, ech hidden unit Tble 2 detected specific crd. For exmple, notice tht Definite fetures for bnds from hidden units of network 1 hidden unit 0 ws highly ctivted by ptterns (i.e., Hidden Bnd Definite fetures n bnd C in Tble 2) whose definite fetures showed unit lbel input units 10 nd 11 shring correltion of A I0± I2, I0 ± I10, I2 5 I with input units 0 nd 1, respectively. Recll tht 0 B I0± I2, I1 ± I input units 10 nd 11 represent crd 3 vlues, 0 C I0± I2, I0 5 I10, I1 5 I11, 768 wheres input units 0 nd 1 represent the ntecedent I2 ± I10 of the rule. In other words, hidden unit 0 ws highly 1 A I0± I2, I0 ± I7, I2 5 I B I0± I2, I1 ± I ctivted when the desired response, p, ws locted 1 C I0± I2, I0 5 I7, I1 5 I8 768 t crd 3. Notice lso tht hidden unit 0 ws I2 ± I7 modertely ctivted by ptterns (i.e., bnds A nd 2 A I0± I2, I0 ± I4, I2 5 I B) whose definite fetures showed input units 10 nd 2 B I0± I2, I1 ± I shring correltion of with input units 0 2 C I0± I2, I0 5 I4, I1 5 I5 768 I2 ± I4 nd 1, respectively. In other words, hidden unit 0 ws not highly ctivted when the vlue t crd 3 ± indictes perfectly negtive correltion between input units; 5 indictes perfectly positive correltion between input filed to mtch the ntecedent of the rule. units. The sme nlysis cn be pplied to hidden units 1 n, number of ptterns flling in ech bnd. nd 2. For exmple, hidden unit 1 detected desired

10 216 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) responses t crd 2 (i.e., inputs units 7 through 9), generlly. Cosmides (1989) socil contrct theory nd hidden unit 2 detected desired responses t crd nd Johnson-Lird s mentl models theory (1983) 1 (i.e., input units 4 through 6). Although crd 4 re lso uncler bout the influence the evidence vlues were not directly detected by ny of the might hve on prticipnts performnce. All three hidden units, crd 4 vlues were detected indirectly theories focus primrily on the nture of the rule nd by ll three hidden units. In vlue unit rchitectures, its ssocited context. In contrst, ccording to Rips it is possible for zero signl to moderte high (1994) syntctic theory, the nture of the evidence ctivity or 1 response if both the bis of the should hve little influence on performnce since rchitecture is equl to zero nd the signl being sent mentl rules operte on the syntx of the selection is equl to zero. In other words, in stte where tsk rule. In the next study, we trined network to none of the three hidden units were ctivted (i.e., solve the tsk by selecting the p nd not-q crds. the desired response ws not found t crd 1, 2 or 3), This is response rrely exhibited by humn prthis stte ctivted output unit or crd 4 (i.e., input ticipnts. units 13 through 15) Discussion 4. Network 2: selection of the p crd nd the Network 1 lerned to select the p crd in not-q crd response to the conditionl rule by hving ech hidden unit detect vlues t specific crd loctions Method This specilized hidden unit lgorithm did not discriminte mong the rules used in the tsk since Network rchitecture ll three hidden units detected ll rules. Insted, the Network 2 ws trined under the sme method specilized lgorithm discriminted mong crd loc- used to trin network 1 identicl input encoding, tions to solve the tsk. We speculte tht the net- trining ptterns, nd output units were used. This work s specific focus on crd loction points to the time, however, two output units insted of one were importnce of the evidence in the selection tsk. The designed to turn on in response to every input crds in the selection tsk illustrte the evidence with pttern since the solution of the tsk involved the which to test the rule. Although few existing theories selection of two crds. As shown in Fig. 5, network of selection tsk performnce focus on the evidence, 2 required eight hidden units insted of three to lern some investigtors hve proposed tht the kind of the tsk. Pilot simultions reveled tht the network evidence vilble to prticipnts plys n importnt could not lern to generte the desired mpping from role in how prticipnts choose to test rule (e.g., inputs to outputs with fewer thn eight hidden units Klymn & H, 1987; Libermn & Klr, 1996). For when the network ws required to select two crds. exmple, Libermn & Klr (1996) suggested tht if prticipnts perceive the evidence needed to test hypothesis s typicl, they might forego using the Trining typicl evidence nd choose to test the hypothesis Network 2 ws trined similrly to network 1 with using more conventionl evidence. The results ob- the exception tht network 2 ws trined to select tined from network 1 suggest tht the evidence in two responses insted of one. Becuse network 2 the selection tsk might ply n importnt role in selected two responses, it needed to lern to disprticipnts responses. tinguish between propositions p nd not-q At this time, it is uncler how the findings in its responses. We did not indicte to the network obtined from network 1 support or chllenge exist- before its trining which vlues represented p nd ing theories of selection tsk performnce. For which vlues represented not-q becuse lerning to exmple, Cheng nd Holyok s (1985, 1989) prg- mke this distinction is n integrl prt of lerning mtic resoning theory is vgue bout how prticip- the tsk. When humn beings solve the selection nts might view the evidence or its role in resoning tsk, they pproch the tsk lredy knowing which

11 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Fig. 5. Illustrtion of the PDP network trined to generte both the crd ffirming the ntecedent nd the crd negting the consequent of the conditionl rule (i.e., the p nd the not-q crds). vlues represent p nd which vlues represent not- the selection tsk; they utomticlly focus on crd q, but t some point in their lerning history humn ctegory s opposed to crd instnce. beings hve hd to lern to mke the distinction between propositions. Lerning to distinguish propo Results sitions ws prt of the network s trining on the tsk. Network 2 ws nlyzed following the sme As with network 1, prior to trining, the network s pproch used to nlyze network 1. As illustrted in connection weights were rndomly set to vlues Figs. 6 nd 7, jittered density plots of ech of the between nd 1 1.0, while unit bises were set eight hidden units in network 2 reveled high to 0.1. The lerning rte ws nd no momen- degree of bnding. Such pronounced bnding sugtum ws used. At the end of trining, the network gested tht the network s solution to the tsk ingenerted hit in response to every pttern. A volved the detection of definite fetures. An exmidesired response or hit consisted of n ctivtion of ntion of the plots, furthermore, suggested tht pirs 0.9 or higher in the output units corresponding to the of hidden units hd similr ptterns of bnding; for desired response (i.e., p or not-q ). An ctivtion exmple, hidden units 0 nd 6, hidden units 1 nd 4, of 0.1 or lower chrcterized outputs units corre- hidden units 2 nd 5, nd hidden units 3 nd 7. The sponding to other responses. The network lerned to visul similrity observed in the plots between pirs generte the desired response to ll ptterns fter 414 of hidden units ws supported when we rn epochs. correltion mong hidden unit ctivity. In prticulr, In ddition, becuse the network ws exmined we discovered tht the ctivity of hidden units 3 nd fter it ws presented lrge number of trining 7 shred correltion of when desired ptterns, it hd the opportunity to determine for itself response ws locted t crd 1 (see Tbles 3 nd 4); the underlying nture of the tsk. For exmple, ll of hidden units 0 nd 6 shred correltion of 0.99 the networks tht we trined lerned to ignore the when desired response ws locted t crd 2; lst bit of the encoded crds, nd insted lerned to hidden units 2 nd 5 shred correltion of 1 when select crds bsed on the first two bits (i.e., cte- desired response ws locted t crd 3; nd hidden gory). This is exctly wht humn prticipnts re units 1 nd 4 shred correltion of 0.81 when expected to distinguish when they re presented with desired response ws locted t crd 4. Strong

12 218 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Fig. 6. Jittered density plots for the first set of four hidden vlue units used in network 2. Fig. 7. Jittered density plots for the second set of four hidden vlue units used in network 2. correltions between pirs of hidden units disp- through 4 represent the rule of the tsk, the first two pered when undesired responses were locted t bits represent the ntecedent of the rule while the lst their respective crd loctions. two bits represent the consequent. Hidden unit 2 ws An exmintion of the definite fetures detected lso highly ctivted by ptterns tht hd desired by network 2 s hidden units suggested specilized response (in reltion to the rule) locted t crd 3. lgorithm similr to tht found for network 1. Tbles This mens tht input ptterns tht hd desired 5 8 show the exhustive list of definite fetures tht response p or not-q locted t crd 3 highly ll eight hidden units detected. To illustrte network ctivted hidden unit 2. 2 s lgorithm, we will focus on hidden unit 2 (see Hidden unit 2 s ctivity ws highly correlted with Tble 7). Although it would be too lborious to hidden unit 5 s ctivity in detecting desired redescribe here the entire list of definite fetures tht sponses t crd 3. Decoding the list of definite hidden unit 2 detected, decoding the list indictes fetures detected by hidden unit 5 indictes tht this tht hidden unit 2 ws highly ctivted by ptterns hidden unit ws highly ctivted by ptterns tht hd whose definite fetures hd the following vlues t the following vlues t input units 1 through 4: 0011, input units 1 through 4: 0011, 1100, 0110, 1001, 1100, 0110, 1001, 0111, nd Hidden unit 5 did 0111, 1110, 0010, Recll tht input units 1 not detect ny definite fetures ssocited with crd

13 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Tble 3 Correltion mong hidden units when crds 1 nd 2 re selected network 2 H0 H1 H2 H3 H4 H5 H6 H7 Crd 1: H0 1 H H H H H H H Crd 2: H0 1 H1 0 1 H H H H H H Tble 4 Correltion mong hidden units when crds 3 nd 4 re selected network 2 H0 H1 H2 H3 H4 H5 H6 H7 Crd 3: H0 1 H H H H H H H Crd 4: H0 1 H H H H H H H vlues. In short, hidden units 2 nd 5 were both 4.2. Discussion highly ctivted by lrge set of rules nd helped to detect responses locted t crd 3. Similr ccounts Network 2 solved the tsk by mens of specilized my be mde for the remining pirs of hidden units pirs of hidden units. In prticulr, hidden units 0 (see Tbles 5, 6 nd 8). nd 6 detected desired responses locted t crd 2,

14 220 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Tble 5 Definite fetures for bnds from hidden units 0 nd 6 of network 2 H0- Definite fetures n H6- Definite fetures n bnds A I0± I1, I0 ± I2, I0 5 I3, I0 ± I7, 192 A I0 ± I I0 ± I8, I1 5 I2, I1 ± I3, I1 5 I7, I1 5 I8, I2 ± I3, I2 5 I7, I2 5 I8, B I0 ± I2, I1 ± I3 576 I3 ± I7, I3 ± I8, I7 5 I8 C I05 I1, I0 ± I2, I0 ± I3, I0 5 I7, 192 B I05 I1, I0 ± I2, I0 5 I3, I0 ± I7, 192 I0 ± I8, I1 ± I2, I1 ± I3, I1 5 I7, I0 ± I8, I1 ± I2, I1 5 I3, I1 ± I7, I1 ± I8, I2 5 I3, I2 ± I7, I2 5 I8, I1 ± I8, I2 ± I3, I2 5 I7, I2 5 I8, I3 ± I7, I3 5 I8, I7 ± I8 I3 ± I7, I3 ± I8, I7 5 I8 D I05 I1, I0 ± I2, I0 ± I3, I0 ± I7, 192 C I0± I2, I0 5 I7, I1 ± I3 384 I0 5 I8, I1 ± I2, I1 ± I3, I1 ± I7, I1 5 I8, I2 ± I7, I3 ± I8 I1 5 I8, I2 5 I3, I2 5 I7, I2 ± I8, I3 5 I7, I3 ± I8, I7 ± I8 D I0± I2, I1 5 I3 576 bnds E I05 I1, I0 ± I2, I0 ± I3, I0 ± I7, 192 E I0 ± I1, I0 ± I2, I0 5 I3, I0 5 I7, 192 I0 5 I8, I1 ± I2, I1 ± I3, I1 ± I7, I0 5 I8, I1 5 I2, I1 ± I3, I1 ± I7, I1 5 I8, I2 5 I3, I2 5 I7, I2 ± I8, I1 ± I8, I2 ± I3, I2 ± I7, I2 ± I8 I3 5 I7, I3 ± I8, I7 ± I8 I3 5 I7, I3 5 I8, I7 5 I8 F I0± I1, I0 ± I2, I0 5 I3, I0 ± I7, 192 I0 5 I8, I1 5 I2, I1 ± I3, I1 5 I7, I1 ± I8, I2 ± I3, I2 5 I7, I2 ± I8, I3 ± I7, I3 5 I8, I7 ± I8 G I05 I1, I0 ± I2, I0 ± I3, I0 ± I7, 192 I0 ± I8, I1 ± I2, I1 ± I3, I1 ± I7, I1 ± I8, I2 5 I3, I2 5 I7, I2 5 I8, I3 5 I7, I3 5 I8, I7 5 I8 H I0± I2, I1 5 I3 768 I I0± I2, I0 5 I7, I1 ± I3, I1 ± I8, 384 I2 ± I7, I3 5 I8 ± indictes perfectly negtive correltion between input units; 5 indictes perfectly positive correltion between input units. n, number of ptterns flling in ech bnd. hidden units 1 nd 4 detected desired responses bout both network 1 nd network 2 is tht their locted t crd 4, hidden units 2 nd 5 detected lgorithms for solving the tsks involved specildesired responses locted t crd 3, nd hidden units ized hidden units detecting desired responses t 3 nd 7 detected desired responses locted t crd 1. specific crd loctions. In comprison to the tsk network 1 hd to solve, network 2 s tsk ws more difficult. Wheres network 1 required only three hidden units to solve its 5. Network 3: selection of the p crd nd the tsk, network 2 required eight hidden units to solve q crd its tsk. It is not surprising tht network 2 required greter number of hidden units to solve its tsk, Both networks 1 nd 2 solved the tsk by mens however, given tht it generted two responses of specilized hidden units tht focused on crd insted of just the one. Nevertheless, wht is striking loction. Tht hidden units specilized to discrimi-

15 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) nte crd loctions ws intriguing to us becuse this were set to zero. The lerning rte ws nd no spect of the selection tsk hs not been exmined momentum ws used. At the end of trining, the closely in the pst. However, we were not only network generted hit in response to every intrigued by this finding but lso by the pprent pttern. A desired response or hit consisted of n difficulty of generting two crd vlues insted of ctivtion of 0.9 or higher in the output units just one. Although we expected the selection of two corresponding to the p nd q crds, nd n crd vlues to be more difficult tsk, we did not ctivtion of 0.1 or lower in output units correexpect network 2 to need more thn double the sponding to the other crds. The network lerned to hidden units required to trin network 1. Recll tht select the desired responses fter 115 epochs of the number of hidden units required by network to trining. solve tsk is indictive of the tsk s difficulty. We conjectured tht nother reson for network 2 s tsk difficulty might hve something to do with needing 5.2. Results to select the not-q response. Solving the tsk by selecting the not-q response is highly uncommon As shown in Figs. 8 nd 9, jittered density plots of for humn prticipnts; it is much more common for ech of the hidden units displyed high degree of prticipnts to select the p crd or both the p nd bnding, which suggested tht network 3 s solution q crds (Evns et l., 1993). involved the detection of definite fetures. An in- We tested our conjecture by trining third spection of the jittered density plots indicted tht network to select both the p nd the q crds. In some hidden units were detecting similr definite this wy, we hold constnt the number of crds the fetures. For exmple, four out of the eight hidden network must select in responding to the tsk, while units (i.e., hidden units 0, 2, 4, nd 6) hd similr t the sme time testing to see if selecting the q jittered density plots. Tbles 9 nd 10 show the response is esier to lern thn the not-q response. correltions mong the hidden unit ctivity levels. Trining third network to select the p nd q lso From the tbles, we see tht the ctivity levels of served s further test of the lgorithms found for both hidden units 3 nd 4 nd hidden units 5 nd 7 both networks 1 nd 2. correlted 0.99 when desired response ws locted t crd 1. In contrst, when desired response ws 5.1. Method locted t crd 2, hidden units 0 nd 4 shred correltion of Moreover, when desired response ws locted t crd 3, hidden units 4 nd Network rchitecture shred correltion of 0.99, nd when desired The sme method used to trin both networks 1 response ws locted t crd 4, hidden unit 2 nd 4 nd 2 ws used to trin network 3. As with network shred correltion of Unlike network 2, 2, two output units were designed to turn on in detecting desired response t crd 1 required four response to every input pttern since the desired hidden units, while detecting desired responses t response involved the selection of two crds. Surother crd loctions required only two. In ddition, prisingly, we found tht network 3 lso required hidden unit 4 ws involved in ll response selections. eight hidden units to converge, which is the sme A closer nlysis of how network 3 solved the tsk number of hidden units required by network 2. We reveled severl interesting results. First, the hidden hd expected network 3 to require fewer hidden units units in network 3 did not detect s mny definite bsed on our prediction tht selecting the q crd fetures in solving the tsk s we found for network might be more esily lerned thn selecting the 2. For exmple, notice tht network 3 s jittered not-q crd. density plots did not exhibit s mny individul bnds for ech hidden unit s we observed for Trining network 2. Second, nd more specificlly, n inspec- As with networks 1 nd 2, prior to trining, tion of the definite fetures detected by hidden unit 4 network 3 s connection weights were rndomly set to reveled correspondence to the four input units vlues between to 1 1.0, while its unit bises used to represent the rule in the tsk. Tble 13 shows

16 222 J.P. Leighton, M.R.W. Dwson / Journl of Cognitive Systems Reserch 2 (2001) Tble 6 Definite fetures for bnds from hidden units 1 nd 4 of network 2 H1- Definite fetures n H4- Definite fetures n bnds A I0± I2, I1 ± I A I0 5 I1, I0 ± I2, I0 ± I13, I0 5 I14, 384 I1 ± I2, I1 ± I13, I1 5 I14, I2 5 I13, B I0± I1, I0 ± I2, I0 ± I3, I1 5 I2, 384 I2 ± I14, I13 ± I14 I1 5 I3, I2 5 I3, I13 5 I14 bnds C I05 I1, I0 ± I2, I0 5 I3, I0 5 I13, 192 B I0 5 I1, I0 ± I2, I0 5 I3, I0 ± I13, 192 I0 ± I14, I1 ± I2, I1 5 I3, I1 ± I13, I0 ± I14, I1 ± I2, I1 5 I3, I1 ± I13, I1 ± I4, I2 ± I3, I2 ± I13, I2 5 I14, I1 ± I14, I2 ± I3, I2 5 I13, I2 5 I14, I3 5 I13, I3 ± I14, I13 ± I14 I3 ± I13, I3 ± I14, I13 5 I14 D I05 I1, I0 ± I2, I0 5 I3, I0 ± I13, 192 C I0 5 I1, I0 ± I2, I0 ± I3, I0 ± I13, 192 I0 5 I14, I1 ± I2, I1 ± I3, I1 ± I13, I0 ± I14, I1 ± I2, I1 ± I3, I1 ± I13, I1 5 I14, I2 5 I3, I2 5 I13, I2 ± I14, I1 ± I14, I2 5 I3, I2 5 I13, I2 5 I14, I3 5 I13, I3 ± I14, I13 ± I14 I3 5 I13, I3 5 I14, I13 5 I14 E I05 I1, I0 ± I2, I0 5 I3, I0 5 I13, 192 D I0 ± I1, I0 ± I2, I0 5 I3, I0 ± I13, 192 I0 5 I14, I1 ± I2, I1 5 I3, I1 5 I13, I0 5 I14, I1 5 I2, I1 ± I3, I1 5 I13, I1 5 I14, I2 ± I3, I2 ± I13, I2 ± I14, I1 ± I14, I2 ± I3, I2 5 I13, I2 ± I14, I3 5 I13, I3 5 I14, I13 5 I14 I3 ± I13, I3 5 I14, I13 ± I14 F I0± I1, I0 ± I2, I0 ± I3, I0 ± I13, 192 E I0 ± I1, I0 ± I2, I0 ± I3, I0 ± I13, 192 I0 5 I14, I1 5 I2, I1 5 I3, I1 5 I13, I0 5 I14, I1 5 I2, I1 5 I3, I1 5 I13, I1 ± I14, I2 5 I3, I2 5 I13, I2 5 I14, I1 ± I14, I2 5 I3, I2 5 I13, I2 ± I14, I3 5 I13, I3 ± I14, I13 ± I14 I3 5 I13, I3 ± I14, I13 ± I14 G I0± I1, I0 ± I2, I0 ± I3, I0 5 I13, 192 F I0 ± I1, I0 ± I2, I0 ± I3, I0 5 I13, 192 I0 ± I14, I1 5 I2, I1 5 I3, I1 ± I13, I0 ± I14, I1 5 I2, I1 5 I3, I1 ± I13, I1 5 I14, I2 5 I3, I2 ± I13, I2 5 I14, I1 5 I14, I2 5 I3, I2 ± I13, I2 5 I14, I3 ± I13, I3 5 I14, I13 ± I14 I3 ± I13, I3 5 I14, I13 ± I14 H I0± I1, I0 ± I2, I0 5 I3, I0 5 I13, 192 G I0 ± I1, I0 ± I2, I0 5 I3, I0 5 I13, 192 I0 ± I14, I1 5 I2, I1 ± I3, I1 ± I13, I0 ± I14, I1 5 I2, I1 ± I3, I1 ± I13, I1 5 I14, I2 ± I3, I2 ± I13, I2 5 I14, I1 5 I14, I2 ± I3, I2 ± I13, I2 5 I14, I3 5 I13, I3 ± I14, I13 ± I14 I3 5 I13, I3 ± I14, I13 ± I14 I I0± I1, I0 ± I2, I0 5 I3, I0 ± I13, 192 H I0 5 I1, I0 ± I2, I0 5 I3, I0 5 I13, 192 I0 ± I14, I1 5 I2, I1 ± I3, I1 5 I13, I0 5 I14, I1 ± I2, I1 5 I3, I1 5 I13, I1 5 I14, I2 ± I3, I2 5 I13, I2 5 I14, I1 5 I14, I2 ± I3, I2 ± I13, I2 ± I14, I3 ± I13, I3 ± I14, I13 5 I14 I3 5 I13, I3 5 I14, I13 5 I14 J I05 I1, I0 ± I2, I0 ± I3, I0 5 I13, 192 I I0 ± I1, I0 ± I2, I0 5 I3, I0 ± I13, 192 I0 5 I14, I1 ± I2, I1 ± I3, I1 5 I13, I0 ± I14, I1 5 I2, I1 ± I3, I1 5 I13, I1 5 I14, I2 5 I3, I2 ± I13, I2 ± I14, I1 5 I14, I2 ± I3, I2 5 I13, I2 5 I14, I3 ± I13, I3 ± I14, I13 5 I14 I3 ± I13, I3 ± I14, I13 5 I14 K I05 I1, I0 ± I2, I0 5 I3, I0 ± I13, 192 J I0 5 I1, I0 ± I2, I0 ± I3, I0 5 I13, 192 I0 5 I14, I1 ± I2, I1 5 I3, I1 ± I13, I0 5 I14, I1 ± I2, I1 ± I3, I1 5 I13, I1 5 I14, I2 ± I3, I2 5 I13, I2 ± I14, I1 5 I14, I2 5 I3, I2 ± I13, I2 ± I14, I3 ± I13, I3 5 I14, I13 ± I14 I3 ± I13, I3 ± I14, I13 5 I14