에피소드 15: Dr. Michael Perone

7:16

correct right i'm sure you've you've seen novice dog trainers train dogs to do bad things yes in fact with that there is a saying that all bad behaviors have been most likely positively reinforced some at some point of time right exactly exactly exactly um now you know i'm not suggesting that the way to train a dog to uh do whatever it is that you want to do is to uh electrify its feet and turn off the electricity when it does something right that would be that would be barbaric but i will point out that uh uh uh negative reinforcement is what keeps you from stepping off the curb without looking when you're walking down the street now consider uh when you walk down the street in an urban environment uh there are cars whizzing back and forth and maybe they're only going 25 or 30 miles an hour but if you step in front of one of them you're going to be dead or or maybe worse and uh and yet we walk down up and down the street every day within a few feet of these massive machines that could be deadly and we don't worry about it at all we're not scared we're not upset we're not anxious and why not because we are proficient in avoiding the danger that lurks just a few feet away we have a non-set negative reinforcement contingencies are so effective that they control our behavior and they don't bother us in the least now maybe when if we could look back far enough into our childhood maybe when our parents were teaching us how to cross the street and to look both ways and all that maybe we were a little anxious as as as that kind of behavior was being acquired but by the time one is i don't know 10 or 11 years old probably earlier than that we negotiate uh uh travel throughout cities without being scared at all and this is negative reinforcement negative reinforcement is why we uh button our coats when it's cold yes there's a there's a lot of there's a lot of good things that happen because of negative reinforcement and to suggest that uh negative reinforcement uh in itself is a bad thing would just be incorrect do you find that i mean like i as i mentioned in the dog training world this is this is a problem it's it's you know we're quite divided and and for whatever reason associating just just throwing the word negative reinforcement or positive punishment or any use of aversive immediately uh resonates with something horrible happening is that do you find anything like this do you have to deal with something like this when you when you even when you teach your in your classes or how yes oh yes uh i certainly do

10:36

and uh what i find is that uh the the antidote is showing students the basic research in these areas and uh if possible getting the getting them to participate in the research in these areas um over the years i've uh i've had i've done lots of studies involving um avoidance of electric shock and one might think that a rat that is exposed to an avoidance schedule day in and day out month after month these are steady state experiments would be a skittish or difficult animal to work with but in fact they're completely friendly and docile they um probably largely because they they very uh quickly become proficient avoiders and um and in my laboratory a rat can sometimes go an hour or two without getting a single shock in fact that's what makes it so fascinating ivan is if you were to look inside an operant chamber in my laboratory you might see a rat that's pressing a lever and nothing's happening nothing happens before the rat presses a lever nothing happens after the rat presses a lever and they go on and they do this for an hour or two at a time and you'd say how is this possible uh nothing's happening it appears to be extinct that the schedule's extinction but it's not it's a shock postponement schedule and somehow the animal animal's behavior comes under the control of this schedule the rat itself is quite calm right question calm pretty laid back uh not the least bit skittish and when the experiment's over very easy to handle um it's uh it's it we still don't know we still don't have a uh a consensus about what is reinforcing this behavior obviously the reinforcement is obscure we know what the schedule is doing we know that we know what the schedule is but we don't know the reinforcement mechanism that the schedule is activating and that's what makes it so interesting but i'm not a bad person because i because i study this i'm not mean to the rats i love the rats and their their health and well-being is of critical importance to us because our experiments often last uh most of the lifespan of the animal so we need them to be hale and hearty so obviously when we use any form of aversive things can go sideways sure and normally what what what needs to go wrong for this to happen well um like we we talk about contingent and non-contingent which is very big deal right um but what what do you find in in when you work with them well you know um i think it's important to uh maintain a uh a careful distinction between as you say negative reinforcement and positive punishment they're both they're they're unified in the sense that they both fall under the category of aversive control

14:10

um my guess and i'm guessing uh is that a lot of dog training would would probably be more likely to involve punishment than negative reinforcement um and and the thing about punishment that's very clear in the laboratory is that very mild stimuli can function as effective punishers um and i'll give you an example um in in my laboratory i've studied both punishment and and avoidance and i've used other stimuli besides electric shock but just with electric shock to to uh get a rat to acquire and maintain coefficient avoidance requires an electric shock of about one milliamp now to you and me one milliamp is nothing you can barely feel it but to a little rat it's it's an unpleasant experience one milliamp but to suppress responding you need much less than that so if you train a rat on let's say a variable interval schedule of food reinforcement and then you superimpose on that electric shock punishment you only need 0.2 or 0.3 milliamps to suppress behavior now notice that a 0.2 milliamp shock will not sustain negative reinforcement it will not it will not sustain avoidance a rat will not work to avoid a shock that mild but in a punishment paradigm that mild of a shock will serve as a highly effective punisher now this is this is not a peculiarity of rats you can see this in people too there have been a number of of human operant experiments that have shown that a very tiny fraction of a penny will function as an effective punisher of a punishment contingency and and so uh uh uh we are very sensitive we we mammals at least seem to be very sensitive to um punitive contingencies and it doesn't take much of a uh of a punishing stimulus to be effective so i think the danger that uh that it is that you need to be aware of the most is that if you're going to use a punishment contingency you can be very gentle yes a very gentle rebuke can be effective somehow humans and dog trainers and probably just humans overall we think of oh this is punishment we need to be very uh uh amplify the the intensity and and that's exactly like this is very very um interesting and i think it's for for sure very um well recorded in in a lot of research there is a lot of different rules that need to happen through punishment so it doesn't become a conditioned reinforcer and and sure right so the but when all those rules are met as you said a very low level will do the job to punish to where to convince the dog to actually avoid or escape and and it's a different different story then it depends on yeah that's

17:41

very very interesting well i don't i mean uh i've i've had pets over the years i've had dogs and cats both and i know this i'm not going to teach a dog to do any tricks by punishing it by punishing its behavior but i might be able to facilitate uh the uh acquisition of behavior by um gently punishing conflicting behavior or um a distracting behavior and i suspect that any dog owner uh uh that that cares about its dog engages in punishment on a frequent basis but they would never label it that so they might just they might just say no no no no no no yes you know something like that well that's that's that's a punishment procedure you know people tend to think of punishment as something that you do to the animal in anger you know you're frustrated you're angry and you whack the dog well that that's uh that's not an effective uh punishment procedure yes that's revenge in certain in certain dog training circles the that condition punishers like like saying no um it's not even called punisher they they call it interrupter just because the word punishment has it's such a loaded word and for some reason we are in the dog training uh industry we are very afraid to to say oh we have to punish something because it will help us suppress and then therefore we can reinforce the good behaviors when they happen but um finding finding finding the it's very polarized the dog training is and i i believe somehow the whole society at the moment is it's very polarized on on what is what where the boundaries boundaries between becoming unethical and and and doing doing doing stupid things to to an animal versus intelligently applying aversive to to to accomplish to to get a result and with negative reinforcement it's it's almost the same thing um the way i look at negative reinforcement in when when when done right it's actually very like when we look at even a silly game as uh what hot hands you know it's a it's a game of negative reinforcement it has an aversive element but it's so interesting and it's like little kids will play it forever and yes we can go wrong but as long as like if we if we focus on the negative reinforcement part we have able to explain the way out then actually i i believe that it becomes uh almost it builds certain level of confidence of i i'm somebody i can i can i can avoid things i know how to manage and how to be more resilient with uh um just just go on with life right in many cases well first i don't have a i personally have no problem if instead of saying

21:37

we're going to punish certain behavior in the dog that we're going to discourage it for example i mean it it it it it's it's one thing you know when i use the word punishment in my research i'm using it as a technical term and it's unfortunate that when it's used outside the technical literature it has such a um as you say it has it's a loaded term um and so uh in working with clients uh and with colleagues if if if there's a different kind of jargon uh that emerges in the world of dog training that makes sense to me and uh i certainly uh wouldn't fault anybody for it mm-hmm you know i think in i think um one of the one of the most interesting uh forms of negative reinforcement is often referred to as compassion i mean think about this if you if you uh watch television uh uh you will you will certainly see commercials where they show dogs and cats and other cute animals that are being abused you know they show them chained up they show them starving they show them crying and they and then they say why don't you send us 19 a month so we can stop this well people look at that and they and it brings it their eyes get moist i mean if they have any kind of soul right they look at that their eyes get moist their heart goes out to these animals and they feel pretty good about writing a check for 19 or signing up to send the check every month we call we don't call that negative reinforcement but that's exactly what it is we call it compassion and and you know that's there's so many appeals like that uh uh and and and i'm not saying they're wrong some people might say it's an emotional manipulation i i wouldn't go that far people are showing the state of the world and saying you can make it better but they they they get you to make it better by showing you something bad and telling you that you can make the bad go away that's that's negative reinforcement many years ago the uh uh the great observer of the human condition h l mencken uh in an essay mentioned uh uh talked a little bit about altruism now you know altruism is supposed to be the purest form of charity and he says uh i'll paraphrase but he says altruism even in its purest form uh happens because it it's uncomfortable to have unhappy people around you know it we we don't our heart goes out to unhappy people to crying children to to to starving animals uh to crying colleagues you know to students who are upset our heart goes after them and we do what we can to help that's negative reinforcement negative reinforcement is part of what us helps us make

24:54

the world a better place wow interesting very very interesting um that that's something to really yeah it is uh i i almost need to but but it's so true this is uh there are so many different takes on on what negative reinforcement is um they're thinking about it too narrowly ivan when you uh do you cook of course i love it well so you you uh here you are in your kitchen and you're uh you're picking up uh let's make it very simple you're picking up a pot of boiling water are you scared are you upset are you anxious do you break up into a into a sweat knowing that you're gonna have to do this no you just go about it i don't i doubt that you even think about it much but you've acquired a repertoire of very skilled behavior and you know exactly how to pick up the pan and how to move it from burner to stove from burner to burner burner to counter without spilling hot water and hurting yourself um this is this is avoidance behavior that's uh that was it's not too hard to figure out how it got started and it's maintained by negative reinforcement contingencies but these are good things they keep you from getting your fingers burned or worse and they don't generate a bunch of emotional byproducts too many people love to cook even even though they're they're inches away from uh heat that could do serious serious damage yes and sometimes they're not scared they're not scared exactly exactly very very good point and sometimes you pay the price for being negligent but you know exactly what you did and you know exactly how to change it for the next at least two years before you lose your guard again and and about every every two or three years i i managed to burn burn something getting a something out of the oven yes i'm i admit it and i always feel very stupid when i've done it and just like you pointed out which is super interesting how how a rat can be in in in the process of navigating in the the laboratory in and another example i have i have a school for dog trainers and i bring a my my analogy is with the electricians that go in this high voltage poles and they don't go to work every day stressed out sweating that they have an answer and it is the most critical point of negative reinforcement from what i from my experience in dog training is to to be able to show to the dog or animal or a human immediately how to escape and eventually how to completely avoid the aversive if we have that even if we have a a very brief element of oh wow this is a a little bit i'm i'm stressed about it it's a puzzle can i solve it but

28:24

the moment it gets solved all that fear and all that stress is resolved correct yes yeah and go ahead go ahead well i the i guess like in at least in the dog world we tend to get stuck on that moment of solving the puzzle and pointing out that okay it's very bad but this is at the moment when the dog is actually searching for solution and it's very temporary moment assuming that we have a a proper schedule and and we are on the way to learning how to escape first how to escape and next obviously how to avoid and and if the animal gets stuck in that place we have not done our job correctly right that's right and you know the the uh the dog is equipped to cope with uh aversive contingencies we know this because its mother uh engages in a aversive contingencies right i mean the the mother will let let out a low growl or low pitched bark uh when the when the puppy does something wrong and uh uh i have a i have a 16 year old cat right now that's coping with a kitten and the cat uh the old cat sets boundaries for the kitten yes and will will uh gently but firmly you know give it a bat with its paw and give it a little noise with its with its meow you know to let the kitten know that uh it's transgressed and this is not this isn't something that anybody had to train this is natural natural behaviors and and uh if the organisms were devastated by this we wouldn't have cats or dogs anymore because they wouldn't tolerate being around their conspicuous so these are not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of which is not just these treatments regardless of navigate through through all this correct and so taking like like blaming negative reinforce i shouldn't even say negative reinforcement but an aversive experience all together as something bad we it has such an important place as you said like we would all of us will we will not be alive if we don't know how to respond correctly to aversive and it doesn't necessarily always need to go wrong it's actually very beneficial for for any living i mean any simple cell organism will back off from from something that it alarms to danger correct yes and you know i think the uh

31:45

i think the objection uh by uh therapists and i suppose uh animal trainers as well i think the objection would probably be go something like this well the natural world is one thing but we're contriving these contingencies as part of a training process and we shouldn't do anything to our our students our pets our clients um uh that would would cause intentionally cause them distress and you know i i'm inclined to agree i think that you certainly don't want to create undue distress but there are many uh forms of negative reinforcement and punishment that don't really generate the kinds of byproducts that people are talking about or if they do they're mild and temporary temporary is a key key word yes yes yeah you know um uh there are many surprising results that that uh people are often unaware of in the category that positive reinforcement is good people overlook the fact that we have lots of research in the laboratory that shows that schedules of positive reinforcement have aversive character and there are portions of sessions where animals will reliably uh engage in a response to turn off the experiment they will actually act to escape from schedules of positive reinforcement this is something that's been known since the 60s although we know more about the conditions that generated today than we did in 1960 but a schedule of positive reinforcement um well you would think that that if you take a hungry pigeon and you put it on an intermittent schedule of food reinforcement that that would be the best thing and it would be a positive thing through and through and a stimulus correlated with the schedule would be a a positive condition reinforcer but it doesn't work that way ivan it depends on the context so if you have let's say a vi one minute schedule and it alternates with extinction a one minute schedule becomes a very positive thing and the stimulus correlated with it becomes a condition the punisher same schedule same schedule same schedule same stimulus same stimulus same goes from being a reinforcer goes from being a reinforcer to a punisher because of the context in which it's embedded so you can't just get tunnel vision and look at just one element and say well that's positive reinforcement and a stimulus correlated with it's going to be good because the context matters and it's so easy to lose sight of the context interesting and ultimately the way that the brain takes it the way the brain perceives it it experiences the same emotional and and uh uh uh unaversive response correct right and right

35:26

you know uh if i told you that uh i was gonna i was gonna hire you for a job next year and i was gonna pay you five million dollars i suspect you'd be happy but there are professional athletes all over the world soccer players baseball players football players basketball players if they if they were told they were going to make five million dollars next year they'd be very upset they'd be very unhappy yes because in their world that's uh that's uh an insult same amount of money different environmental context is five million dollars good or bad it depends right right and and then there is that that other element that sometimes actually not sometimes quite often if we keep if we stay stay outside now step outside the the skinner box but if we keep rewarding a behavior without trying to focus on how we can create whatever we need to do to make the behavior itself rewarding sometimes a very natural behavior that the person or the animal likes to do we can deflect it we can totally derail it but positive reinforcement expecting some some uh treat reward or money from for doing what they originally actually really love doing in dog training it's a big uh um um i i i am sure with humans it's the same probably the moment we start expecting okay you will pay me then how much you're gonna pay me and then you can as you said we become oh you're not paying me enough i i forget that i actually love doing this already without any reward uh at all correct yes yes well um you know uh sometimes it's argued that uh extrinsic rewards uh are maladaptive because they they take control over behavior that would otherwise be reinforced by its intrinsic consequences and i i guess that's possible but in a carefully crafted uh training program uh extrinsic reinforcers or contrived reinforcers are only used uh until uh the behavior comes under the control of its natural consequences skinner points out that um if you if you left the acquisition of reading to natural consequences it would probably never be acquired so you know reading has to be taught explicitly and reinforced with grades and praise and who knows what else and eventually through these contrived reinforcers and contrived contingencies the child develops a strong enough reading repertoire that they will come to read comic books or story books or novels or what have you instructions on how to build something things that where the reinforcer comes uh naturally out of the act of reading but to get there you have to use contrived contingencies and artificial reinforcers and and uh the the trick is

39:26

to know when to transfer control from one to the other you know once the kid starts reading superman comic books and enjoys doing it you don't pay him for reading that would be a mistake right you might you might you might you might have paid him to do his reading and phonics homework but once he learns how to read well enough that he can enjoy reading material you you leave him alone very true since we mentioned skinner we we can go in a little bit further with it because i mean obviously there there is so many things that he brought to the table that are so valid but at the same time i feel like there there was it's there is a lot of controversial stuff in dog training this this is one one thing i want to hear your opinion on um it's in certain circles of dog trainers it's very common very popular to do errorless learning and of course skinner presented at the time it was a big thing like i i've you know i the only exposure to it i have is youtube videos of of these machines that look like computers but you know and and basically go through the process but for some reason it never really took off and we obviously are not using this at school uh that that uh system but we are still talking about it and we're still believing that it's it's um there is a place and in and actually it can it can be just as good as trial and error how how i i want to hear your opinion on this well there's there's there's two at least two issues here one is um is uh is uh is errorless learning um possible uh and if so does it have an advantage over learning with with errors the uh the uh the the original or the classic experiment was uh herb terrace's doctoral dissertation and um he used different methods to train a discrimination a simple discrimination in pigeons and he showed that the pigeons that that uh acquired the discrimination with a very very small number of errors um were he first he showed it was possible to do that and secondly uh he showed that the stimulus that was correlated or well let me put it this way there were no negative behavioral byproducts of the errorless discrimination now later uh mark rilling at michigan state he and his students showed that uh he pointed out some of the confounded variables in terraces work and showed that um it wasn't the errors per se that that uh determined whether the uh there was going to be uh emotional byproducts in discrimination learning it was the timing of the introduction of the negative discriminative stimulus and so um um it was an oh it was a it was an over generalization uh to argue that errorless discrimination was inherently

43:11

superior now skinner skinner with his his um development of teaching machines uh i don't know if he was as concerned about errorless learning as he was about shaping up behavior in a in a way that um academic kind of behavior in a way that was uh that engaged the student and advanced in such small steps that the the uh student was unlikely to make an error and uh uh that's what skinner accomplished with his uh programmed instruction uh that he uh that he wrote that he wrote with um uh james holland uh and i think other people who've who've gone to the trouble of of uh creating a high quality programmed instruction have had similar results uh but i think it's a very difficult thing to do well and uh i'm not i'm not an educational psychologist and i don't follow that literature but i i don't think there's a lot of it um and i think it's because it's it's uh it's pretty labor intensive yeah and at least i mean to generate the it's labor intensive to generate the materials once they're generated everything's fine um i i i've like i've read some and i'm not sure like i i don't want to give the wrong references but there is a point where it does work to some point and it does build some confidence in the student that i can i am i'm good at this because we are avoiding the pointing the finger of no you failed correct but as it gets a little bit more complex then there is the need of ops you you went the wrong path well in in if if you look at uh at skinner's programmed instruction um he he's he's taking advantage of the extensive verbal repertoire and knowledge base of the student he's taking advantage of that but when you when you train a dog there's you don't have that kind of repertoire to leverage and in human work uh you're not going to create a skilled perform the skilled performance of a surgeon or a baseball pitcher or a sculptor or a painter i'm talking about an artistic painter you're not going to shape up that kind of behavior without lots of errors right no no you you show me somebody who claims that they can train a kid to paint like uh rembrandt uh right out of you know without ever making without ever producing junk and i and i will uh i will tell you that you're talking to a fool um so you know skilled skilled physical performances uh have to be shaped up in a in a different way than academic performances that are already building on on strong uh verbal and academic behavior yeah yes and then there is the the obvious difference between humans and animals that we can instruct and humans and it we really cannot do that with animals and then this is

46:57

where the averse aversion has a place because it's a it really goes back to that fundamental law what what do you think about where where how how far down you think the use of aversive or punishment or negative reinforcement where where exactly was that starting point that we said this is not a good idea and it's not it didn't not really even morally i think originally it was like it just doesn't work i i don't know if i'm right well i think uh i i haven't done a historical study of it but uh my i what i believe is that this is largely uh skinner's influence uh skinner was um a hedonist you know i say that in the most uh non-judgmental way he wanted he wanted people to be happy he wanted people to enjoy life and he saw the uh the route to that to be positive reinforcement and he pointed out some of the uh uh reliances that governments have on aversive control and he felt that these were the kinds of things that made people unhappy with their governments and generated rebellion and other forms of counter-control so he he actively argued against punishment as a social control measure and i guess by extension as a topic of experimental analysis um sidman uh murray sidman uh who of course is closely associated with the development of the free operative avoidance schedule later wrote a book called the coercion fallout and argued that any form of punishment or negative reinforcement was by definition coercive i think the influence of of of of skinner in general and then uh sidman's later uh writings in this area have done a lot to discourage the uh analysis of aversive control by operant psychologists by behavior analysts but there's a whole bunch of research on it by neuroscientists and cognitive psychologists so we've left it to them and the reason they're studying it is because it's clearly a part of the natural world and uh i think that uh perhaps skinner has a can make some valid points about social control mechanisms but when it comes to basic experimental analysis we want to understand all the world's processes and uh and and if these are processes that take place in the natural world we should be studying them yes you know a a new a physicist doesn't say i'm not gonna i'm gonna ignore gravitation because when people fall out of trees they hurt themselves right you know they study it because it's part of the world that's that's excellent that's brilliant no it's very true yeah yeah yeah um wow i knew we're gonna have a very interesting conversation nurture nature the endless debate

50:31

um there is a i don't know if you've it's it's kind of uh uh older book now popular book referred to a lot the talent code basically it refers to if you if you repeat something long enough you will get very proficient and and and you will overcome your genetic predisposition but irregardless of that um it's an endless never-ending debate and how how what is your take on genetic predisposition versus the environment of course they will overlap but i i i'm not gonna say anything more and let you talk well um i mean uh well as you know uh uh that functions as a reinforcer for a dog probably wouldn't function as a reinforcer for a uh first grade child you know so uh clearly uh uh the uh natural predispositions that uh must be uh uh the result of genetic heritage have to have to be obeyed and they do set limits on on uh the kind of repertoire that can be developed the the thing that i uh i like to remind students of though is that the organism's uh genetics is itself a product of the environment you know i tell the students that uh behavior is controlled by the environment and the environment has three aspects there's the contemporary environment what's happening right now there's the historical environment the experiences that the person has had in the past and then there's the ancestral environment the environment the environment that is one of the organism's ancestors uh that over generations uh selected the genetics the genetic heritage that this individual has today so when you say to someone you say to a child you hold up you hold up an object and you say what is this and they say apple well that what what's happening right now is the is this the this this object and the question what is this is this but the child can only respond to apple if it's had certain experiences including for example experiences in a verbal community that taught it english and in which it experienced apples but then there's also the genetic heritage that it had to have it has a certain it has a visual system to see the apple and an auditory stimulus to hear the question if the if the organism's ancestors had uh spent uh spent their time on the ocean floor it wouldn't have the that kind of uh uh sensory systems so all these things work together but inevitably they're all traced to the environment in a sense there is no nature versus nurture it's all it's all nurture right right that's not a it's a theoretical position it's not very practical doesn't help it doesn't help someone to train dogs i suppose well but it does give you a framework yes yes where it some sometimes where doc

54:14

dog trainers can get into get stuck i should say trying to let's say correct something appropriate in their mind behavior is that uh it's a very normal and genetically programmed behavior and if we continuously try to suppress it and it goes again that generic predisposition it will actually reach certain level of breakdown instead of finding a way to to accept that this is there and it's strong and looking for a way to just guide it into something else instead of really go against the genetic makeup um so that's a that's a common quite common in dog training especially with i shouldn't say young trainers but trainers that are in in beginning um and you know we as dog trainers we believe that we can control and we can control behavior no matter what and very often i think we go head on against uh something that it's selectively especially with dogs we we breed them selectively to do certain things like imagine if we decide that we have a hunting dog a pointer and we say oh we're gonna stop him from pointing uh it just as an example this will i mean this will burn the brain simply because the the dog's not even at a certain level not even consciously like it's not the the body that's doing it it's the it's everything programmed to do that behavior right right you might as well say we're going to stop the dog from seeing or stop the dog from hearing yeah it can't do those behavior you know it's interesting we never talk about about seeing and hearing that way you know we're gonna we're gonna train the dog to stop hearing but that's behavior it's just behavior that's so deeply embedded into the uh organisms of biological makeup that it never occurs to us to to try to modify it and it's a good thing too i don't think it would work very well right i had bearded collies and uh you know that's a herding breed yeah and uh i was pretty good at getting them to stop hurting two or three children if two or three children are running around in the backyard they wouldn't try to hurt them but if there were five or six i have to i couldn't stop i couldn't stop them yes and even even if we are able to to stop it at the moment we we are not taking it out of their uh um repair to our right the animal you what i i would i would guess that you're training the dog to uh uh i guess you could say actively suppress the behavior they're they're they're actively doing something that's incompatible you know i have a colleague here at the university you know actually he's retired now he's a geologist but he's a natural dog trainer and he he did this with

57:50

two different dogs so i know it wasn't a pure accident but he could take this dog and have it sit he'd say sit the dog would sit and then he would take a piece of cheese and he would put it on the dog's snout and the dog would sit there and he you could see the dog is kind of getting a little anxious and it's drooling it's drooling and then he'd say go and the dog would flip the cheese up in the air as it came down snap it right up he did this with two different dogs this is this goes against everything natural to this dog but he could the dog couldn't suppress everything it's clearly you know it's you can see it's struggling to stay still and it's drooling yes but an impressive performance nevertheless we get confused quite often when we talk about operand and classical conditioning and to a very basic level sometimes what can you tell dog trainers um like like the what are the different takeaways from from both how how do they how how are they different i don't know to you know to what extent um classical conditioning is involved in dog training i guess the most obvious case is clicker training yeah i guess the way what we when we talk about classical conditioning a lot oftentimes we include it we include that emotional aspect of the classical conditioning and and you know basically pairing the behavior with certain emotions and then we talk that there is well i guess you know um in in one of the things that we know about classical conditioning is that the uh the conditioned stimulus and the unconditioned stimulus have to occur in close temporal proximity and the uh condition stimulus has to come first and there has to be uh something distinctive about about both stimuli so i guess if uh if uh john doe what you know who's a tall man with a beard and wears a red cap uh walks into the dog training session and starts kicking the dog i think there's a pretty good chance that the next time john doe shows up the dog is going to uh run away from him that is that he will become a conditioned aversive stimulus by virtue of classical conditioning now what the most relevant aspect of john doe is might not be clear it might not have anything to do with the cap the beard or his height it might be a scent uh it might be his sound uh but um uh clearly that would create you know that that would be a distinctive stimulus a new stimulus and a new con a new pairing of the pain with that stimulus but if if john doe were part of the normal training of the dog part of the environment for months and then uncharacteristically engaged in this behavior it would be much less likely that he would

1:01:26

become a conditioned aversive stimulus so you know look if you uh if if you if you take the story of pavlov's dog uh you know pavlov when the what pavlov observed is when the uh the feeder would come into the uh the room where the dog was housed the dog would start to salivate because the feeder always signaled that food was forthcoming but the dog didn't salivate to the door to the site of the doorknob to the room yes because it was a static feature was there whether food was coming or not yes it wasn't the salad john doe is there all the time and then and then suddenly there's a pairing of john doe with with an aversive stimulus uh it's it's it's less likely that that effective classical conditioning will occur so i guess what that would mean is if if a dog trainer is doing you know a good job and uh is primarily associated with positive reinforcement uh an occasional mistake that might involve aversive stimuli shouldn't uh disqualify the trainer it shouldn't be it shouldn't create the any emotional response should be temporary that's what that's what the uh basic literature would suggest and and and and and actually goes further than that the the clinical literature on punishment such as such as it exists uh finds very little evidence of serious uh emotional byproducts of punishment when it's carried out by the same therapists who also provide reinforcement yes it's very i think this is a very important uh distinction to be made that the the being the person or the animal they have to be they have to relate and trust and and have some some relationship with the the person or trainer that does implement the punishment it's critical and then there is a of course a a place for a school or a conciliation if needed to where it's like no no i i really i don't hate you i don't want to kill you it's just don't don't do this right well i mentioned to you that i have a kitten and uh the the not too long ago this kitten was very small and it followed me around like you wouldn't believe and was would get right under my feet and one time i i took a step backwards and i stepped on the cat the kitten and this was a very unhappy kitten it hurt and and she ran away obviously but uh a few minutes later she was back and she was sitting on my lap and purring you know and uh i think if her entire experience was with me is that if i just walked into her life and stepped on her she wouldn't have come back yes but that was just one incident in what otherwise was a pretty richly reinforcing relationship because i fed her all the

1:04:53

time you know yes and loved her and so on yes yeah this this is something that somehow we we always in in dog training we kind of failed to to understand the importance of this like like if you i'll give you some examples we let's say a trainer gets a dog in training in like you know board and train and there is a behavior problem and very often especially trainers that are inexperienced they would rush into tackling that problem today the moment the dog comes in without having to build that trust and and like hey i i like you like we we actually you know and and then things even though we are suppressing behavior it seems like we're successful the dog has no nowhere to to go back to uh um what is the word i'm looking for to to to get up comfort of hey i i'm you are not against me as a being you're just not liking that behavior and and this is a i think it's a critical point when we well there is many many rules with punishment but this is one big one to to have a report uh to where the animal or the human really looks at you in a certain way before before we start something to do like that you know uh in his book on coercion and his fallout sidman says that a person who uses shock becomes a shock yes but it but it's not true it's it's only true if that's the only thing the person does um if it were true we would all hate our parents you know be uh or at least traditionally you know when i i grew up in the 50s and uh when you misbehaved you were spanked but i didn't stop you know i can remember my mother saying wait until your father gets home right classic but i didn't hate my father yes that is a very good point and another one he makes i i i will paraphrase because i'm not sure but it's the the the the person that's punishing basically gets this reinforcement and grat it's gratifying so so the more you punish the more you want to punish and that also i don't think uh he has any grounds well you know uh i if if uh i would think that that would be most likely if the um well if if the dog is misbehaving if you're a professional dog trainer and the dog is misbehaving and you do some sort of mild punishment procedure to to stop the behavior so you can move on with the positive elements of the training um it may be reinforcing to get the misbehavior to stop but that's not as potent of a reinforcer as it is when someone is um becomes really angry at some situation lashes out and then the situation goes away that's uh that's the kind of thing that becomes such a potent reinforcer that uh

1:08:46

uh uh you do worry about someone who will they're they're the first tool that they pull out of the uh toolbox is punishment yeah that's certainly not what not not what we want to see correct correct do you think certain people tend to like what what makes a certain person go that route instead of is it is it the upbringing of this person because a lot of times what we also would say well that person grew up in this kind of environment he he got all this so therefore he is now uh um you know yeah i mean you know there's there's an old expression uh uh it probably goes back to the 18th century spare the rod and spoil the child the uh the idea was that if you don't uh if you don't punish misbehavior you're going to have a rotten kid i guess and um we don't see things quite that way anymore we see punishment as something that uh should be used uh cautiously and selectively and only in the context of a uh an environment that reinforces appropriate behavior punishment by itself doesn't accomplish a lot except to stop i mean look uh if if if you were if you were if you were uh standing in your front yard and your small child suddenly darted out into the road your natural reaction might be quite frightening to the child and uh it would be a natural reaction and it would probably uh uh greatly reduce the likelihood that the kid runs out into the street again um that's a good thing right because you really have to have that kind of behavior stop but but in terms of but it doesn't teach the kid how to cross the street yes but that's why we have reinforcement that's right that's exactly right so uh punishment by itself isn't going to accomplish uh what you need to have accomplished it might accomplish certain things but it doesn't it doesn't train new behavior on the other hand um if you if you have a kid that runs out into the street a lot you're not going to have many opportunities for positive reinforcement delayed conditioning does it work and if it works when would that be like like well um in the laboratory my colleague down the hall andy letal has shown in a series of very elegant experiments that animals will learn they will acquire behavior under conditions of fairly long delayed reinforcement but but it takes a long time and the the uh the the end result is very small compared to what you would get with just a few immediate reinforcers um you know what happens in the short term triumphs over anything that happens in the long term uh so i would think that uh it would be fair to say that delayed reinforcement

1:12:26

or delayed punishment for that matter yeah yeah it doesn't work with with very long delays uh unless there's some sort of signal that connects the two i think that's what clicker training is about and you know if if uh if if if the dog does something and a buzzer sound a unique buzzer sounds for 10 seconds and then the food is delivered that probably will work fine but if the dog does something and 10 seconds later it gets the food that's probably not going to work very well during initial acquisition it might it might work okay in maintenance but when the behavior is is really weak uh as it is in the early stages of acquisition immediate reinforcement is um well it might not be the only thing that works but it's going to work better than anything else how how about the backwards conditioning like that that's always very difficult one to discuss at least in dog training if you look at if you look at backwards conditioning uh the you know the classic kind of backward conditioning experiments that uh are of the sort that that are reported in textbooks things like that you will see that occasionally backwards conditioning will generate some uh conditioned responses um or what count as conditioned responses but it's it's not a reliable effect and it's probably an artifact of some kind you know it's uh it's noise it's it's uh the the uh if you if if you want to generate uh classically conditioned responses the the conditioned stimulus has to precede the uh unconditioned stimulus even if the if they occur at the same time it doesn't work very well um one has to precede the other it has to in in the mathematical sense predict like the tone has to predict the food um if it if they come at the same time the tone doesn't do anything correct it doesn't predict anything but in fact it might not even be noticed by the animal because it it will be overshadowed by the food which is much more salient stimulus there were some studies and i i i'm so bad with references but there there was something to where where they were pointing out the backward conditioning could actually be could could work very well in case of uh um some predatory instincts to to where let's say let's say the something something happens to the rat and then the cat walks away and that is there is that genetic buildup that can create that association and i i i cannot think of the name of the the researchers um that it was very interesting and yeah i really don't know much like like to to most of dog trainers backward conditioning is not something we we rely or search on uh for

1:16:05

learning to to happen yeah you wouldn't program you wouldn't write a training program that relies on backwards condition right um yeah well the the thing that you're describing i can imagine that that could be the case um uh in the um in the 70s there was a lot of interest in um uh preparedness you know the idea that uh organisms are prepared to i wouldn't put it this way but it's a little bit cognitive for me but they're prepared to associate certain kinds of events and this was shown in a number of ways so just in the case of operant conditioning if you take if if you want to teach a rat to avoid shock and the response is jumping out of the box they'll learn it in two or three trials if uh because it's a natural thing for the animal to do is to try to move physically move away from the source of aversive stimulation if you require the rat to avoid shock by stance staying in place and pressing a lever it takes a lot longer and the idea is that the animal's prepared to learn one way rather than the other uh another example is with uh condition taste aversion yes john garcia's uh experiments you give the you give the rat uh a distinctive flavor something it's never tasted before and you follow that up with something that makes it sick well they'll after just one trial they will avoid that flavor but if instead of a flavor you use a visual stimulus like a flashing light or a sound they don't avoid that so they it's the the choice of the of the discrimin the choice of the condition stimulus isn't purely arbitrary it depends on something that the the rat is bringing to the laboratory something about their genetic heritage so you know in light of that i can imagine there could be situations where um very close backward conditioning might might be effective but it still wouldn't be as effective as forward right so my next one is very big one learned helplessness and it is a very like it's a very common very often brought up in the dog training industry to where like if we that it's very easy to reach that state and there is no way out and and it's there forever i have opinions i've i've read different uh um studies i know i know that even mayer right after like two or three years after the original study they they did a follow-up studies to where they exposed the animal and they taught it how to escape and avoid and they showed much more resistance and resilience to to be coming into that uh uh mental state to giving up but i i really want to hear your take on this one it's a very big one in the dog training community

1:19:40

is it is it is it a big issue because people see this uh phenomenon interfering with their training no it's almost fictitious it's almost like no no if if you're doing certain things you will you will like if you use some form of aversion in in the training it you're really risking the dog to go into learning helplessness therefore you should avoid it at all costs that that's the the narrative i was uh i was in graduate school when uh seligman and meyer were doing that original research and one of the striking things about it remember the original research was done with dogs and uh they were using such severe electric shock applied as if i remember correctly to the pads yes that it it was such a severe shock that it produced projectile defecation so you know this wasn't the run of the mill kind of uh aversive stimuli that are likely to be used in by a professional dog trainer this was uh this was really very special extreme intense uh stimulation now i'm not an expert it's been a long time since i've looked at that at that work but that's what uh struck many people about the original research i don't think that uh you know in in the um in the it's it's it's interesting in the human uh applied literature when uh punishment was was more was viewed as more acceptable the issue was never uh reducing learned helplessness the issue wasn't a concern with learned helplessness it was a concern with having a a punisher that was effective in suppressing the bad behavior um it was it was it wasn't like oh if we're not careful all behavior is going to stop it doesn't seem likely in a in an environment where punishment is a very small component of an otherwise reinforcement based procedure and that's not what was going on in those experiments and learned helplessness not like the animal was uh like this procedure was embedded in a uh in a therapeutic environment it wasn't that at all it was yes purposefully yes yes you know it was not translational research right it was it was basic research it wasn't it it really um um i think i think the cons the the applied concerns that are generated were probably overstated okay you don't hear much about it anymore i mean it didn't we are actually we you will be surprised it's it's okay you may hear a lot yes in the dog industry the dog training in uh communities it's in the top of the list in certain really yes yes it's a very it's a it constantly is brought up um and and yes if we do if we if we do what you just said of course we're gonna have some extreme results um uh but yes it is a

1:23:20

um it's one of the the cards that could be played to to to convince somebody no no you will this is well documented and and this is where you're headed you will you will create uh learn helplessness and the dog will just basically never try to avoid uh and and will give up and there will be no way out for for the rest of its life uh it's a very scary tactic in the literature that i read i don't see much uh reference to it um uh maybe there are literature i'm talking about basic sure uh operant literature um but you know it does remind me of a procedure that's commonly used in uh the neurosciences and that's uh the forced swim test and the way this works is you take a rat or a mouse you take a rodent and you put it in a tank of water warm water so it's not a shock or anything like that and what is what is the what does the rat do it does the dog paddle you know it treads water and after a while it stops now when it stops it doesn't drown it floats and this is commonly the time it takes for the animal to give up the way they would put it is give up is is supposed to be uh a sign of how long it takes for the how depressed the animal is likely to be right and and giving up is a sign is is is taken as a sign of depression depression but of course the alternative is that the animal has learned to float it's learned to stay on top of the water with much less effort and there's a lot there's a there's a lot of people around that i talked to in in our health sciences center that see it that way they say i have to use this in my research because the grant agencies expect it but i think it's ridiculous because it's not a good model for depression it's a good model for learning for learning so is it is it i mean there's it's like learned helplessness well no it's not it's learning how to float finding answers to the puzzle exactly right michael anything that you want to talk about because i know that i i am asking a lot of questions but i know that there is probably something that you anything that you feel that it's interesting and related to dogs and it will benefit dog trainers from your experience of of i i don't i don't know i don't know that i have anything uh worthwhile to say to dog trainers uh because i'm just a rat psychologist you know i uh i just study basic principles and in most cases we're not uh so in the basic principles with rats what are the fundamentals that need to be there in order to be able to continue with with experimenting with the rat and not not uh not have the rat being scared to do things for you and and

1:26:52

have these wrong responses well you know uh when when we teach a rat how to avoid electric shock one of the things that we do is we are very judicious in the number of times we deliver a shock shock it's a it's a unique stimulus it's not something the animal encounters in nature and um it clearly frightens them they uh they will uh they will whine they will defecate and urinate and uh that happens when they get their first first or second shock they will we'll see that kind of emotional reaction but within a few hours of training over a period of two or three days all that goes away as the animal acquires effective avoidance behavior and uh and after you know a couple weeks of training you know a couple hours a day five or six days a week what you see is a completely relaxed animal they uh they just don't uh there's there's nothing in their overt behavior that would suggest that there are uh that much for life yeah right yeah oh absolutely not i mean they live they live as long as uh as as the rats and the food experiments do you know i've never i've never been bitten by a rat in a shock avoidance experiment i have been bitten by rats in food experiments food reinforcement experiments doesn't happen very often it happens occasionally and i think it's because the animals um are uh are food deprived and uh that generates a certain degree of agitation when they're when you start handling them and it it sort of predicts food so they start protesting the extinctions yes yes yes it's it's not it's not really aggression as much as it is uh uh confusion about uh what it is that's out there to eat um but you don't see that with the animals in the shock of wind studies they're pretty laid back they're they're uh they're pretty friendly yes because one of the big one of the big arguments and then i mean skinner had quite a few arguments against punishment but one big one was the the so-called counter attack the response of the animal to to punishment how how what about i know you've done some work with uh experimenting and and studying timeout um how how how is timeout affecting so so let let me let me back up here we in dog training world certain trainers believe that timeout is actually very benign and we can just use it almost any time at anything and it's the go-to approach if we have to do positive punishment versus timeout what is that correct statement well uh you know my students and i uh within the last year published a uh a paper that had seven experiments concerned with timeout from positive reinforcement all in all in rats and um we included

1:30:35

a uh within those seven experiments one in which we ran the procedure but instead of using timeout punishment we used electric shock punishment and timeout in in our hands in the rat lab works pretty much like shock punishment does um we don't see emotional byproducts in either case but but they're both effective in suppressing behavior now in the laboratory you don't want you want to use a mild enough punishment so that you don't completely suppress shut down because you do that you have nothing to study so you we look we try to calibrate the stimuli so we get sort of an intermediate level of suppression uh but timeout from positive reinforcement is uh is a very effective uh punishment procedure and uh it's a punishment procedure you know people can call it whatever they want but it's negative punishment yes and uh it is aversive yes it's aversive and and and the uh uh it's hard to see it in pure punishment studies but the richer the time in environment you know the richer the the richer the environment is then the more aversive a timeout should be now that's hard that's hard to see in a in a in a punishment study for technical reasons so we did some in this set of experiments we did some avoidance experiments where the animals avoiding timeout what we did was we we put the rats in the chamber and they just got every once in a while on a variable time schedule they got free food but if 30 seconds passed without a lever press they got a timeout timeout also happened to last 30 seconds so under those circumstances the rat learned to press the lever to avoid the timeouts and what we were they didn't have to press to get the food the food's free they had to press to avoid getting timeout stay in that's right and what we did was across conditions we changed the rate of the free food deliveries and it turns out that if the rate of food deliveries is very high it's very dense there's a lot of food time the time in is really good then they will work harder to avoid the timeouts than if the food is relative the food's uh deliveries are relatively uh sparse so the better the the world is the more aversive the timeout becomes um it's kind of interesting because giving more food makes the world a better place and i guess it does but it makes the time out much more aversive so timeout from a rich a richly rewarding environment should be a more effective punisher than timeout from a uh of a um a less rewarding environment but it's it's really interesting the number of people who i've heard over the years uh like daycare centers they'll say we don't use punishment we use timeout

1:33:51

of course that is punishment yes it's just playing words like playing yes yes it's negative punishment it's just like it's like taking the car keys away from your adolescent son because he broke curfew you know that's the the son will tell you he's been punished and uh and he has but it's it's time out from the from access of the car what about the duration of timeout and what about how when do we because there is that risk always of bringing the animal back in when it's not the the proper demotional state right we have not we have not studied that in my lab we've only studied the uh uh we've only we hit on a 30 second time out because 30 seconds the literature suggested that that was long enough to be effective and uh and it was in our experience so we didn't fiddle with it because we weren't interested in that variable but that is a variable that's well worth studying you know there's all these rules of thumb that are out there in the uh applied literature about how long a timeout should be depending on the age of the child but there's uh very little empirical uh evidence to support those claims my guess is that a time it's a guess uh is the timeout uh can be relatively brief and still be pretty effective and then we have the the few different types of timeouts to where i i mean obviously probably with the rats it's a little bit different but let's say in dog training we can take the dog and put it all the way away in the crate in the car out from the field or we can keep it present kind of like a hockey game and you get a penalty but you're actually on the bench watching versus being totally taken away and removed from the game yeah you're not sent to the locker room right and and they have they clearly have different effect on on um i think the in a lot of cases that that timeout that you you're still there but you're not allowed to participate um can be more effective than completely taking away and then the dog or or whatever the animal is it's almost at the point that it forgets i i was just put away there is not really association anymore of of why am i coming back on the field now is it just another session or is it uh we are back from penalty that's interesting uh you know that's uh that's a very interesting observation that had that had never occurred to me and i have to think about whether there's a a way to mimic that sort of thing in the laboratory see if we can model that in the laboratory i have to make a note of that after we're done that's uh that's very interesting indeed

1:37:00

yes hadn't thought about that one that's that's a good one okay michael i think we we i mean i would say excellent conversation and i i'm sure this gonna create a lot of conversations among the dog training community like i i didn't know where we're gonna go with this but uh i'm i'm very happy of of everything that we covered it's it's very interesting and i cannot thank you enough for for accepting that my pleasure i've i've thoroughly enjoyed myself and you've left me with a uh an idea to uh talk about with my students see if we can design some new experiments yeah any anything that you may need dogs for experiments or ideas we're here they are uh they are very expensive animals to have as subjects yes dogs are but everybody when you say dogs and research there is a the the interest goes off the roof immediately because it it's a special place with the dogs yeah well it would be uh would be something to think about you know you there probably would be uh uh dog owners in the in the university community that would be happy to uh lend their dogs to us for a few hours and we went to their homes well i like to try it with rats first right that's your yes yes of course i mean for for whatever reason or pigeons you know yes for whatever reason that's where it starts right it's uh well they're so convenient yes they live right there i just have to walk down the hall and there they are thank you really like i cannot thank you enough for for coming to the podcast and as i said like it it will be this will be watched over and over from from a lot of dog trainers because i i i will personally watch it myself because it's different when i'm talking and listening to where i can just sit back and listen to you talking about all this stuff it was very very interesting and you're very thank you very kind thank you thank you so much so so so Thank you.