Final Thoughts

I thoroughly enjoyed training Lil Sebastian. I thought it was entertaining to use the operant boxes and see Lil Sebastian's progress due to my own input.  Probably my favorite thing was shaping Lil Sebastian and seeing her become attached to me.  At the beginning of the process she was very jumpy and skittish with me.  But by mid training, she was much more calm when I picked her up and would even take food out of my hands which I thought was cool.

I think the current process of how to operant train is pretty good, especially as a instructional tool prior to our own project which we will be starting in a couple weeks. I will probably continue to work with Lil Sebastian for my second project if my partner doesn't take her first.

The most surprising thing for me was how easy shaping was.  I may of been pessimistic about the whole process and probably doubted myself a little too much, but I thought that the shaping process was going to be much more difficult than it was. I was surprised to see how quickly a little rat could catch on to some things.

On that note, I definitely overcame a misconception that rats aren't very intelligent, but actually they are pretty smart. Knowing what some other people have done in the past with their rat is impressive that a rodent can pick up on such outlandish behaviors such as painting or dunking a marble through a hoop.

I look forward to continuing to work with Lil Sebastian, or to train a new rat for my second project!

Live vs. Virtual Rat

Some of this will reiterate what was said in my previous Sniffy post, but I will answer the questions specifically to ensure everything has been covered.

Shaping for my Lil Sebastian took 2, 30 minute training sessions. Whereas shaping for Sniffy only roughly took 30 minutes.

Below is the cumulative record of Lil Sebastian's FR1 Schedule

Below is the cumulative record for Sniffy's FR1 Schedule

There are small differences between the two cumulative record for the live and virtual rat.  As you would probably assume, the virtual rats cumulative record is much more consistent and "ideal" with responses occurring much more steadily. On the contrary, the live rats cumulative record shows more variability and more breaks in responses, which should be expected since the live rat isn't programed to respond.

There were not many differences between magazine training the live rat and the virtual rat.  They both took roughly 20 minutes, and it simply was presenting the food when they were not focusing on the hopper.

Shaping the live rat was more difficult than the virtual rat.  The virtual rat is programed to eventually respond correctly, and timing was less sensitive. Though sometimes the program would not respond timely enough to reinforce Sniffy. Obviously a live rat has a temperament and different moods and can be satiated, and Sniffy was not effected by these factors.  A multitude of different behaviors are also present at one time with a live rat, whereas Sniffy only has behaviors that were programed for him, so they are more limited.  The difficulty really arose when having to use proper timing with Lil Sebastian's reinforcements.  Lil Sebastian would also be more susceptible to distractions, which was not a concern with Sniffy.

I learned from Sniffy the overall outline and steps you should use when training a live rat, which was helpful when shaping Lil Sebastian for the first time.  But I don't believe you can actually understand the sensitivity and complexity of shaping a live rat without actually attempting it.  There is just so much variability that goes into shaping a live organism which taught me to fully respect trained animals and their owners for the time they dedicated to training them.

I think the Sniffy program was helpful, especially since the majority of the class probably had not had experience with shaping or training an organism to do something in a lab setting.  I think actually getting to train a live rat is an awesome experience. It makes learning the concepts in class much more rewarding because we can see what we're talking about in the lab.

Responses Chart and Weight Table

Below is the bar graph and table for the average number of responses of Lil Sebastian per 30 minute session for each FR schedule:

(Click to enlarge)

Finally, here is Lil Sebastian's weight table and how much we fed her each day:

TARGET WEIGHT: 204 g

Extinction

My partner and I decided we would do extinction over two days, and we started extinction on training day 13. Extinction is extinguishing a shaped behavior a organism has learned.  This is done by simply not reinforcing any behavior whatsoever.

The first day of extinction was pretty predictable.  Right away Lil Sebastian went straight to the lever and started pressing it.  She pressed it 18 times before stopping, then wandered around, and then going back to it.  She didn't check the hopper for a pellet until 2 minutes and 20 seconds had passed.  After this, Lil Sebastian definitely was performing an extinction burst. After 5 minutes had passed, Lil Sebastian had lever pressed 142 times, with no reinforcement and only checked the hopper about 3 times.  The number of lever presses slowly decreased every 5 minutes after that, and Lil Sebastian's erratic behavior increased. This was displayed by long periods of her completely focusing on the lever trying to get food, and pressing it many times, then she would seem to give up.  When she wasn't focusing on the lever, she would spend long periods wandering around, rearing up on back legs a lot more than usual, and sniffing or cleaning. Below is a video of her behavior when focusing on the lever.

Here is another video of her typical behavior during extinction while not focusing on the lever.

 

Below is the cumulative record for extinction day 1.

(Extinction Day 1)

Lil Sebastian did display some emotional behavior by jumping up on the wall for a brief period and touching the ceiling of the box with her nose.  This had not happened before. At the end of the first day of extinction, Lil Sebastian had pressed the lever 301 times, and the number of lever presses for every 5 minutes can be seen at the end of this post.

The second day of extinction was much of the same behavior as before.  We did not observe spontaneous recovery because Lil Sebastian was pressing the bar just as much as she was at the end of the previous extinction day.  She was much less active this day, and instead of moving around the box a lot like the day before, she would sit in the middle of the box and sniff or clean often.  She also spent a significant amount of time sniffing and chewing on the floor in the front left corner, which has been a common behavior throughout training.  She displayed another emotional behavior also. She climbed up on the wall of the box in the front right corner 3 times in a row and was able to stay on the wall for up to 20 seconds one time.  Below is a video of this behavior.

Below is the cumulative record for extinction day 2

(Extinction Day 2)

At the end of the second day of extinction, Lil Sebastian had lever pressed only 67 times, which was a huge decrease from the day before.  Below is a table showing the number of lever presses for every 5 minutes for each extinction day.

Problems with Shaping

My partner and I luckily did not face too many problems, and the most prevalent ones have already been discussed in previous posts, but I will review them here:

1. During magazine training and shaping, I messed up with timing the reinforcements periodically, especially at first.  This caused problems with Lil Sebastian learning the correct behavior, and resulted in a multitude of behaviors instead of a desired one (attention to the lever).  Eventually I got better with timing the reinforcements appropriately, and Lil Sebastian became less confused and learned the behavior appropriately. Unfortunately I do not know of a way to avoid this problem without simply practicing it with Lil Sebastian, because I did improve my timing with time and practice.

2. Another issue my partner and I had was Lil Sebastian was overly interested and preoccupied with the front left corner of the operant box (box #2).  We learned from Madeline who used the same box last year, and from other experimenters using our box that their rats had the same issue, and it was disruptive to the learning process.  Luckily, Lil Sebastian's interest in the corner dwindled less and less as the FR schedules moved forward. None the less, during magazine training and shaping this was a nuisance. For whatever the reason, Lil Sebastian would rather stick her nose in that corner than do anything else for moderate periods of time. Below is a video displaying this weird behavior:

My partner and I thought there was something on the floor of the cage, or in the chips.  So we cleaned the chips out, but Madeline said she cleaned the whole cage last year and it did nothing to her rats corner obsession.  So unfortunately this annoyance was simply endured during training periods, but luckily Lil Sebastian became less interested in it as the training sessions went on.  The only way I can think of to get rid of this problem would be to train in a different box or get a whole new box, since this behavior is universal for everyone who used this box.

3. One of the earlier training days, pellets were not being dropped in our hopper.  At first I thought I set up the program incorrectly, but then noticed the food chamber was releasing pellets.  I simply slid a pipe cleaner down the chute of the hopper, and many pellets were released at the bottom.  After cleaning them out, we started over with our training for that day and this never occurred again.

4. Keeping Lil Sebastian's target weight proved to be difficult task throughout training. It almost seemed she would just gain or lose weight at will, regardless of how much food she ate.  Her weight fluctuated really inconsistently.  Eventually my partner and I disregarded this problem and simply fed her a consistent amount each day because it did not seem to disrupt training sessions very much after we recognized the issue.  Lil Sebastian still seemed motivated enough on each day. Her motivation only seemed to be effected by our last problem discussed below.

5. The last problem my partner and I had was we struggled to keep the same training time for every day, though the majority of our training sessions were at the same time of each day.  We are both very busy and our schedules change frequently day to day so sometimes we would have to come in much earlier than planned to train. This clearly confused Lil Sebastian sometimes and those days tended to be less successful than when we were more consistent with our training time in successive days. After the first week we recognized this problem and tried to keep our training sessions no earlier than an hour and half before our regular time. We could of simply scheduled a better training time, but unfortunately there is not a single time for each day of the week that does won't interfere with both our schedules.

Fixed Ratio Schedules

After training day 4, FR1 was established.  The next step in Lil Sebastian's training was to achieve the highest partial reinforcement schedule possible to ensure the learning of lever pressing even more.  The highest FR schedule Lil Sebastian attained was FR12, and this was followed by extinction which will be touched on in another post.

FR2

At the start of training day 5, Lil Sebastian started pressing the lever right away. At the beginning, there were long pauses between lever press 1 and lever press 2.  Lil Sebastian would press the lever, check the hopper for a pellet, then not finding a pellet would sniff around eventually going back to the lever and pressed it again, finally revealing a pellet.  This persisted for a little while, but slowly these long pauses diminished. Lil Sebastian seemed to get frustrated because instead of just pressing the lever twice to receive a pellet, she would press the lever 3 to 5 times in succession, then check check the hopper for her reward.  This clearly made it difficult to distinguish when exactly the pellets were distributed.  Eventually after 16 minutes, Lil Sebastian started to only press the lever twice in succession for her reward, which must of meant she recognized that only 2 presses were necessary to receive a pellet.  At the end of the 30 minute session, Lil Sebastian had pressed the lever 320 times,  being reinforced 160 times.  Below if the cumulative record for training day 5 (FR2).

(Cumulative record for FR2)

Below is a video of Lil Sebastian on the FR2 Schedule

FR3

The goal for training day 6 was to establish FR3 since FR2 was mastered the previous day. At the beginning of the session, Lil Sebastian was easily distracted from the lever.  She would focus an amount of time on the lever, then would get distracted and walk away from the lever to clean or sniff around for a while.  This type of behavior occurred for 15 minutes until finally she focused more on the lever and started to press it in succession, recognizing the number of lever presses mattered to receive food because she did not check the hopper often between lever presses.  Unfortunately this did not persist for very long and around 20 minutes, Lil Sebastian was distracted from the lever again and showed even less interest in the lever. Lever presses were far and few between in the next few minutes, so I decided to end the session at 24 minutes.  At the end of the session Lil Sebastian had pressed the lever 237 times, and was reinforced 79 times, which was significantly less than the day before. Below is the cumulative record for this day.

(Cumulative record for FR3 [Day 1])

Training day 7 was to solidify FR3 since the previous day was not as successful as desired. For the first 5 minutes of the session, Lil Sebastian's distracting behavior continued from the previous day.  She spent a lot of time pawing the lever and not pressing it, then leaving it to clean or sniff around. After this unpromising start, Lil Sebastian started pressing the lever a lot more, with few pauses between presses.  She would use her head to press the lever often, and it seemed like she was trying to smell what was outside through the small crack once the lever was pressed.  Because of this, Lil Sebastian would very often press the lever very quickly in succession without letting off the lever. Towards the end of the session, Lil Sebastian would get distracted for about a minute at a time after pressing the lever for multiple reinforcements.  We ended the session after 28 minutes and 56 seconds and Lil Sebastian pressed the lever 286 times and was reinforced 95 times.  Since the distractions were less prevalent this day, we determined FR3 was solidified. Unfortunately we have lost the cumulative record for this day, or it just didn't properly save to the thumb drive.  I do have a video of Lil Sebastian lever pressing on a FR3 schedule below though!

FR5

The goal of training day 8 was to complete FR5.  At the start of the session, Lil Sebastian immediately went to the lever and started pressing it, checking hopper after each press.  After about a minute, she started pressing lever in succession.  Lil Sebastian still bar pressed with her head often, and would still paw the lever without pressing it sometimes.  She would have periods of continuous bar pressing for reinforcements, followed by periods of climbing up the wall more often than before.  None the less, this session was much more successful than the past two days because Lil Sebastian was more focused on the lever and her periods of distraction were shorter resulting in a very productive day.  The prior day we fed her less after training and this may have caused more motivation for this day.  At the end of the 30 minute session, Lil Sebastian had pressed the lever 535 times, thus being reinforced 107 times. Below is the cumulative record for this day.

(Cumulative record of FR5)

FR7

The goal of training day 9 was to establish FR7.  At the beginning of the session, Lil Sebastian went straight to the lever but did not press it.  She spent a lot of time at the beginning of the session exploring and cleaning.  After 2 minutes, Lil Sebastian finally pressed the lever 3 times, then checked hopper, then continued pressing lever.  She would still press the lever with her head like the previous days.  After about 7 minutes, she no longer checked the hopper for the pellet between lever presses, and would only approach it after a pellet was presented.  This leads us to assume Lil Sebastian had finally formed an association between the secondary reinforcers (light going off, hopper light turning on, clinking noise of pellet) and the presentation of food. This was a very good sign that training was going well, but despite this, Lil Sebastian slowly was showing less interest in the lever as time past, and her sensitivity towards these secondary reinforcers was diminishing.  My partner and I determined that she must be satiating because we did have our training session a hour and a half early today.  After 20 minutes, we ended the session with the intention of solidifying FR7 the next day.  Lil Sebastian had pressed the lever 350 times and was reinforced 50 times. Below is the cumulative record for this day.

(Cumulative record for FR7 [Day1])

The second training day of FR7 showed much more improvement than the previous day.  From the beginning of the session, Lil Sebastian had more motivation because she immediately focused on the lever and pressing it. Her sensitivity to the secondary reinforcers were maintained from the previous session, and she would only check hopper when there was a pellet in it.  She would still press the lever with her head instead of her paws sometimes, and she jumped up on the clear wall with all four paws twice towards the end of the session.  Was much more active today.  At the end of this successful 30 minute session, Lil Sebastian had lever pressed 579 times, and was reinforced 82 times. Below is the cumulative record for this day.

(Cumulative record for FR7 [Day 2])

FR10

The goal of training day 11 was to establish FR10.  Lil Sebastian's behavior was similar to the past few training days.  She was still pressing the lever with her head sometimes, she was responding to the secondary reinforcers, and she would sometimes have periods of exploring or cleaning between multiple reinforcements.  At the end of the 30 minute training session Lil Sebastian had pressed the lever 530 times and was reinforced 53 times.

(Cumulative record for FR10)

Below is a video of Lil Sebastian performing on a FR10 schedule.  In the video is a visual of her pressing the bar with her head, and her only approaching the hopper after responding to the secondary reinforcers.

FR12

The goal of training day 12 was to complete FR12.  This is the last day of training Lil Sebastian.  She immediately went to the lever and started pressing until was reinforced.  As the session went on, she slowly started to not respond to the secondary reinforcers like she has been in the past few days.  She would check hopper before pellet was presented and was distracted more than usual between lever presses.  Another weird event was she was much more interested in the front left corner of the box, like in the beginning of the shaping process.  Below is a picture of this behavior.

We had started this training session about an hour and half earlier than our normal time and we concluded that this unusual behavior was due to the earlier start time.  Lil Sebastian still pressed the lever 496 times, and was reinforced 39 times after the last 30 minute session. Below is the cumulative record for this day.

(Cumulative record for FR12)

Final Thoughts

At the end of the FR schedule process, I would say the biggest change of behavior throughout the different schedules was that Lil Sebastian started responding more consistently to the secondary reinforcers towards the later training days.  During the first few variable schedules, she would check the hopper for food regardless if the secondary reinforcers were presented or not. Besides this change, a lot of her variability in behavior throughout the process seemed to be due to my partner and I struggling to stick to the same training time every day.  I'm sure Lil Sebastian's motivation each day would of been much more consistent if we could of been on a more consistent time schedule each day. Also, Lil Sebastian's target weight was difficult to maintain, and we ended up never really maintaining it very close.  Luckily, I do not believe this has affected our training in a negative way, considering Lil Sebastian still was able to perform satisfactorily the majority of the days.

Finalizing Shaping and FR1

The goal of the next training session was to solidify shaping and to smoothly move to FR1 almost simultaneously.

Day 3 of training, Lil Sebastian started much more positively than expected. She immediately went to the lever and propped her paws on it.  She was only reinforced for propping her paws on the lever, and for actually pressing the lever, which was set up to automatically deliver a pellet if this event occurred.  Lil Sebastian was easily distracted and would have pauses between getting reinforced at the beginning of the session.  After 16 minutes, Lil Sebastian ceased from her distractions and finally pressed the lever.  From then on out, almost all her time was spent near the lever, or pressing the lever.  At the end of the 30 minutes, Lil Sebastian had pressed the lever 46 times, so clearly an association between the lever and the presentation of food was formed. I reinforced Lil Sebastian 43 times. The cumulative record for this day is below:

(Cumulative Record for Training Day 3)

On Day 4, we loaded up FR1 to assure Lil Sebastian's association with the lever pressing and food before starting on FR2.  Initially, Lil Sebastian spent time roaming the box and only pressed the lever intermediately.  After 5 minutes, Lil Sebastian focused most her time on the lever and pressed the lever frequently, with only small pauses between presses.  For the next 10 minutes, lever pressing continued uninterrupted and I felt confident FR1 was established.  Near 20 minutes into training, Lil Sebastian started to become less interested in the lever because cleaning, and chewing the bottom of the cage became interchangeable with lever pressing.  Because of the loss of interest in the lever, I ended the session early around 21 minutes.  Lil Sebastian had pressed the lever 100 times, and I reinforced her for only 15 times.  Below is the cumulative record for FR1

(Cumulative Record for Training Day 4 [FR1])

Shaping obviously blended into FR1 a bit, but I think this was good because it made the learning process more seamless. Below is a video of Lil Sebastian lever pressing on FR1 schedule!

Shaping

Once magazine training was established, the next goal for Lil' Sebastian to accomplish was shaping for FR1 lever pressing.  Lil' Sebastian would be shaped to lever press for sugar pellets, in order to eventually learn on different reinforcement schedules.  The reinforcement program was booted up on "Shaping 1 lever" in order to manually present the pellets with the button and pellets would be released automatically when the lever was pressed.

On the first shaping day, the original plan was to only reinforce Lil' Sebastian when she was near the lever and props up on the lever wall.  This happened less than expected and Lil' Sebastian was not being fed frequently enough.  Thus, when food was presented in the hopper, Lil' Sebastian did not approach the hopper immediately, and when she did go to the hopper the reward was not correlated with the appropriate behavior.  She would chew the floor for a moderate amount of time, clean herself, and "explore" while there was food in the hopper from appropriate behavior in the past.  This resulted in Lil' Sebastian being confused about what behavior was needed to receive a sugar pellet.  So in order to refresh Lil' Sebastian's responsiveness to the food in the hopper, I started presenting food more often and basically started over with magazining again.  After 20 minutes, she became much more active and would move around the box quicker and would check the hopper more than before.  After she ate her sugar pellets, she would keep her head in the hopper and sometimes would "dig," as if more food would appear.  At the end of the 30 minutes, I was able to restore her responsiveness to the hopper in order to get her to bar press the next day.  Was reinforced 64 times and never lever pressed.

The second shaping day I initially made sure Lil' Sebastian was responsive to food in the hopper.  Fortunately this was true so the goal was to get her to lever press.  I started only reinforcing her when she was near the lever.  Thankfully she still seemed to be highly active from the first shaping day, so getting her to stay near the lever was not too difficult.  She continued to be overly interested in the hopper after she received her pellet, and her preoccupation with the empty hopper interrupted progress for the first 15 minutes.  Luckily, this behavior lessened and she began to prop herself on the lever.  I of course reinforced her for any touching with the lever, then only for putting her paws on it, which became very associated with the presentation of food.  At the end of the 30 minutes, she had pressed the bar once so the goal was completed and I was confident FR1 could be established the next day.

Below is a picture of Lil' Sebastian propped on the lever:

This shaping experience highlighted the importance of timing of reinforcement as we discussed in class and as the book notes.  Lil Sebastian had a hard time learning to associate the lever with the food because the timing of when she was receiving the food was much later than when she performed the appropriate behavior (rearing up on lever wall, touching the lever, etc.).  A lot of the struggle with Lil Sebastian's shaping experience had to do with my own timing errors. So due to my own errors, Lil Sebastian did not learn lever pressing very easily.

Below is the cumulative record of the shaping process for day 1 and day 2:

(Cumulative Record for Shaping Day 1)

(Cumulative Record for Shaping Day 2)