Problem Statement

This project inspiration comes from wanting my dog to stay off the couch but my sneaky dog wanting the opposite (spoiler alert: I win). And yes, don’t worry, she has a perfectly comfortable bed to sleep in, but of course that’s not enough for her.

Take 1: Stream a Feed of the Living Room

The initial thought was to get a live stream of the living room. If I saw her on the couch, then I would go and tell her to get off of it. While this helped for me to know when she was on the couch when I wasn’t in the room, this didn’t actually solve the problem. She learned over time that she could get around this: it’s just a “waiting and staying alert” game. She would wait 15-20 minutes until the coast was clear, then hop on the couch. Once she heard me coming, she would hop off and act like she was never on it (the warm couch cushion and video surveillance would tell me otherwise). I knew this wasn’t the right solution because my dog should not be smarter than me! With the technology I have available to me, there’s got to be a way to figure this out… So, I moved on to my next idea.

Take 2: Object Detection

I realized I needed something to close the feedback loop. Something that handled the problem on its own and ran without me being present or monitoring a screen. Insert vision AI object detection!

I used a Raspberry Pi 4 to do this using the project yolov5 on GitHub as a starting place. I cloned the repository and installed the required tools and libraries. I got a basic version of the object detection program working without writing any code because a dog and a couch were both objects the existing model in the repository recognized. But there were some gaps that needed to be filled before I could use it the way that I wanted. The first problem was with my camera: it only worked in well-lit areas. I needed a camera that would work at night. Once I got new hardware (a camera that switched between day and night modes), I realized the prebuilt model doesn’t work as well with infrared images. So, I needed a new model too!

I took some pictures of the couch and of my dog at various angles and lighting conditions, and I trained a model based on those images. I ended up renaming the dog label to Jenny because that’s our dog’s name and why not? The detection piece was now in place!:

Now, I had to piece together the detection location and an action. I used the bottom of each detection box as its position. If both of these objects were detected in a frame and the dog detection box was above the couch detection box, I assumed Jenny must be on the couch. Next, I needed something that would prompt her to get off the couch. I wrote a function to play a sound that would be called if she was on the couch (man shouting “No!” sounds assertive enough to me!). I added this logic to the code repository I mentioned earlier and tested it out:

And just like that, Jenny has learned the couch is not where she belongs and has not been back since!