Cat Bowl Monitor – A smart cat bowl that uses facial recognition

Our two cats, Foxy and Layla, have different foods. Layla is strictly on Rx dry food and Foxy is on regular dry food. Layla loves Foxy's food and will eat it all if we don't stop her. Foxy couldn't care less about the Rx food and won't touch it. So I decided to build a feeder that could keep Layla out of Foxy's food. The feeder has been working well for over a year with better than 95% accuracy. Even when it's initially wrong it quickly corrects itself when the cat gets closer to the bowl.

The project consists of a custom Android app for cat face recognition and a commercial feeder altered with a custom Arduino controller to accept Bluetooth open/close commands from the phone. The Android app employs a cat face detector that in turn feeds the faces into the recognition model. The user trains the app by tagging cat face images captured by the cat face detector. It can handle up to 4 cats.

Motivation

This is potentially a long story. I'll begin at the beginning. What prompted this project was the need to keep Layla out of Foxy's dry food. No real momentum on the project happened until I stumbled upon a cat face detector article while researching object detection models. At that point I decided I would attempt to build an app to control a cat bowl.

Development Process

Platform and Tools

First decision was that the app would run under Android on a smartphone. The reasoning for this was:

•  The model would require a reasonably powerful platform to run at a speed that could handle video input of at least 10fps
• Google TensorFlow Lite was a viable target format for the machine vision model and it was readily supported under Android
• Android offered the "Camera2" interface, which could efficiently supply video frames to an app
• I had several old Android smartphones available for testing and development
• Google Android Studio was a viable and free IDE (integrated Development Environment) that could easily be set up and used (unlike Apple) Google's programming languages have evolved now, but at the time Java was the preferred language for Android, and I was somewhat familiar with it.

Cat Face Detection

Once the platform and tools were decided the next task was to find a model that could detect cat faces, was fast enough for a video feed and would output bounding boxes around each face in a video frame. A pre-trained Haar Cascade model was found that could detect cat faces called: haarcascade_frontalcatface.xml. This model will run in an OpenCV library supporting a Cascade Classifier. The one I used was : "org.opencv.objdetect.CascadeClassifier". Tests with this model went well.

Cat Face Recognition

Next was to find a suitable face recognition model. The most compact and readily available model at the time was MobileFaceNet, which was found in a TFLite format. This model was, of course, trained on human faces. Nothing pre-trained on cat faces was found at the time and to train a recognition model from scratch takes a huge amount of resources and labor. So, I reasoned, a cat face isn't all that different from a human face - two eyes, a nose, 2 ears. To improve the chances of it working I decided to do additional training of the model with cat faces. That's where the fun really started. The MobileFaceNet model found was already in TensorFlow Lite, which is a compact format for deployment on mobile devices. The catch is that it's not possible to do training with a TFLite model. So the model had to be reconstructed as a Keras model for training and could then be converted back to a TFLite model once training was done.

It took a considerable amount of time to replicate the structure of the MobileFaceNet model with a Keras model. The structure had to match exactly so that all training parameters from the MobileFaceNet model could be loaded into the Keras model as the starting point for further training. This work was all done in Google CoLab in Python. CoLab can be used for free with limitations on the amount of CPU consumed. Fortunately, the additional training with cat faces fit within those limits.

Training images were collected from the internet. The images had to be a portrait style format of just the cat's face (ears included). A total of about 5000 cat faces were collected for the training. The images were divided up into training and test groups. Training was done with APN (Actual/Positive/Negative) labelled triplets. This dataset is extremely small by model training standards, but it still took a considerable amount of manual effort to compile.

Using Google CoLab the model was further trained with the cat face data and ultimately achieved reasonably good accuracy. The trained Keras model was then converted back to a TensorFlow Lite model for deployment.

It's important to note that the recognition model doesn't output a "yes/no" result. It wasn't trained to recognize specific...

Read more »

Log In/Sign up to comment