Can Computer Vision Detect Heart Rates?🫀

Apr 05, 2024

I never thought a camera could capture heartbeats.

But I finally managed to build my own contactless heart-rate monitoring system.

So how did I do it and can you build your own?

To answer this question, on a high level, we used computer vision, and yes again, with my help you can build your own ;).

Now for those who have been with me for a while and already know what Computer Vision (CV) is, you must be saying "Yeah, yeah Ritz, we already know about computer vision, come on give us some more detail".

Sure, but before I do, for those who are new to computer vision, in a nutshell, CV gives intelligence to cameras.

Right now it's amazing at the vast applications of CV which I will get into later in this email. But first, you have to know the main tools of CV which are:

Image Classification
Pose Estimation
Object Segmentation
Tracking
Object Detection

For monitoring heartbeats, we used Object Detection.

Wait Object Detection! How is this possible?

Haha, okay so object detection isn't what measures the heartbeat. I'll tell you what does in a bit.

But before I do, just a bit of background about object detection and my experience with it. If you don't know already, object detection is simply the detection of objects duh...haha..okay on a serious note it returns the bounding box drawn around the detected object as well as the location.

How does one run Object Detection?

Well, there are several models for object detection, but if you were part of my journey, you would know that I love using YOLO (You Only Look Once)!

I've been teaching YOLO since YOLOv3 right up till YOLOv8 and YOLO-NAS. By the way, YOLOv9 will be coming soon to Augmented AI University ;).

Okay, I'm digressing now...I used YOLOv8 for detecting heads.

Why did I detect people's heads for heart rate detection?

Great question! When performing heart rate detection, which I'll explain in a bit, we need to keep the person's head as still as possible.

We could tell the participant to just keep still but you know that's hard for a lot of humans to do even me 😅.

Once a person's head is detected, I crop that image and then finally pass it into the heart rate detection model.

How does the heart rate detection model work then?

I came across a paper with code from MIT in which they amplified the micro-movements of bodies in motion.

So no matter how still someone is, you can distinctly see them either breathing or swaying.

This algorithm is called Eulerian Magnification.

I recall witnessing a striking scene where a construction crane appeared to sway precariously, almost to the point of snapping, yet in reality, it was the effect of motion amplification revealing the subtle swaying caused by the wind.

To the naked eye, the actual crane stood motionless, a stark contrast to the dramatic oscillations displayed on my screen.

So what do micro-movements have to do with heart-rate detection?

Haha, this is where it gets really exciting!

Because now, all you have to do is isolate the red and green channels of a camera and amplify the motions in those spectrums.

By doing this, guess what you, can acquire the systolic and diastolic signals from a patient's heart rate!

Amazing right?

But wait, are cameras fast enough to get a reliable heart rate

Well if you consider that we can at minimum capture 10 frames per second (FPS) for a heartbeat that is between 60-160 Beats per minute.

If we convert this to seconds, it is 1-2.6 beats per second, and according to Nyquist's theory, we need double that sampling frequency to get a reliable clean signal.

Using YOLOv8 models at 10FPS is more than enough for missing any critical data.

A nice model to not skip a beat🤣 😅.

Okay, puns aside... this brings us to our next question.

So how well does it perform?

Well, I'll admit, while this is not medical-grade precision, However, I've acquired some really great results from my initial tests as you can see in this gif.

If I could tap in the Still-Force (The Flash reference) and be super still, then the BPM from the computer vision model closely follows the BPM from my Apple Watch.

I truly believe that with further research it can eventually pass the FDA.

How can I learn Computer Vision and build cool apps like this?

We have the most comprehensive and innovative list of Computer Vision Projects that you can try out. There's just too much to mention, but if you sign up for Augmented AI University, you'll gain access to all 500 AI-CV tutorials and projects.

You'll start at beginner level and work your way through the following:

YOLO Object Detection (v3 to v9)
Object Tracking with FairMOT, NorFair, DeepSORT & more
Pose Estimation - MediaPipe, YOLOv7/8 with reinforcement learning
Object Segmentation with U-Net and YOLO with SAM

[Click here to unlock your financial freedom with AI – No In-Depth Trading Knowledge Needed]

Join Augmented AI University

What is Augmented AI University?

For a price less than a Netflix or Disney+ subscription of $15 per month (only if you order today), you can get this course along with our other cutting-edge AI courses and every new course that we develop thereafter.

Check out the line-up of AI Courses:

AI in Agriculture Course 2024 - Unlocking the Future of Farming: AI in Agriculture
LangChain Chatbots Course - Revolutionize Your Conversations with LangChain Chatbots
AI in Trading Course 2024 - Master the Market: AI in Trading
YOLO-NAS & v8 Object Detection Course - Discover the Power of YOLOv8 (YOLOv9 Coming Soon)
Self-Driving Car Course - Take the Wheel: Innovations in Self-Driving Car Technology
AI on Edge with Jetson Course - Harnessing AI on the Edge with Jetson
AI Influencer Course 2024 - Creating Digital Personalities: The Rise of AI Influencers
OpenAI CV Kit Course - Exploring Visionary AI with OAK: OpenAI CV Kit
AI-Driven Traffic Dashboard Course - Navigating the Flow: AI-Driven Traffic Dashboard
Generative Adversarial Networks (GANs) Course - Imagining New Realities with GANs: Generative Adversarial Networks
AI Drone Course 2024 - Elevating Perspectives: The AIDrone Revolution (Coming Soon)
Ironman ChatGPT JARVIS 2024 - Personal Assistant on Raspberry Pi (Coming Soon)
AI in Medical & Healthcare 2024 - Computer Vision + Big Data + LLMs for Diagnosis & Prognosis (Coming Soon)

These courses are designed to provide a deep dive into the respective areas, equipping you with the knowledge and skills to leverage AI in these cutting-edge fields.

Don't miss out on this opportunity to stay ahead of the curve and become a master of AI development.

Discover a world of possibilities with LangChain with other cutting-edge courses at Augmented AI University – all for just $15 per month, billed annually.