Remote photoplethysmography

Research & experiment

Browser-based photoplethysmography

Photoplethysmography (PPG) is a method to measure heart rate, and possible other physiological observables by observing tiny, periodic changes in the optical properties of the skin. The measurement can be performed using dedicated instruments but, perhaps surprisingly, it is also possible to do it using ordinary consumer equipment like a phone camera or webcam. When the instrument does not touch the skin, the method is often called remote photoplethysmography, abbreviated rPPG.

Technically, this is not "artificial intelligence" but, well, it can be called "AI-adjacent": the process involves some non-trivial signal processing to extract signals that may characterize a patient's heart rate and, possibly, their respiratory rate.

Our RPPG demo implementation was primarily intended to dispel myths: though the mathematics is not entirely trivial, it is also not terribly complex. Moreover, modern browsers already offer features that make it possible, almost trivially easy, to monitor a "live" video capture feed of the participating patient, and extract the necessary information.

Signal processing in a Web browser

Visible light photoplethysmography relies on observing the green channel in an RGB video stream, looking for temporal variations that may characterize blood flow in the capillaries of the observed person's skin.

Software that runs in modern browsers has access to associated camera equipment on the user's device, with the user's permission. The video stream can also be analyzed for content.

The fundamentals are simple. Our rPPG application simply performs the following steps:

  • Mask the video frames to process only the oval region that contains the user's face;
  • Extract and sum green channel information from this masked region;
  • Perform a recurrent fast Fourier transform (FFT) on the resulting time-series data stream to transform the data into the frequency domain
  • Look for a signal peak in the region that is most likely to be associated with a human heart beat or respiration
  • Average the signal over a 60-second period to improve the reliability of the result.

This is not a medical tool. It is a technology demonstration, showing how comparatively easy it is to build an application of this nature. This may also shed light on why variations on the rPPG theme are available in app stores for desktop and mobile devices. Without proper calibration and an appropriate, reliable light source, however, the results are at best approximate and should not be used for diagnostic purposes.

Resources... »