ESP32-CAM Face Detection and Recognition: On-Board vs Server Options

Face recognition door locks and attendance systems are popular final-year projects, and the ESP32-CAM appears in many of them because the CameraWebServer example once offered face detection and enrolment with a single click. In practice, the ESP32 is a microcontroller with limited RAM and no hardware for neural networks, and on-board recognition has serious limits in accuracy, speed and security.

This tutorial explains the difference between detection and recognition, compares the realistic options for an ESP32-CAM project, and then builds the most reliable one for students: the ESP32-CAM streams video and a Python program on a laptop detects faces with OpenCV.

Detection Is Not Recognition

Face detection answers: is there a face in this image, and where? The output is a bounding box. It is enough for counting people, triggering a photo or centring a pan-tilt camera.
Face recognition answers: whose face is this? It first detects the face, then computes a numeric feature vector (an embedding) and compares it with stored vectors of enrolled people. It needs good images, careful thresholds and much more computation.
Anti-spoofing, telling a real face from a photo of a face, is a third and harder problem. Simple systems, including the ESP32 demos, generally do not attempt it.

The Realistic Options

Option	Where it runs	Typical use	Main limits
CameraWebServer face detection	ESP32, using Espressif's esp-dl models	Quick demo of detection and enrolment	Small frame sizes only, low frame rate, availability depends on core version
ESP-WHO framework	ESP-IDF projects, aimed mostly at ESP32-S3 boards	Detection and recognition products	Not an Arduino sketch; the S3 has vector instructions the ESP32 lacks
TinyML model (for example Edge Impulse FOMO)	ESP32 with PSRAM	Detect a person or object class on-board	Low input resolution, you train and test the model yourself
Server-side OpenCV or face_recognition	Laptop, Raspberry Pi 4 or cloud	Reliable detection and recognition, logging	Needs Wi-Fi and a computer running all the time

In the CameraWebServer example the face detection and recognition switches only work at small frame sizes, around CIF (400 x 296) or smaller, because the models need image data in RAM and enough CPU time per frame. In recent releases of the ESP32 Arduino core this feature has been reduced or depends on the esp-dl library being included for your chip, so it may be missing from the web page. If you need it for a demo, check the example source for your installed core version.

Server-Side Face Detection with OpenCV

Run any MJPEG streaming firmware on the ESP32-CAM, such as CameraWebServer or the minimal MJPEG server from this series, at VGA resolution. On the computer install OpenCV with pip install opencv-python. The script reads the stream, detects faces with OpenCV's bundled Haar cascade, draws boxes and saves a snapshot whenever a face appears, at most once every five seconds.

Detect faces in the ESP32-CAM MJPEG stream with OpenCV and save snapshots

import time
import cv2

STREAM_URL = "http://192.168.1.50:81/stream"   # ESP32-CAM stream
COOLDOWN_S = 5                                 # min gap between saves

# Haar cascade file shipped inside the opencv-python package
cascade = cv2.CascadeClassifier(
    cv2.data.haarcascades + "haarcascade_frontalface_default.xml")

cap = cv2.VideoCapture(STREAM_URL)
if not cap.isOpened():
    raise SystemExit("Cannot open stream - check the URL and that no one else is viewing it")

last_save = 0
while True:
    ok, frame = cap.read()
    if not ok:
        print("Stream lost, reconnecting...")
        cap.release()
        time.sleep(2)
        cap = cv2.VideoCapture(STREAM_URL)
        continue

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.equalizeHist(gray)          # helps under uneven lighting
    faces = cascade.detectMultiScale(gray, scaleFactor=1.1,
                                     minNeighbors=5, minSize=(40, 40))

    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    if len(faces) and time.time() - last_save > COOLDOWN_S:
        name = time.strftime("face_%Y%m%d_%H%M%S.jpg")
        cv2.imwrite(name, frame)
        print(f"{len(faces)} face(s) - saved {name}")
        last_save = time.time()

    cv2.imshow("ESP32-CAM faces", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):   # press q to quit
        break

cap.release()
cv2.destroyAllWindows()

How the Script Works and How to Extend It

cv2.VideoCapture opens the HTTP MJPEG stream through OpenCV's FFmpeg backend and returns one decoded frame per read() call.
Haar cascades work on grey images; equalizeHist spreads the brightness range so faces in dim rooms are still found.
minNeighbors=5 reduces false detections, and minSize ignores tiny faces that are too far away to be useful. Raise minNeighbors if you see boxes on curtains or posters.
The reconnect block makes the script survive Wi-Fi drops and ESP32 resets.
For recognition, replace the cascade with a face embedding library such as face_recognition (built on dlib), enrol a few photos of each person taken with the same camera, and compare distances against a threshold you test yourself.

Haar cascades are fast but miss turned or partly covered faces. OpenCV's DNN module can load more accurate pre-trained face detectors when you need better results and the computer is powerful enough.

Practical Tips for Better Results

Lighting matters more than code: light the face from the front. Backlit faces against a window are often not detected at all.
Camera height: mount the ESP32-CAM roughly at face height, facing people as they approach.
Resolution: VGA is a good balance; very small frames lose facial detail, very large ones reduce frame rate.
Privacy: tell people a camera is active, store face images securely and delete them when no longer needed.

Summary

The ESP32-CAM can run small face detection models on-board, but only at low resolution and frame rate, and support depends on the Arduino core version. Recognition on the ESP32 is a demo, not a security system. For dependable student projects, let the ESP32-CAM stream video and run OpenCV detection, or a recognition library, on a computer, as the Python script shows.