This is a platform I worked on during my internship at MorpheLabs that lets organizers upload event photos, so attendees can find themselves in the gallery using face recognition. The version I inherited only knew how to do that one thing: you upload a selfie, and it hands back your matches.
That worked, but it only worked once per person, then kept making you prove who you were again every time you came back.
The problem with searching every time
Someone checks right after the event, comes back days later once more photos are up, maybe helps a friend find a shared photo. Every visit started the same way: take a selfie, upload it, wait for the match, even though the app had already matched your face the first time. Selfie search is fine for getting someone into the gallery once. It’s bad at making them come back.
The idea: let people identify themselves instead of proving it every time
The app already detected every face in every photo, it just threw that information away the moment a search finished. What if attendees could recognize themselves in a lineup of already-detected faces instead of searching again?
That’s face clustering: group every detected face in an event by who it belongs to, and let attendees pick themselves out instead of re-uploading a selfie. Once you’re in a cluster, you’re identified for as long as the event’s gallery exists, not just for one search.
Why the original architecture couldn’t do this
Clustering sounds like something you could bolt onto anything, but the existing system genuinely couldn’t. It used AWS Rekognition for face matching: a selfie went out, a FaceId came back. It never held an actual representation of a face, just a reference living inside someone else’s collection. You can’t cluster faces you don’t have. Grouping means comparing every face against every other, which needs the numerical representation, not a lookup ID.
So I moved face recognition in-house. DeepFace generates a 512-dimensional embedding for every detected face using ArcFace, RetinaFace handles detection, and the embeddings get stored in PostgreSQL with pgvector. Once the embeddings lived in our own database, comparing every face against every other became a query instead of an impossibility.
What clustering unlocked
Browsing instead of searching
Once every photo in an event finishes processing, the system groups all detected faces using Agglomerative Clustering with cosine distance and average linkage, at a distance threshold of 0.4. Each cluster is one person as they show up across the event, ranked by how many photos they appear in, with a representative face picked by detection confidence.
That turns the gallery into something you can filter by person instead of only search by selfie. Attendees browse detected faces, pick out themselves or a friend, and the gallery narrows to those photos. Selfie search still works for a first visit, but it’s no longer the only way in.
A visual way to explore who’s in the gallery
I also projected the 512-dimensional embeddings down to two dimensions with UMAP (falling back to PCA when needed), so every detected face becomes a point on a scatter plot, color-coded by cluster. Attendees can zoom, pan, and jump to a source photo from any point, another way to spot themselves or a friend.
Staying identified between visits
The original workflow discarded your results the moment you left the page. With clustering, being “found” is tied to a cluster that persists for as long as the event’s gallery does. I added browser-side caching so returning to an event you’ve already searched doesn’t require repeating it.
Implementation
Supporting clustering and browsing meant photo processing had to stop being synchronous. Previously an upload sat there while detection and embedding generation ran inline. I moved that into background jobs with Dramatiq and Redis: the API saves the photo, queues a job, and returns immediately, while a worker detects faces, generates embeddings, and writes them to pgvector. Clustering runs once all photos in an event finish processing, and can be triggered again if new photos come in later.
What I learned
The insight here wasn’t a better model or a faster database. It was noticing the app already had the information it needed to recognize returning attendees and was discarding it every time. Face clustering was the idea; owning the embeddings with DeepFace and pgvector was what made it possible, since you can’t group faces you don’t have direct access to.