The problem
To study emotional reaction you need recordings of faces reacting to known stimuli, with each recording reliably associated with the stimulus that caused it. That sounds trivial and is not: the alignment between "what was on screen" and "what the camera captured" is where these datasets fall apart.
Most labs solve it once, locally, with a script that only its author can run. The result is that comparable studies are not actually comparable, because the collection protocols differ in undocumented ways.
The approach
Capture and segmentation as two separable pieces. The toolkit records reactions during stimulus presentation, and separately provides the means to split those recordings per stimulus afterwards. Keeping them separate means a lab can bring its own stimulus set, or re-segment existing recordings when the study design changes.
Consumer webcams as the target rather than laboratory rigs. A method that requires specialist hardware is a method most groups cannot replicate, and ecological validity arguably improves when the camera is the one participants normally sit in front of.
Released publicly so the protocol can be inspected and criticised. In research tooling, being auditable matters more than being polished.
What was hard
Timing. Video playback and webcam capture are independent asynchronous processes on a consumer machine, and drift between them silently corrupts the labels — a reaction gets attributed to the wrong stimulus, and nothing downstream will ever flag it.
Ethics and consent are also structural rather than procedural. Facial recordings are biometric personal data. Collection design has to account for consent, storage and withdrawal from the start, because retrofitting that onto an existing dataset is usually impossible.
Why it is still on the site
This is the oldest project listed here, and it is the foundation of the rest. Behavioral measurement is where the discipline came from: being careful about what a signal does and does not license you to claim.
That instinct is what shows up later as subject-wise validation splits in SpineSound and as honest error reporting in the timber work.
Public research thread
EFEP extended this research direction into a modular webcam-based pipeline: read a person's expression, estimate the reaction it would naturally elicit, and synthesize an agent's facial response. The public coverage describes 60 participants, five social scenarios, action-unit-level facial movement and a deliberately low-risk EU AI Act framing for interactive media, creative applications and UX research.
This is not positioned as surveillance, hiring, health diagnosis or emotion policing. It belongs here because it shows the same principle as the studio work: if the signal is human, probabilistic and ethically sensitive, the tooling has to be inspectable and bounded.