Subtle acting in AI video: facial expression words that hold
Ask for a glance, a held breath, a half smile or a slow blink instead of "emotional". Which expression words read in a short clip, and which ones warp a face.
Type "a sad woman" into a video model and you get a face that is sad the way a stock photo is sad: the whole mouth, the whole brow, all at once, and held until the clip ends. Type "she looks down, then back up" and you get something an editor could use. Acting on screen is mostly small, and a clip is short, so the small things are the only ones that fit.
This post is about the face and nothing else: which words to use, how many, and when to stop fighting a model and start from a picture. Camera work is a different job, covered in nine camera moves for AI video. Every character below is invented; none of this needs a real person's picture, and you should not use one without the right to.
Why "emotional" gets you nothing
An emotion is a label for something that happens over time. A video model renders frames, so it needs the event, not the label. "Heartbroken" does not say whether the eyes drop, the chin trembles or the shoulders sink, so the model averages every heartbroken face it has seen, and the average is a mask.
The fix is to swap the label for a behaviour you could film. Ask yourself what a camera would actually see, then write that. Below is the short vocabulary I keep coming back to.
Eight small actions that read on screen
- A glance. "Her eyes flick to the door, then back to the letter." A glance needs a start and an end target, like a camera move does. Without a target the eyes wander.
- A held breath. "She stops breathing for a moment, chest still, then lets out a slow exhale." It reads as tension and costs the face almost nothing, because nothing on it has to change.
- A half smile. "One corner of the mouth lifts slightly and falls." Say "slightly" and say it only once. Unqualified smiles show every tooth.
- A slow blink. "A single slow blink." It suggests tiredness, calm or a decision, and it is the one eye movement models handle most reliably.
- A swallow. "He swallows, jaw tight." Nerves in one word, mostly in the throat, so the face stays stable.
- A brow flick. "Left eyebrow lifts a little." One brow, one direction.
- Looking away. "She turns her eyes to the window and does not look back." A face that stops performing for the lens is easier to render than one that holds eye contact.
- Stillness. "Face almost motionless, only the eyes move." The most underrated direction there is.
Spend one expression per clip
A short clip has room for one beat: something is there, something changes, something settles. Ask for three emotions in a row and the model compresses them into a flicker, with the face morphing between them. Pick the change that matters and let the rest of the clip be calm. Clips in the studio run up to 30 seconds at the very most, and even a long one carries a single beat better than a chain of them.
Medium close-up of an old lighthouse keeper at a rain-streaked window, evening light. He is nearly still. His eyes move to a ship's lamp far out at sea, he holds his breath for a moment, then exhales slowly and one corner of his mouth lifts slightly. Camera static, shallow depth of field.A young pilot in a dim cockpit, face lit by green instrument glow. She gives a single slow blink, her eyes drop to the dial in front of her and stay there. Her expression otherwise does not change. Camera locked.What makes a face warp
Faces warp when the model is given more to change than it can keep consistent. The usual causes, in the order I meet them:
- A stack of emotion words. "Angry, sad, hopeful, afraid" has no single face, so it tries all four. Cut to one behaviour.
- A fast head turn. Turning from profile to front means the model must invent the far side of the head. Say "turns slowly" and keep it to a quarter turn, or cut the turn and use two clips.
- Big mouth action. Laughing, shouting and talking move teeth, lips and cheeks together. If you need it, keep it brief and keep the shot wide enough that small errors do not show.
- Hands crossing the face. A hand that covers an eye and then leaves it often takes the eye with it.
- Several faces at once. Every extra face gets less of the model's attention. Put the acting on one person and leave the others as a blurred background.
- A drifting camera. A move plus an expression is two jobs. Lock the camera when the face is the point.
Frame the face so the beat is readable
A glance or a blink is a few pixels of movement. In a wide shot it vanishes. Ask for a medium close-up or a close-up, say that the eyes are the subject, and leave empty space in the direction the character looks. Keep the light still while the face acts; a lighting change on top of an expression reads as flicker, as in light changes in AI video without flicker.
When to start from a still
If the character's face has to look the same in several clips, or you already have the exact face you like, do not ask a text prompt to invent it again. Make the still first with a text-to-image or edit model, such as an image edit model, until the face is right, then animate it with image-to-video. The picture fixes who it is. Your prompt only has to say what the face does, in a single sentence, with no description of the face itself.
Pick a still where the face is relaxed and a little off-centre, eyes open, mouth closed or barely open. A frame caught mid-laugh gives the clip nowhere to go. Test the wording on a cheap draft first; the draft, lock, finish workflow covers how, and P-Video-2 image-to-video has a draft toggle for it.
The woman in the picture slowly lowers her eyes, then raises them again to look just past the camera. Her face stays relaxed, her lips barely part. Nothing else in the frame moves. Camera static.Where this breaks
Models do not act. They produce frames that resemble acting, so a half smile may come back as a full one, a glance may land on the wrong object and a blink can stick halfway. Plan to reroll a few times, change one word per run, and keep the takes that read. Close-ups of eyes and mouths are also where models are the least forgiving, and no wording removes the risk entirely. If a face is not working after a few tries, make the shot wider or make it two simpler clips.
Questions
How do I get a smile that is not a big grin?
Say "half smile" or "one corner of the mouth lifts slightly", once, and add that the rest of the face stays relaxed. Starting from a still with a closed, calm mouth helps too.
Why does the face change between two clips of the same character?
A text prompt invents a new face each time. Make one still of the character, then use it as the first frame of every clip so the face is fixed before the acting starts.
Can I make a character speak?
Mouth movement is the hardest facial action for a video model. Some models generate sound; check the model's own page before you render. For a dialogue scene, keep the shot wide, keep the line short, or add the voice in an editor over a calm clip.