Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognition
Abstract
While Multi-modal Large Language Models (MLLMs) excel in general video understanding, their capability in fine-grained and motion-centric tasks remains limited. This limitation is particularly critical in micro-gesture recognition (MGR), where micro-gestures (MG)—subtle, short-duration, and spatially localized human movements—serve as key discriminative signals for implicit affective analysis, yet are easily neglected following common prompting practices. Although MGR has been intensively studied by many discriminative approaches, the use of MLLMs on MGR is underexplored, with notably poor performance. We hypothesize that the motion-sensitive representation ability of MLLMs is constrained by its inherent single-pass forward inference, which can be substantially enhanced through carefully designed test-time guidance. Motivated by this, building on our prior findings regarding temporal insensitivity in Video LLMs, we diagnose zero-shot MGR errors in the Negative Log-Likelihood (NLL) space. We observe that MLLMs suffer from two bottlenecks: 1) insufficient localized evidence and 2) severe score biases driven by language and motion-agnostic appearances. Thus, we propose a novel test-time evidence calibration framework that improves both reasoning details and prediction reliability. Specifically, we introduce a tree search mechanism to progressively acquire localized, fine-grained visual evidence, coupled with a test-time calibration module to mitigate score biases. The multi-cue fusion module then integrates evidence from multiple cues without relying on a single cue for final prediction. Our framework achieves a mean-class accuracy of 26.84% on iMiGUE and 22.10% on MA52, significantly outperforming the Qwen2.5-VL baseline that produces 16.15% and 10.20%, respectively. The code will be available at https://zero-melo.github.io/Zero-MELO.
Key Contributions
- Two-view failure analysis of MLLMs on MGR. We revisit why vanilla MLLMs perform poorly on micro-gesture recognition and quantitatively show that the bottleneck is not solely limited model capacity, but is largely caused by insufficient localized evidence and severe score biases driven by language and motion-agnostic appearances.
- Test-time reasoning expansion for zero-shot MGR. We demonstrate that enhancing evidence exploration and aggregation at inference time can effectively reveal the latent capability of MLLMs, motivating test-time compute as a principled solution for zero-shot MGR.
- Zero-MELO framework. We propose a training-free MLLM framework that couples cue-guided zoom search with dual-reference calibration to address missing local evidence and prior-driven score bias, together with a cue-conditioned evidence fusion strategy enabling robust multi-cue recognition.
- State-of-the-art performance. Extensive experiments on iMiGUE and MA52 show that Zero-MELO consistently improves over standard MLLM inference by large margins, suggesting that limitations of current MLLMs on MGR are highly related to insufficient localized evidence and severe biases.
Why Do MLLMs Struggle with MGR?
A two-view analysis reveals that MLLM failures stem from two complementary and diagnosable bottlenecks.
Bottleneck 1: Insufficient Localized Motion Evidence
We analyze vanilla MLLM inference using Token Activation Maps (TAM) for token-level visual attribution. Even when the model partially captures the correct semantics, it fails to consistently attend to the truly decisive motion phase. As shown below for the gesture Pressing lips, highlighted regions are not consistently grounded on the lip area across three successive generated tokens — naive single-pass MLLM cannot capture compact motion-centric local evidence.
(a) Token activation map — generated token 1
(b) Token activation map — generated token 2
(c) Token activation map — generated token 3
Figure 3: TAM visualization of a failure case for the gesture Pressing lips. The wrong prediction is Bulging face, deep breath. The highlighted regions are not consistently grounded on the lip area or the most discriminative motion phase, indicating that naive single-pass MLLM fails to capture compact motion-centric local evidence.
Bottleneck 2: Prior-Driven Score Bias
An NLL-based analysis further reveals a strong language prior bias: when visual input is removed, the model continues to produce confident predictions. Under free-form generation, a large portion of test clips collapses to a few dominant labels, and this pattern is consistent across multiple MLLMs.
(a) Qwen2.5-VL Top-1 predictions vs. ground-truth distribution on iMiGUE
(b) Prediction collapse severity across different vanilla MLLMs
Figure 4: Failure analysis of the prompt-based free-form generation on iMiGUE. (a) A large portion of test clips maps to a few dominant labels — the visual input is often insufficient to override the language bias. (b) This prediction-collapse pattern is observed across multiple MLLMs, with GPT-4o mini collapsing 82.5% of predictions to the majority Top-1 label.
Method: Zero-MELO
Zero-MELO consists of three key modules operating at test time, requiring no additional training.
Figure 2: The overall algorithm of Zero-MELO. It mainly consists of three key modules: the cue-guided search (Sec. 3.3), the test-time calibration (Sec. 3.4), and the multi-cue fusion (Sec. 3.5). The global video input will first be fed to the MLLM for global inference. Then, the model will acquire local evidence via the cue-guided tree search. Next, the test-time calibration will be conducted, and the calibrated scores will be yielded. Based on the global and local calibrated scores, the multi-cue fusion contributes to the final prediction.
1. Cue-Guided Tree Search
Given an input video clip and a set of semantic cues (e.g., "pay attention to the lips", "pay attention to the hands"), Zero-MELO performs a cue-guided tree search over anchor frames sampled from the clip. For each cue, high-confidence spatial proposals are returned per anchor, aggregated across anchors via robust cross-anchor aggregation, and cropped to yield a temporally-stable zoomed local view. This design adapts zoom-search to video MGR via motion-aware temporal selection, anchor-based localization, and cross-anchor robust aggregation.
2. Dual-Reference NLL-Based Test-Time Calibration
To mitigate the two complementary biases identified in our analysis, we introduce a dual-reference calibration module. For any input view x, calibration computes the NLL relative to two controlled references: a blank calibration input xb (masked video, suppressing visual content to expose language prior) and a lowrank calibration input xr (static repeated frame, preserving appearance but removing motion). The calibrated NLL subtracts these reference scores, yielding a bias-corrected energy that focuses on genuine motion-discriminative visual evidence.
3. Label-Conditioned Multi-Cue Fusion
Not all cues are semantically compatible with every label (e.g., a "hands" cue is irrelevant for a face-centric gesture). Zero-MELO uses a label-conditioned normalized fusion strategy that selects compatible cues per label based on body-part semantics, and aggregates calibrated local evidence with the global branch via family-weighted averaging. This avoids branch selection failures and enables robust multi-cue decisions under shared global context.
Experimental Results
Zero-shot performance on iMiGUE (test set) and MA52 (validation set). Backbone: Qwen2.5-VL-7B.
| Method | iMiGUE | MA52 | ||||
|---|---|---|---|---|---|---|
| mCA@1 | mCA@5 | MF1 | mCA@1 | mCA@5 | MF1 | |
| Prompt-based Zero-shot | ||||||
| GPT-4o mini | 0.1113 | 0.3392 | 0.0699 | 0.0576 | 0.2362 | 0.0363 |
| Gemini2.5-Flash | 0.1917 | 0.3689 | 0.1367 | 0.1666 | 0.3072 | 0.1217 |
| InternVL3.5-8B | 0.2034 | 0.4358 | 0.0927 | 0.1796 | 0.3677 | 0.0914 |
| Qwen2.5-VL-7B | 0.1615 | 0.4101 | 0.0728 | 0.1020 | 0.3133 | 0.0680 |
| NLL-Based | ||||||
| Qwen2.5-VL-7B | 0.0523 | 0.2524 | 0.0072 | 0.1178 | 0.3299 | 0.0714 |
| Test-Time Methods (Qwen2.5-VL backbone) | ||||||
| ZoomEye | 0.0760 | 0.2340 | 0.0326 | 0.1185 | 0.3508 | 0.0723 |
| VCD | 0.2077 | 0.4717 | 0.1045 | 0.1516 | 0.3364 | 0.0888 |
| Zero-MELO (Ours) | 0.2684 | 0.5775 | 0.1325 | 0.2210 | 0.4359 | 0.1221 |
Zero-MELO improves mean-class accuracy from 16.15% to 26.84% on iMiGUE and from 10.20% to 22.10% on MA52 over the Qwen2.5-VL backbone.
Case Study
Qualitative examples illustrating the complementary roles of the three Zero-MELO components.
Figure 5: Case study of Zero-MELO. Each panel shows the global video clip, the zoomed-in view of the most supportive cue, and the trajectory of the prediction Top-1 across global inference, calibration, and final multi-cue fusion. Cases (a), (b), (c) are successful examples: calibration mitigates prior bias, tree search identifies discriminative local evidence, and multi-cue fusion consolidates cue-specific evidence to improve final recognition. Case (d) is a failure case where the hand crop fails to retain sufficient object-related evidence, causing cue-conditioned fusion to over-emphasize the local cue.
BibTeX
@inproceedings{zeromelo2026,
title = {Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs
for Zero-Shot Micro-Gesture Recognition},
author = {Anonymous Authors},
booktitle = {Proceedings of the 32nd ACM International Conference on Multimedia},
year = {2026},
address = {Rio de Janeiro, Brazil}
}