FineVLA · 2026

Vol.01

FineVLA

Fine-Grained Instruction Alignment
for Steerable Vision-Language-Action Policies

Not just what to do, but how to do it.

https://finevla.xlang.ai/

Motivation · Fine-Grained

02 / 12

Raw coarse instruction

"Put the block into the bowl"

RELABEL

The right hand grasp the purple block from above,
move forward and place it into the pink bowl.

RELABEL

The right hand grasp the purple block from the side,
move forward and place it into the pink bowl.

Page 02 · Why Fine-Grained?

Motivation

Motivation · Object Pose

03 / 13

Raw coarse instruction

"Put the pen into the box"

RELABEL

The left hand grasp the lying pen from above,
place it into the box.

RELABEL

The right hand grasp the standing pen from side,
place it into the box.

Page 03 · Object Pose

Motivation

Motivation · More Dimensions

04 / 13

Fine-Grained Instruction Enables Steerability

Arm Selection

Pick with left hand, place into left bowl

Pick with right hand, place into right bowl

Target Color

Left hand pick up The leftest blue pen and insert it into the pen holder

Right hand pick up The rightest red pen and insert it into the pen holder

Rotation Direction

Grasp the black pen from above and rotate it clockwise

Grasp the red pen from above and rotate it counter-clockwise

Same goal, different fine-grained instructions → different robot behaviors

Page 04 · Steerability Dimensions

Motivation

Closing

11 / 11

FineVLA

Specify how, not just what.

https://finevla.xlang.ai/

— Thank You —

Motivation · The Gap

02 / 11

Why Fine-Grained?

What vs. How

Before · Goal-Level Only

"Pick up the cup"

Single coarse instruction per trajectory
No approach direction specified
No contact region guidance
No motion path or arm choice
Cannot steer execution style

After · Fine-Grained

"Use right hand, approach from left, grasp rim..."

10 fine-grained dimensions annotated
Active arm, approach direction specified
Contact region, motion path described
State transitions documented
Steerable execution control enabled

Page 02 · The Steerability Gap

Motivation

Framework · Overview

Act II · 03 / 11

Four Pillars

A closed loop for action-instruction alignment: data construction, benchmark, VLM annotator, and steerable policy.

FineVLA-Tool + Data·RoboFine-Bench·RoboFine-VLM·FineVLA-Policy

FineVLA Framework Overview — Four Pillars

The FineVLA Framework

— · —

Data · Scale

Act II · 04 / 11

FineVLA-Tool + FineVLA-Data

FineVLA-Data

Trajectories

972K

10 datasets unified

Fine-Grained

47K

Human-verified

Length

10.4×

9.3 → 96.8 words

Dimensions

10

Annotation axes

Browse Examples · Fine-Grained Annotations with Video Demonstrations

FineVLA-Data · 47,159 trajectories

Act II · Data

Evaluation · VQA Track

05 / 11

RoboFine-Bench

Benchmark

VQA Track

1,030 questions across 10 dimensions — Grounding & Reasoning evaluation over 500 held-out videos.

Page 05 · RoboFine-Bench

VQA Track

Evaluation · Caption Track

06 / 11

RoboFine-Bench

Benchmark

Caption Track

11.6K atomic facts decomposed from ground truth — Easy & Hard modes measuring consistency, coverage, and hallucination rate.

Page 06 · RoboFine-Bench

Caption Track

Annotator · VLM

07 / 11

RoboFine-VLM

Scalable Annotator

Fine-tuned Qwen3.5-397B-A17B on FineVLA-Data for robotic action understanding.

VQA Results

Model	Overall
RoboFine-VLM (Ours)	68.2
GPT-5.4	60.2
Gemini-3.1-Pro	59.6
Doubao-Seed-2.0-Pro	58.5
Qwen3.5-Plus	55.9
Qwen3-VL-Plus	47.7

Caption-Hard Results

Model	Overall	Cons	Cov	AHal
RoboFine-VLM (Ours)	82.2	80.4	71.5	94.8
GPT-5.4	78.0	73.8	66.8	93.4
Gemini-3.1-Pro	75.9	75.7	58.5	93.4
Doubao-Seed-2.0-Pro	73.4	72.4	63.7	84.0
Qwen3.5-Plus	72.4	71.0	55.1	91.2
Qwen3-VL-Plus	64.4	67.4	54.3	71.6

Page 07 · RoboFine-VLM

Annotator

Annotator · VLM

08 / 11

RoboFine-VLM

Scalable Annotator

Caption Quality Comparison — fine-grained action descriptions across leading VLMs.

cam_high

cam_left_wrist

cam_right_wrist

RoboFine-VLM (Ours)

Left arm grasps the zipper pull tab on the black bag from the left side and lifts it upward, while the right arm approaches from the right to grasp the pull tab.
Right arm pulls the zipper pull tab horizontally to the right to close the bag, while the left arm stabilizes the bag by holding the left side of the zipper tape.
Both arms release the black bag and retract outward to the left and right sides, leaving the bag stationary on the white table.

3 steps · precise & concise

GPT-5.4

Move the left gripper in from the left and grasp the metal zipper pull ring near the left top corner.
Lift the zipper pull upward and slightly left/back to bring the slider onto the top edge.
Move the right gripper in from above-right, take hold of the zipper pull/slider.
Left gripper braces the bag while right gripper pulls the zipper slider across the top opening.
Release the zipper pull near the right top corner and withdraw both grippers.

5 steps · verbose

Page 08 · Caption Comparison

Annotator

Findings · Insights

09 / 11

Finding i & ii

Insights

i

No Sacrifice

Fine-grained supervision does not sacrifice goal-level task success. FG-only consistently outperforms Raw-only by +1.4 to +8.1 pts. The OFT-vs-GR00T gap shrinks from 6.4 to just 0.8, showing strong cross-architecture generalization.

ii

Complementary

Raw and fine-grained instructions are complementary. Performance follows a clear inverted-U trend, peaking at FG:Raw = 1:1. Best mixed setting reaches 86.8% / 82.5% in simulation and 62.7 in real-world, a +12.8 gain over Raw-only.

Page 09 · Two Key Findings

Insights

Insights · Finding iii

10 / 11

Finding iii

Insights

iii

Steerable

Fine-grained language directly improves factor-level steerability, especially on attributes underspecified by goal-level instructions. Largest gains: pose +23, approach direction +18, color +18.

Approach Direction