embedl

/

dinov3-quantized-tensorrt

like 2

Follow

Embedl 82

Hi! I'm trying to reproduce this quantized model with the public embedl-deploy package (0.8.1, latest on PyPI) and the official docs workflow, but I can't get the same QDQ layout.

What I did:

1. Load facebook/dinov3-vitb16-pretrain-lvd1689m via transformers.DINOv3ViTModel (224×224 input)
2. transform(model, args, patterns=TENSORRT_PATTERNS)
3. quantize(fused, args, config=QuantConfig(activation=INT8 symmetric per-tensor, weight=INT8 symmetric per-channel), forward_loop=calibration, freeze_weights=True)
4. torch.onnx.export(..., opset 20)

Result: 98 QDQ pairs (61 weight + 24 LayerNorm output + 12 Linear output + 1 input).

Your ONNX: 120 QDQ pairs (72 weight + 24 LayerNorm output + 12 MatMul output + 12 GELU output).

Key differences I found by inspecting the official ONNX:

• The official graph has quantization on GELU outputs and attention (attn@V) MatMul outputs — with QuantStub_66 etc. present in the initializers
• In my exported graph, GELU is fused inside FusedLinearAct and attention is fused into FusedScaledDotProductAttention, so those outputs have no QDQ points
• The official graph's attention seems not fused (RoPE / MatMul / Softmax nodes are explicit), while the default TENSORRT_PATTERNS fuses it

Question:
How was this ONNX generated? Is there a pattern-list configuration (e.g. skipping ScaledDotProductAttentionPattern / LinearActPattern) or a specific library version / internal tool that produces
the 120-QDQ layout? The public 0.8.1 with the documented workflow seems unable to reproduce it.

Thanks!