[WIP] Add INT2_ASYM weight compression mode - #4168
Draft
Cyberpunk1210 wants to merge 3 commits into
Draft
Conversation
CompressWeightsMode stopped at INT2_SYM, so 2-bit asymmetric compression was not expressible. At 2 bits there are only four levels, and a symmetric grid spends one of them on a sign that one-sided weight groups never use. INT2_ASYM reuses the existing generic asymmetric path: is_asym_mode selects level_low=0 / level_high=2**num_bits-1 with both scale and zero point derived from min/max, which is already generic in num_bits. compression_dtype maps to TensorDataType.int2 because OpenVINO has no i2 type -- int2 weights are physically unsigned, which is also why the symmetric path has to shift by +2**(num_bits-1). Asymmetric codes land in [0, 3] natively, so the weight constant keeps its dtype and the zero point is emitted as u2 as well. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3 tasks
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new weight-compression mode, CompressWeightsMode.INT2_ASYM, to enable 2-bit asymmetric weight quantization in NNCF’s weight compression pipeline (notably for OpenVINO export paths).
Changes:
- Add
INT2_ASYMto the publicCompressWeightsModeenum (with docstring description). - Extend
WeightCompressionConfigto recognizeINT2_ASYMfor bit-width (num_bits) and asymmetric-path selection (is_asym_mode). - Map
INT2_ASYMtoTensorDataType.int2incompression_dtype(leveraging OpenVINO’s physical u2 storage for int2).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/nncf/quantization/algorithms/weight_compression/config.py |
Adds INT2_ASYM handling for bit-width, asym-mode selection, and compression dtype mapping. |
src/nncf/parameters.py |
Exposes INT2_ASYM in the public API enum and documents its behavior and motivation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
87
to
+92
| def is_asym_mode(self) -> bool: | ||
| return self.mode in [CompressWeightsMode.INT4_ASYM, CompressWeightsMode.INT8_ASYM] | ||
| return self.mode in [ | ||
| CompressWeightsMode.INT2_ASYM, | ||
| CompressWeightsMode.INT4_ASYM, | ||
| CompressWeightsMode.INT8_ASYM, | ||
| ] |
Comment on lines
72
to
78
| CompressWeightsMode.NVFP4: 4, | ||
| CompressWeightsMode.FP8_E4M3: 8, | ||
| CompressWeightsMode.MXFP8_E4M3: 8, | ||
| CompressWeightsMode.INT3_SYM: 3, | ||
| CompressWeightsMode.INT2_SYM: 2, | ||
| CompressWeightsMode.INT2_ASYM: 2, | ||
| } |
Cyberpunk1210
changed the title
Cyberpunk1210
marked this pull request as draft
August 10, 2026 07:47
Two mode allow-lists predate INT2_ASYM and did not pick it up: - lora_correction.py listed the integer modes explicitly, so INT2_ASYM fell through to `raise InternalError` even though do_integer_dequantization is generic over them (INT4_ASYM already used it). The error message had also gone stale and omitted INT3_SYM/INT2_SYM; it now lists what the branch above it actually accepts. - gptq.py collected zero points only for [INT8_ASYM, INT4_ASYM], so INT2_ASYM took the `else` branch and had its zero points silently set to None -- an asymmetric mode dequantized as if symmetric, with no error. Replaced with `is_asym_mode`, which is the property that defines "has a zero point" and is the idiom used elsewhere (weight_lowering.py, optimized_functions/models.py), so the list cannot go stale again. Reachable in both cases: quantize_model.py only blocks the float modes for the AWQ/scale-estimation/GPTQ/LoRA group on the OpenVINO backend, so `mode=INT2_ASYM` with either algorithm reaches these dispatches. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers what was previously untested for the new mode: - test_int_asym_compressed_weights_range: codes fill [0, 2**num_bits - 1] and the zero point is actually used. Parametrized over all three asymmetric modes so INT2_ASYM is checked to behave like its siblings rather than in isolation. The quantization range is clamped to include zero, so one-sided data is what exercises the zero point: all-positive gives zp == 0, all-negative gives zp == level_high. - test_int2_asym_group_wise_zero_point_shape: a per-group zero point exists and has the same shape as the scale, which INT2_SYM has no equivalent of. - test_int2_asym_config: num_bits, is_asym_mode and the int2 (physically u2) compression dtype. - test_int_quantization_with_precomputed_parameters: INT2_ASYM added to the existing parametrization, including the two cases where supplying only one of scale/zero point must raise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
the
NNCF OpenVINO
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Adds
CompressWeightsMode.INT2_ASYM, reusing the existing genericasymmetric path (
is_asym_mode): level_low=0 / level_high=3 with scale andzero point derived from min/max.
compression_dtypemaps toTensorDataType.int2(OpenVINO has no i2 type; int2 weights are physicallyunsigned u2).
Also picks up two mode dispatches that predate the new mode (found via
review feedback on this PR):
lora_correction.pylisted the integer modes explicitly, soINT2_ASYMraised
InternalErrordespitedo_integer_dequantizationbeing genericover them.
gptq.pycollected zero points only for[INT8_ASYM, INT4_ASYM], soINT2_ASYMsilently gotzero_points = None-- an asymmetric modedequantized as if symmetric, with no error. Replaced with
is_asym_modeso the list cannot go stale again.
Reason for changes
CompressWeightsModestopped atINT2_SYM, so 2-bit asymmetric compressionwas not expressible. At 2 bits there are only four levels, and a symmetric
grid spends one of them on a sign that one-sided weight groups never use.
Related tickets
N/A
Tests
Added to
tests/openvino/native/quantization/test_weights_compression.py:test_int_asym_compressed_weights_range-- codes fill[0, 2**num_bits-1]and the zero point is used (zp == 0 for all-positive data, zp ==
level_high for all-negative). Parametrized over INT2/INT4/INT8_ASYM so the
new mode is checked against its siblings.
test_int2_asym_group_wise_zero_point_shape-- per-group zero pointexists and matches the scale's shape.
test_int2_asym_config-- num_bits, is_asym_mode, int2 compression dtype.INT2_ASYMadded totest_int_quantization_with_precomputed_parameters,including the cases where supplying only one of scale/zero point raises.
Also validated end-to-end via optimum-intel's OpenVINO export path
(companion PR: huggingface/optimum-intel#1922) on a
2-bit MoE model.
Caveat on my own verification: both machines I have access to are cut off
from PyPI/conda-forge, so I could not execute the suite locally. The new
assertions were derived by transcribing the exact formulas from
calculate_integer_quantization_params,calculate_scale_zero_pointand_calculate_integer_quantized_weightand checking them numerically(including that no value lands on a rounding tie, which would make the test
non-deterministic under banker's rounding). CI is the real check here.