Skip to content

Use encoder-only method for reduced-tx-part-set option #5313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
urvangjoshi merged 2 commits into from
Aug 25, 2026
+10 −6
Merged

Use encoder-only method for reduced-tx-part-set option #5313

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
73aa966
Use encoder-only method for reduced-tx-part-set option
urvangjoshi Aug 14, 2026
bd27a66
add some comments
urvangjoshi Aug 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion av2/common/blockd.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2331,7 +2331,8 @@ typedef struct macroblockd {
/*! \brief Flag to decide whether CFL is allowed for a particular chroma
* block as passed down from the parent tree */
CFL_ALLOWED_FOR_SDP_TYPE is_cfl_allowed_in_sdp;
/*! \brief Flag to indicate whether reduced transform partition mode is on */
/*! \brief Flag to indicate whether reduced transform partition mode is on and
* is the same as `cm->seq_params.reduced_tx_part_set` */
int reduced_tx_part_set;
} MACROBLOCKD;

Expand Down
7 changes: 6 additions & 1 deletion av2/encoder/encoder.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -577,7 +577,12 @@ void av2_init_seq_coding_tools(AV2_COMP *cpi, SequenceHeader *seq,
seq->enable_mhccp = oxcf->intra_mode_cfg.enable_mhccp;
seq->enable_inter_ddt =
seq->single_picture_header_flag ? 0 : oxcf->txfm_cfg.enable_inter_ddt;
seq->reduced_tx_part_set = oxcf->txfm_cfg.reduced_tx_part_set;
// Note: we always keep the sequence header flag `reduced_tx_part_set` off,
// ignoring the encoder option `oxcf->txfm_cfg.reduced_tx_part_set`. This is
// because the restricting the transform partition types during encoder RDO
// only has been found to be more compression-efficient than using the
// sequence header option.
seq->reduced_tx_part_set = 0;
Comment thread
wantehchang marked this conversation as resolved.
seq->enable_cctx = oxcf->txfm_cfg.enable_cctx;
seq->enable_ibp = oxcf->intra_mode_cfg.enable_ibp;
seq->enable_adaptive_mvd =
Expand Down
6 changes: 2 additions & 4 deletions av2/encoder/tx_search.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3048,8 +3048,7 @@ static void select_tx_partition_type(
uint8_t full_blk_skip[MAX_TX_PARTITIONS] = { 0 };

for (TX_PARTITION_TYPE type = 0; type < TX_PARTITION_TYPES; ++type) {
if (cpi->common.seq_params.reduced_tx_part_set &&
type > TX_PARTITION_VERT) {
if (cpi->oxcf.txfm_cfg.reduced_tx_part_set && type > TX_PARTITION_VERT) {
Comment thread
wantehchang marked this conversation as resolved.
break;
}
// Skip any illegal partitions for this block size
Expand Down Expand Up @@ -3362,8 +3361,7 @@ static void choose_tx_size_type_from_rd(const AV2_COMP *const cpi,
int64_t cur_rd = INT64_MAX;
const bool is_rect = is_rect_tx(max_tx_size);
for (TX_PARTITION_TYPE type = 0; type < TX_PARTITION_TYPES; ++type) {
if (cpi->common.seq_params.reduced_tx_part_set &&
type > TX_PARTITION_VERT) {
if (cpi->oxcf.txfm_cfg.reduced_tx_part_set && type > TX_PARTITION_VERT) {
break;
}
// Skip any illegal partitions for this block size
Expand Down