Add BCn encoders/decoders with RDO support

[walcht]

As discussed in #1159: ktxTexture2_CompressBCn and ktxTexture2_DecodeBCn are introduced in this PR to allow libktx users/consumers to encode/decode BCn textures from/to raw decompressed formats.

https://github.com/richgel999/bc7enc_rdo does not support BC6HU/BC6HS encoding/decoding and also no BC2 (this format is essentially dead since BC3 replaces it).

Please feel free to give feedback, edit, and nitpick as much as possible.

Some context: I am adding KTX2 support to OIIO (PR: AcademySoftwareFoundation/OpenImageIO#5185) and having libktx encode/decode BCn formats significantly simplifies things (also ETC encoding/decoding which I can also open a PR for - if approved).

I haven't updated the KTX-Software-CTS with the added BCn test files.

Once this is finalized, this will fix #587.

Current TODOs:

• add CTS test files from a primary platform. As far as I understand, I shouldn't create/transcode KTX2 test files on non-primary platfroms (non arm64). I did do the testing on my local machine (see https://github.com/walcht/KTX-Software-CTS). => added CTS tests + golden files (still not on a primary platform).
• add RDO post processing step
  • expose RDO parameters with detailed description (some parameters are still missing - I haven't understood them yet)
  • BC1 mode RDO
  • [x] BC2 mode RDO BC2 is rarely used (I don't see any reason why to use it instead of BC3). Postponed or not planned to be implemented at all.
  • BC3 mode RDO
  • BC4 mode RDO
  • BC5 mode RDO
  • BC6H mode RDO (HDR). I don't know if this is possible. If not, then I will not adjust the ert::reduce_entropy function for the moment. postponed (too much work/overload to include in this PR).
  • BC7 mode RDO
  • ultra smooth mode (see: https://richg42.blogspot.com/2021/02/updated-bc7encrdo-with-improved-smooth.html). => trivial to implement, just have to remove dependency on image_u8
• add multithreading for encoder part (up to 9X decrease in time for my 6-core system)
• add multithreading for RDO (bc7enc_rdo makes use of OpenMP so we need to do our own MT same as in encoder - this is trivial) => 5.50X to 7.00X time decrease compared to single-threaded mode.
• add BCn encoding support for ktx encode command
• add BCn decoding to PNG support for ktx extract command
• add BCn encoding support for ktx create command
• agree on which parameters to expose in ktxBCnParams struct (apparently there are many and I am not qualified to know which subset to expose or to expose them all). For the moment I will just expose them all.
  We agreed on using same parameters as UASTC RDO and adding additional ones if they make sense (I haven't benchmarked skip 0 MSE error option so I might remove it if it useless).
• fix bc7enc_rdo compiler warnings (very simple to do, before merging git diff with original files so that we are 100% that we haven't changed anything).
• finalize test cases (not CLI tests but rather libktx tests - e.g., texturetests, etc.).
• enable SIMD acceleration for BC7 encoder (using ISPC - see https://github.com/ispc/ispc) (this should be straightforward and should be enabled by default via a CMake flag; e.g., BC7_SIMD). => since we are planning to use bc7f we will be planning to use bc7g once it is released (this is the SIMD equivalent of bc7f).
• test LIBKTX_FEATURE_BCN_DECODER CMake flag option
• use bc7f for BC7 encoding instead of bc7enc_rdo's encoder (significantly faster + maintained).
• add single image decoders used by GLUpload/VkUpload (should be exposed in the library API but must be independent of the ktxTexture* classes. Software reading a KTX file incrementally will find them useful).
• add BC6H decoder (copied from MIT Licensed https://github.com/iOrange/bcdec from basisu)
• add BC6H encoder from basist::astc_6x6_hdr::fast_encode_bc6h
  • fix issues with signed formats (same way that UASTC HDR handles it) => if we are given a signed input which the encoder can't handle; asserts fail, etc. (obviously, since BC6HU is the only one supported). Need to do some pre-processing step clamp signed and above-maximal-encodable value...
• add BC2 decoder (unpack BC1 followed by unpack sharp alpha).
• [x] add BC2 encoder postponed/aborted (rarely used).
• when encoding to BC5, we must support --normal-mode so a 3 component normal image can be used as input.
• maybe we should support it when encoding to any BCn format with a real alpha channel?
• update BCn Encoding and Decoding in KTX. Currently it just showcases RDO images which were generated using an erroneous old commit.
• RDO with UNORMS? => RDO expects color data but it could be that what is saved in a UNORM is color data so we leave it as it. Added one liner note in ktx tools for --bcn-rdo param.
• external dependencies should be managed via a subrepo
• add usage documentation example akin to that of ASTC (should be in C)

Note1: no LLMs/AI coding tools were used in any capacity whatsoever in writing or aiding in the writing of this PR.
Note2: I am an individual contributor (main reason I am contributing here is to add support for KTX2 in Blender).

Edit: TODO list edits

[walcht]

There are also a lot of compiler warnings from bc7enc_rdo dependency. These should be straightforward to address directly in copied files from bc7enc_rdo.

[CLAassistant]

All committers have signed the CLA.

[MarkCallow]

@walcht, Thank you for this. Can you view the build logs?

One issue I notice immediately is that there are no changes to command_create.cpp. This too needs updating to support BCn encoding.

Regarding RDO and multi-threading, the UASTC encoder also has separate multi-threading options for the encoder and for the RDO step. You can follow the same model.

If we need to fix warnings in the encoder I suggest forking the encoder then incorporating the fork here by way of git subrepo. That will make it easier to potentially contribute fixes back upstream. If you are agreeable I can make the fork and provide instructions for how to incorporate it in your workarea.

Re. SIMD and ISPC, be careful how you support this. We need to support building and running on arm64 processors. Also compile flags to enable SSE or other SIMD options are not compatible with straightforward use of universal build tool chains, which is why this project does not do universal builds. Better is use of compiler pre-defined macros and run-time queries to discover what the software is being compiled for and running on. However since we aren't doing universal builds there is no need to obsess over this last detail.

[walcht]

@MarkCallow - Concerning the command_create.cpp, I am adding BCn encoding for it currently (somehow forgot it) - will also check if I have missed any other commands.

Concerning the build/CI logs: they are mostly failing because of bc7enc_rdo compiler warnings which should be suppressed. They will also fail because I haven't re-generated the golden files yet for ktx CTS (e.g., ktx encode --help output).

If you are agreeable I can make the fork and provide instructions for how to incorporate it in your workarea

Please do so (as far as I understood, this will be forked under the KhronosGroup and any updates here will be pushed there via the git subrepo command). Until that is done, I will keep edit the bc7enc_rdo sub-folder until the CI passes.

Concerning SIMD and ISPC: I think it makes sense to leave this for another PR, do you agree? (reasoning is this: I have to get the basics working properly, add proper testsuite that covers all supported BCn formats, etc. Once that is done, I can open another PR for SIMD performance improvements). Or I will leave this to very end (last in TODO list above).

[walcht]

Updates about CI jobs:

• reuse lint: I don't know why it is not ignoring external/bc7enc_rdo but I have added the SPDX comments in these files.
• MingW CI: failing because of some linkage error with bc7enc_rdo (undefined reference to rgbx::). I have to test this on my Windows machine to see what I am doing wrong/missing.
• MacOS (arm64) compiling bc7enc_rdo fails. I have no idea why (I don't have a MacOS machine so I will be testing this CI on my fork).
• Windows CI: probably some NSIS issue (I don't think I have introduced anything to make it fail)
• Other failures are still due to unused-variables/functions that I don't get locally (have addressed all that were mentioned in the CI logs).

[MarkCallow]

@walcht,

You will need to rebase to or merge current main to get the fix for the NSIS issue on Windows.

I have created a fork of https://github.com/richgel999/bc7enc_rdo. To incorporate it in your add-BCn-decoder branch do the following in the repo root directory:

git subrepo clone https://github.com/KhronosGroup/bc7enc_rdo.git external/bc7enc_rdo -b changes_for_ktx

You can make changes in this subdirectory, as you are now, and when everything is working I can push the changes to the fork.

Concerning SIMD and ISPC: I think it makes sense to leave this for another PR

I agree. I wanted to make you aware that arm64 is a build target.

Re. reuse, you have to add an entry to REUSE.toml to get external/bc7enc_rdo ignored. It is better to do that than add SPDX comments to all the files. The entry in REUSE.toml will have to mention a license. Use the MIT license option.

Here are a few high level points.

• I want to use the bc7f encoder once it is integrated into the bc7enc_rdo repo.
• Re. decoding, currently the ETC and ASTC decoders have different APIs. The former has a non-public unpack function for single blocks. This is called from the GL and Vulkan upload functions, if the device does not support ETC, to convert the data before it is uploaded. The latter has a call that decodes and converts an entire ktxTexture2 object. I want to have both for all decoders: ASTC, BCn and ETC and the transcoders as both have their uses. It was a mistake not to have provided an unpack function and updated the upload functions when the ASTC decoder was added.
• I would like BC6H support to be included in this or a later PR.
• I will be away for 5 days from May 15th and not be able to approve workflow runs during that time. Please plan accordingly.

[MarkCallow]

Re the macOS build failure, because the output from the Xcode build is so voluminous it is run through a script, xcpretty, to prettify it. On a past CI service, without this, the logs exceeded the maximum allowed. Recently, for reasons I have yet to investigate, it has started swallowing compile errors. You can turn it off by editing scripts/install_macos.sh and commenting out the line that installs xcpretty.

[walcht]

@MarkCallow

I want to use the bc7f encoder once it is integrated into the bc7enc_rdo repo.

Agree. I wasn't initially aware of this. Apparently bc7f is significantly faster that the bc7 encoder used here (also the added benefit of being continuously maintained). I will integrate this right now since this seems to be straightforward (bc7enc_rdo also seems a bit not-longer-maintained so l think it's better to just integrate it now rather than waiting for it to be integrated into bc7enc_rdo repo).

Concerning the decoding API: I added BCn decoders for VkUpload/GLUpload as a TODO (will follow same API as in etcunpack). Might also open a PR to add it for ASTC since I have already spent some time getting familiar with this code base.

I would like BC6H support to be included in this or a later PR.

Will add it in this PR. Will add it at the very end though since I have to finalize current formats.

I will be away for 5 days ...

I will try to address CI issues now (It is fine if I incrementally commit here to see if certain jobs pass?).

[walcht]

The integrated basisu_transcoder is a bit outdated (actually, significantly) and doesn't contain bc7f. So for the moment I will just use bc7enc_rdo's BC7 encoder until the fork at (https://github.com/KhronosGroup/basis_universal/tree/fixes_for_ktx_v5_0) is updated to a commit that includes the bc7f namespace.

[MarkCallow]

I was not aware that bc7f is included in basisu_transcoder. By pure coincidence I have just completed integration of Basis Universal release 2.1.0. See the update_basisu_to_2_1_0 branch. We have the v5.0.0 release in flight. I've made a v5.0.0-rc1 release while we wait for some external (to KTX-Software) pieces to be put in place. My plan was to wait until we made the v5.0.0 release before merging this branch. If you retarget this PR and your branch to update_basisu_to_2_1_0 you can start working on bc7f now.

Neither the KTX-Software code nor our golden files required any updates for our extensive test suite to pass with BU 2.1.0. I am therefore amenable to merging it now but will have to discuss within the Khronos WG and can't make any promises.

The single image decoders used by GLUpload/VkUpload should be exposed in the library API but must be independent of the ktxTexture* classes. Software reading a KTX file incrementally will find them useful.

Regarding the current ETC decoder, please note that it does not have a recognized open source license so might not be suitable for your OIIO work.

[walcht]

@MarkCallow - please don't approve the CI workflow yet (it will fail because I haven't updated the golden test files yet).

• I have added initial RDO postprocessing step support for BC1, BC3, and BC7 (only relying on ert.h/cpp without having to use bc7enc_rdo utils.h/.cpp and rdo_bc_encoder.h/cpp because these used OpenMP and it's just better that we only depend on the actual core RDO function in ert.h/cpp).
• Also fixed the encoder in case input texture is not multiple of 4 (still have to absolutely verify this in all cases - this was reading beyond std::vector's size before this commit...).
• No ultrasmooth block support yet (added to TODO list).
• Added initial RDO params for BCnParams in ktx.h (some detailed description for fields I understand).

Will add bc7f once I finish RDO post processing.

[MarkCallow]

I have just merged PR #1170 so you will now find bc7f in main. Please remove external/bc7enc_rdo and adapt this to bc7f. I hope that not too much of your work to date will have been wasted.

One other thing re. bcn_codec.cpp, we need to be able to build with just the decoder when building libktx_read. astc_codec.cpp is one file because of pieces needed by both encoder and decoder so it has ifdefs to allow building only the decoder parts. Unless there is substantial common code you can consider making separate source files for encode and decode. If you do not, then add similar ifdefs.

[walcht]

I added bc7enc_rdo as a git subrepo as you initially suggested.
See this PR: KhronosGroup/bc7enc_rdo#1

I added BC2 decoder tests (which were missing) and fixed some comments.

The CI fails because I removed (by mistake why merging with main) the handle_cmd_output function from iOS CI (I will only push the fix when remaining 3 tasks are done).

[walcht]

When encoding to BC5, we must support --normal-mode so a 3 component normal image can be used as input.

You mean if unnormalized vectors are given in 3 component input and --normal-mode is used then we normalize them before before applying BC5 (then as usual in shader the Z component is easily computed from X and Y). I think I might be misunderstanding your request here (I'm not too versed in normal mapping/non-color compression).

I don't know if the BCn encoder has has any special handling for normals, i.e. adjustments to its error metrics. If so that must be triggered when --normal-mode is specified.

I have verified and there are no explicit normal-map mode (or non-color data) parameters. I'm almost certain the all encoders don't have special parameters for normal maps encoding. Encoding normal maps (or any non-color data) using BC1/BC2/BC3 or BC7 is either way a bad idea (more artifacts, RGB are not separated, etc.).

[MarkCallow]

You mean if unnormalized vectors are given in 3 component input and --normal-mode is used then we normalize them before before applying BC5 (then as usual in shader the Z component is easily computed from X and Y). I think I might be misunderstanding your request here (I'm not too versed in normal mapping/non-color compression).

From ktx create documentation for --normal-mode:

Only valid for linear textures with two or more components. If the input texture has three or four linear components it is assumed to be a three component linear normal map storing unit length normals as (R=X, G=Y, B=Z). A fourth component will be ignored. The map will be converted to a two component X+Y normal map stored as (RGB=X, A=Y) prior to encoding. If unsure that your normals are unit length, use --normalize. If the input has 2 linear components it is assumed to be an X+Y map of unit normals.

3-component normals are assumed to be normalized. If the user thinks they are not, he or she should use --normalize. We don't do this by default because of the processing overhead. This part is the key

The map will be converted to a two component X+Y normal map stored as (RGB=X, A=Y)

The bit in parentheses describes treatment for 4-component compressed formats. For BC5 I think it is common to put X in the R component and Y in the G component but I am not certain. Specifying --normal-mode and a BC format that does not support 8-bit alpha must be a fatal usage error.

--normal-mode will also set a normal-map-related flag on the underlying encoder if one exists. For example the ASTC encoder has a normalMap flag which causes "Input data (to) be treated as a two component normal map, storing X and Y, and the codec will optimize for angular error rather than simple linear PSNR." --normal-mode may also adjust encoder parameters. For example for BasisLZ/ETC1S it disables RDO (no selector RDO, no endpoint RDO) to provide better quality. This is because RDO optimizes for color differences. We should likely disable RDO for BCn. too. --normal-mode does not change anything in the UASTC encoder. (At a minimum we should tell users to not use RDO. Probably should raise an error if both --normal_mode and --uastc_rdo are specified.)

Encoding normal maps (or any non-color data) using BC1/BC2/BC3 or BC7 is either way a bad idea (more artifacts, RGB are not separated, etc.).

This is the main reason for --normal-mode. It does separate the components.

[walcht]

Thank you for the detailed explanation. This phrase explains everything for me The bit in parentheses describes treatment for 4-component compressed formats..

I was finishing my guide about using BCn within KTX and now it is finalized (there are probably typos + minor fixes here and there). It is significantly much better now with better comparisons, PSNR metrics (albeit that metric is not that good, I believe), more comparisons for more parametes/knobs, etc:

https://walcht.github.io/walcht/guides/ktx-bcn-playground.html

Feel free to copy this as a new Wiki page here if you want (it is set under MIT license).

Now what remains: --normal-mode (fairly straightforward) and usage documentation example.

[MarkCallow]

maybe we should support it (--normal-mode) when encoding to any BCn format with a real alpha channel?

I am far from knowing enough of the ins and outs of the various BCn formats to be able to judge if this is worthwhile. It may well be sufficient just to require --normal-mode when encoding to BC5 or vice-versa. Is BC5 available on all desktops?

I am looking forward to reading your guide about using BCn.

[MarkCallow]

Binomial have announced v2.5 of Basis Universal. I believe this includes the bc7g encoder. At some point we will likely update our basis_universal subrepo clone to 2.5. Please take a look and estimate the size of the job of updating this BCn encoder to bc7g. I do not plan to update basis_universal before this PR is released.

[MarkCallow]

Please hold off any more commits until I finish reviewing the latest code. When I'm done you'll see the review here.

[walcht]

I believe this includes the bc7g encoder.

I cloned the basis repo and searched for bc7g and I couldn't find anything. If it is added (or will be added), integrating it in this PR (or after) is almost certainly straightforward (some BC7-specific params might change, I think). There is a mention of bc7e which is (probably?) Intel-based ISPC. Will look more into this. Is there a Release Notes for Basis Universal?

Is BC5 available on all desktops?

I can't confidently answer this (my general knowledge of graphics and hardware support is average at best). I know that on all desktops I did graphics on, BC5 was always supported. E.g., to quote the Unity Engine's documentation
BC5 format compresses textures to 8 bits per pixel, keeping only the red and green color channels. It is widely supported on PC and console platforms.

I think we should add --normal-mode for all BCn formats with alpha (BC2, BC3, and BC7). Users then have another option to compress normal maps other than the usual (and the recommended) BC5 scheme. After you submit the review(s), I will push the local commit that adds this.

[MarkCallow]

I cloned the basis repo and searched for bc7g and I couldn't find anything.

I thought I saw mention in one of the announcements. I could be wrong. Release notes are reportedly at https://github.com/BinomialLLC/basis_universal/wiki/Release-Notes.

[MarkCallow]

Before I forget, I want to fix UASTC multi-threaded RDO in the same way you fixed the bc7 RDO. Have you done that yet? If not, it's fine to later. It should be a separate PR from this.

[walcht]

I haven't touched the UASTC multi-threading RDO yet (because, as you stated, I wanted it to be a separate PR).

I will fix it in another PR (I think it should be straigh-forward; it's all about how work is dispatched to threads).

[MarkCallow]

37 comments including some change requests. I have now reviewed everything to date.

Thanks for this impressive effort on this codec.

[MarkCallow]

Only two schemes are valid for BCn: zlib or zstd. Both are supported in libktx. Anything else is handled by the FILE_DATA_ERROR above. When could this error happen?

[MarkCallow]

As long as it is done before this is released, another PR is okay.

As for how the library is expected to be used, my preference is to provide a consistent set of functionality in the library. The ETC_UNPACK option was put in primarily because of pushback about the weird license for the ETC unpacker.

[MarkCallow]

Probably should allow R16G16B16A16 input as well, dropping the A. Doesn't the BC6H encoder already support that?

[walcht]

Thank you for the detailed review (I will push the commits that address this probably tomorrow).

There is a very important note:

The key here is that sRGB is an encoding of colors. If you are doing any calculation based on the color values, you must decode first, perform the calculation then re-encode.

I really thought, initially, that I should do this (after all, almost always when one reads sRGB, it should be converted to linear, do math, then convert back).

The thing is, the original repo bc7enc_rdo doesn't do this (there is no sRGB->linear conversions at all) and simply performs encoding on sRGB directly. Also, the encode_bc? functions expects unsigned char per channel so converting to linear from sRGB means loosing information. I think these functions expect sRGB inputs and they don't do any internal conversions (I'll double verify again if there is any gamma applied and update this comment but I'm almost certain there isn't).

[MarkCallow]

The thing is, the original repo bc7enc_rdo doesn't do this (there is no sRGB->linear conversions at all) and simply performs encoding on sRGB directly.

The key parameter in basis_universal is called m_perceptual and I see it is used by basis_bc7enc.{cpp,h}. It does not actually cause a decode and re-encode, it modifies the way the encoder works in things like computing the color difference. I haven't looked to see if RDO for BasisLZ and XUASTC use it. The encoders definitely do. If those RDO functions do not then perhaps RDO is done in a way that makes it not necessary.

There are two other Basis compress parameters with srgb in their names. You can ignore them. One is used when generating mipmaps and we don't gen our mipmaps via the BasisU compressor. The other tells it to mark the output file, .basis or .ktx2, as sRGB. We don't use its file writing features either as, among other things, it does not support 3d textures.

[walcht]

Update on this:

Before I forget, I want to fix UASTC multi-threaded RDO in the same way you fixed the bc7 RDO. Have you done that yet? If not, it's fine to later. It should be a separate PR from this.

I spent some hours understanding the RDO code and apparently I was initially wrong about this being fixed and I think there is just no way to make multi-threaded RDO deterministic (i.e., running ST vs. MT yields same results).
The fix (which I figured right now that is not actually a fix) just happened to work on the input test images which were very simple, 16x16, grayscale images. That fix just made it less likely for non-determinism to occur but definitely doesn't solve the problem with actual, real-world images.

Why? Because running RDO on the whole image (single-threaded mode) means that each block N depends on previously RDO'ed M blocks (with M influenced by the window/dict size). In MT mode, image is divided into regions (one region per thread) and RDO is ran on these separately.

I'm currently updating my guide, the no-multithreading parameter description, and a bunch of other things to reflect this.

I also see that a lot of improvements can be done on the RDO function but I will postpone these to after when this PR gets merged.

If you want, I can open a PR to add a comment and documentation about this with UASTC RDO (or any other RDO that is used).

[MarkCallow]

Thank you for your study.

Why? Because running RDO on the whole image (single-threaded mode) means that each block N depends on previously RDO'ed M blocks (with M influenced by the window/dict size). In MT mode, image is divided into regions (one region per thread) and RDO is ran on these separately.

The problem we originally observed was that the MT results were non-deterministic between runs. It wasn't investigated; there is still an open issue on the basis_universal repo. Do you think with the changes you have made MT will at least be deterministic between runs?

We can certainly live with ST and MT giving different results provided we explain it in the docs. An obvious question is which produces the result with the least entropy (i.e. which will deflate best with zstd): ST or MT?

Is the dictionary shared between the threads and would that affect the result.

I'm currently updating my guide, the no-multithreading parameter description, and a bunch of other things to reflect this.

I also see that a lot of improvements can be done on the RDO function but I will postpone these to after when this PR gets merged.

Sounds good. When you have time, please give me a summary of the possible improvements.

If you want, I can open a PR to add a comment and documentation about this with UASTC RDO (or any other RDO that is used).

That will be terrific. I'd like to get this in for v5.0.0. It would have to cover BasisLZ, UASTC LDR 4x4 and UASTC HDR 6x6 Intermediate.

[walcht]

Make this an active cross reference to the documentation for the struct.

I just couldn't figure out how to make Doxygen generate a reference inside a C-code comment that is isn't supposed to consume...

[MarkCallow]

Make this an active cross reference to the documentation for the struct.

I just couldn't figure out how to make Doxygen generate a reference inside a C-code comment that is isn't supposed to consume...

This is a markdown page being processed by Doxygen. According to https://www.doxygen.nl/manual/markdown.html to link a documented entity you can use

[The link text](@ref MyClass)