Skip to content

report: strip deduplicated extabindex/extab entries from base object#349

Open
zcanann wants to merge 1 commit intoencounter:mainfrom
zcanann:dedup-extabindex-removal
Open

report: strip deduplicated extabindex/extab entries from base object#349
zcanann wants to merge 1 commit intoencounter:mainfrom
zcanann:dedup-extabindex-removal

Conversation

@zcanann
Copy link
Copy Markdown

@zcanann zcanann commented Apr 25, 2026

When --deduplicate is active, weak/global functions already seen in earlier translation units are skipped for code accounting. However, their associated extabindex and extab section entries remained in the base object, causing section layout mismatches and incorrect data match percentages.

This patch physically removes extabindex entries (and their corresponding extab entries) from the base object before diffing, when the entry's function relocation targets a deduplicated symbol. Section data, relocations, sizes, and symbol addresses are all adjusted to maintain consistency.

@zcanann
Copy link
Copy Markdown
Author

zcanann commented Apr 25, 2026

In fact, the game.MAP files from FFCC show this quite clearly:

FFCC MAP Evidence: Weak Symbol Deduplication

Excerpts from Final Fantasy Crystal Chronicles (GCCE01) game.MAP showing the Metrowerks linker's weak symbol deduplication behavior. All three std::exception symbols are taken from New.cp (first occurrence) and duplicates in Gecko_ExceptionPPC.cp (and others) are discarded by the linker.

__vt__Q23std9exception (vtable — data)

                 16] __vt__Q23std9exception (object,weak) found in Runtime.PPCEABI.H.a New.cp
                 16] >>> UNREFERENCED DUPLICATE __vt__Q23std9exception
                 16] >>> (object,weak) found in MSL_C++.PPCEABI.bare.H.a locale.cpp
                 16] >>> (object,weak) found in MSL_C++.PPCEABI.bare.H.a iostream.cpp
                 16] >>> (object,weak) found in MSL_C++.PPCEABI.bare.H.a ios.cpp
                 16] >>> (object,weak) found in Runtime.PPCEABI.H.a Gecko_ExceptionPPC.cp
                 16] >>> (object,weak) found in Runtime.PPCEABI.H.a MWRTTI.cp
                 16] >>> (object,weak) found in Runtime.PPCEABI.H.a NewMore.cp

what__Q23std9exceptionCFv (std::exception::what() — code)

                  17] what__Q23std9exceptionCFv (func,weak) found in Runtime.PPCEABI.H.a New.cp
                  17] >>> UNREFERENCED DUPLICATE what__Q23std9exceptionCFv
                  17] >>> (func,weak) found in MSL_C++.PPCEABI.bare.H.a locale.cpp
                  17] >>> (func,weak) found in MSL_C++.PPCEABI.bare.H.a iostream.cpp
                  17] >>> (func,weak) found in MSL_C++.PPCEABI.bare.H.a ios.cpp
                  17] >>> (func,weak) found in Runtime.PPCEABI.H.a Gecko_ExceptionPPC.cp
                  17] >>> (func,weak) found in Runtime.PPCEABI.H.a MWRTTI.cp
                  17] >>> (func,weak) found in Runtime.PPCEABI.H.a NewMore.cp

__dt__Q23std9exceptionFv (std::exception::~exception() — code)

                  17] __dt__Q23std9exceptionFv (func,weak) found in Runtime.PPCEABI.H.a New.cp
                  17] >>> UNREFERENCED DUPLICATE __dt__Q23std9exceptionFv
                  17] >>> (func,weak) found in MSL_C++.PPCEABI.bare.H.a locale.cpp
                  17] >>> (func,weak) found in MSL_C++.PPCEABI.bare.H.a iostream.cpp
                  17] >>> (func,weak) found in MSL_C++.PPCEABI.bare.H.a ios.cpp
                  17] >>> (func,weak) found in Runtime.PPCEABI.H.a Gecko_ExceptionPPC.cp
                  17] >>> (func,weak) found in Runtime.PPCEABI.H.a MWRTTI.cp
                  17] >>> (func,weak) found in Runtime.PPCEABI.H.a NewMore.cp

Summary

The linker picks the first occurrence of each weak symbol from New.cp and marks all others as >>> UNREFERENCED DUPLICATE. Each weak symbol emitted by MWCC in every TU that includes the inline virtuals gets stripped — along with its associated extabindex/extab entries. This is exactly what objdiff's --deduplicate flag needs to simulate when comparing compiled objects against target objects split from the linked DOL.

@zcanann zcanann force-pushed the dedup-extabindex-removal branch 2 times, most recently from 22c3be4 to 7e29b79 Compare April 25, 2026 22:12
When --deduplicate is active, weak/global functions already seen in
earlier translation units are skipped for code accounting. However,
their associated extabindex and extab section entries remained in the
base object, causing section layout mismatches and incorrect data
match percentages.

This patch physically removes extabindex entries (and their
corresponding extab entries) from the base object before diffing,
when the entry's function relocation targets a deduplicated symbol.
Section data, relocations, sizes, and symbol addresses are all
adjusted to maintain consistency.
@zcanann
Copy link
Copy Markdown
Author

zcanann commented Apr 25, 2026

This is the PR that sparked my investigation into this, in BFBB I can get to 99.67% data match on a fully linking.

bfbbdecomp/bfbb#739

With this patch, the objdiff report shows 100% data.

@zcanann
Copy link
Copy Markdown
Author

zcanann commented Apr 26, 2026

Similarly, here is the FFCC PR that shows how we used to get 100% data match by hacking around objdiff, but after removing the hacks, data is 99.77% matched: zcanann/FFCC-Decomp#10488

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant