TRACE definition differs between paper and code

[chenxn2020]

The TRACE definition in the paper and released evaluator appears inconsistent.

The paper defines citation fidelity using only:

• Consistency (Con.)
• Coverage (Cov.)
• Textual Fidelity (Fid.)

with VEF as a fixed 40% component of TRACE:

$$\mathrm{TRACE} = 100 \left[ 0.4,\mathrm{VEF} + 0.6 \sum_{k \in {\mathrm{Con},\mathrm{Cov},\mathrm{Fid}}} W_k E_k\right]$$

However, scoring_evidence.py includes an additional Diversity dimension:

EVI = 100 * (
    w_con * E_con + w_cov * E_cov
    + w_fid * E_fid + w_div * E_div
)

This changes TRACE/EVI and the final MMDR score.

Could you clarify which definition was used for the paper results: the three-dimensional paper formula or the four-dimensional public implementation?

Paper: https://arxiv.org/abs/2601.12346v1
Code: https://github.com/AIoT-MLSys-Lab/MMDeepResearch-Bench/blob/main/detail/scoring_evidence.py

[MMDeepResearch-Bench]

Thank you for identifying this inconsistency. The published leaderboard used the four-dimensional EVI implementation:
EVI = 100 × Σₖ WₖEₖ, where k ∈ {Consistency, Coverage, Fidelity, Diversity}.
VEF was evaluated separately and combined as:
TRACE = 0.6 × EVI + 0.4 × VEF.
The omission of Diversity from the paper’s displayed definition was a documentation error. We will correct the equation and provide the complete computation logic in the revised manuscript and appendix. This clarification does not require recomputing the published leaderboard.

[chenxn2020]

Thank u