PrimeVul exact metrics

[itwastony]

Thank you for your work. Im curious—as I understand it, you evaluated your model on a C/C++ dataset that consisted of many other datasets, but did you also evaluate it on the PrimeVul test set alone?

[rucnyz]

Hi @itwastony . Yes, we did evaluate on the PrimeVul test set alone. However, we found that PrimeVul contains a fair amount of noisy labels — for example, single-line function calls labeled as CWE vulnerabilities. Due to this noise, the performance differences across models were not particularly significant, so we opted not to include those results in the paper.