Wrong answers in `cwq.json`

[ztshao-nju]

Hi, can you share the processing procedure for cwq?
I find that there is only one answer for each question in cwq.json, which is strange.

For example, question "What language is spoken in the location that appointed Michelle Bachelet to a governmental position speak?" with ID "WebQTrn-2069_0fa727f3b282196eb1097410b4be6818" has 5 answers derived from sparql code ['Aymara language', 'Mapudungun Language', 'Puquina Language', 'Rapa Nui Language', 'Spanish Language'].
However, there is only one answer "Spanish Language" in cwq.json.

I also check the answers of this question in other papers, which also have 5 answers.

Thanks.

[GasolSun36]

Hi, can you share the processing procedure for cwq? I find that there is only one answer for each question in cwq.json, which is strange.

For example, question "What language is spoken in the location that appointed Michelle Bachelet to a governmental position speak?" with ID "WebQTrn-2069_0fa727f3b282196eb1097410b4be6818" has 5 answers derived from sparql code ['Aymara language', 'Mapudungun Language', 'Puquina Language', 'Rapa Nui Language', 'Spanish Language']. However, there is only one answer "Spanish Language" in cwq.json.

I also check the answers of this question in other papers, which also have 5 answers.

Thanks.

We did not make any modifications to the cwq validation set. For the cwq test set, since there was no topic entity, we first performed entity extraction and then mapped the entity-id according to our API.

[Atroxgod]

Hi, can you share the processing procedure for cwq? I find that there is only one answer for each question in cwq.json, which is strange.

For example, question "What language is spoken in the location that appointed Michelle Bachelet to a governmental position speak?" with ID "WebQTrn-2069_0fa727f3b282196eb1097410b4be6818" has 5 answers derived from sparql code ['Aymara language', 'Mapudungun Language', 'Puquina Language', 'Rapa Nui Language', 'Spanish Language']. However, there is only one answer "Spanish Language" in cwq.json.

I also check the answers of this question in other papers, which also have 5 answers.

Thanks.

bro，Have you solved this problem? I found that even though some LLMs generate correct answers, they are just aliases. However, these do not match the single answer provided in the CWQ dev dataset.