Question Regarding History Modality in Planning/Navigation Data

[gpt-dance]

Hello, I noticed that the previous actions in planning or navigation data is pure text, So the evalution is the same? Hi,
I noticed that in the planning or navigation datasets, the history of previous actions is represented purely as text.
In this setting, is the evaluation conducted in the same way as other modalities, or are there any modality-specific differences we should be aware of?

Thanks in advance.

[liu-zhy]

Yes. To save the GPU memory, we only use pure text history during training as well as inference. BTW, we now indeed are studying the interleaved image-text context in computer-using agents, which are expected to achieve higher performance.