Release of pre-trained models

[rohit-gupta]

Hi,

This is a really valuable work, in particular I really like that you have results for various degrees of finetuning of language and vision encoders.

I am interested in evaluating some of your models on additional tasks, for that purpose it would be useful if weights for all the models mentioned in Table 1 were released. Is there a plan to do do ?

Thanks

[codezakh]

Hi Rohit, we definitely plan to release the model weights. Give me a day or two to upload them.

[codezakh]

@rohit-gupta Are there any in particular you want, or just all of them? Uploading all of them will take some time (each checkpoint is about .7GB).

[rohit-gupta]

@codezakh Would ultimately like to test all, but even just LilT-LwA-base and CLIP-base to start off with could help provide some preliminary conclusions about the problem

[codezakh]

Here you go: https://drive.google.com/drive/folders/1hfNlZVohcdoWnx-rrQG-BXUHcYjTDNW1
It contains base sized models trained on 500k for each of LilT-{DA,LwA}, LiT, and CLIP.

The others will take a bit longer to add, because I have to manually transfer them from long-term storage and upload them.

[mayug]

Hi, are the 1.5M samples trained model checkpoints available ?

Thanks