Update moe branch for torchrun and fix error while loading trainer.ds_module#7

pnunna93 wants to merge 15 commits into

munael:feature/deepspeed_moemunael/fairseq:feature/deepspeed_moefrom

pnunna93:feature/deepspeed_moepnunna93/fairseq:feature/deepspeed_moeCopy head branch name to clipboard