Issues

• TP does not work on Flan-T5

  EleutherAI/oslo#215 · Filter by author shreyansh26 opened on Nov 16, 2023

• is there any plan to support llama model?

  EleutherAI/oslo#214 · Filter by author yuyaxiong opened on Sep 12, 2023

• To apply FlashAttention

  EleutherAI/oslo#203 · Filter by author dyanos opened on Jun 19, 2023
  1

  

• TODO : Zero optimizer deparallel

  Data Parallelism
  EleutherAI/oslo#186 · Filter by author hyeinhyun opened on May 11, 2023
  

• TODO : Documentation for Expert Parallel

  Expert Parallelism
  EleutherAI/oslo#175 · Filter by author scsc0511 opened on Apr 5, 2023
  

• TODO: Test Code Enhancement

  EleutherAI/oslo#165 · Filter by author bzantium opened on Mar 23, 2023

• Fix TP embedding layers

  EleutherAI/oslo#152 · Filter by author jason9693 opened on Mar 12, 2023
  

• How to pretrain T5

  EleutherAI/oslo#122 · Filter by author TristanThrush opened on Dec 30, 2022

• TODO: documentation and readme

  Documentation
  EleutherAI/oslo#102 · Filter by author hyunwoongko opened on Nov 14, 2022
  1

  

• PatricStar for Zero

  EleutherAI/oslo#97 · Filter by author dongsungkim opened on Nov 6, 2022

• Integration ZeroDDP and ShardedModelv2 from colossal AI

  EleutherAI/oslo#96 · Filter by author dongsungkim opened on Nov 6, 2022
  1

  

• TODO: Test TP + PP

  Pipeline ParallelismTensor Parallelism
  EleutherAI/oslo#48 · Filter by author hyunwoongko opened on Sep 30, 2022
  

• TODO: clean data communication functions for PP

  Pipeline Parallelism
  EleutherAI/oslo#42 · Filter by author ohwi opened on Sep 26, 2022
  

• wand module not found when running test_mlm.py

  Bug
  EleutherAI/oslo#15 · Filter by author josemlopez opened on Aug 30, 2022
  1

• 'TrainingArguments' object has no attribute 'parallel_mode' when running mBart test

  Bug
  EleutherAI/oslo#14 · Filter by author josemlopez opened on Aug 30, 2022
  1

• Milestone: Make oslo-examples repository

  Milestone
  EleutherAI/oslo#6 · Filter by author hyunwoongko opened on Aug 25, 2022