Issues

• Calculate Trainable Parameters Correctly for extend vocab strategy.

  bigscience-workshop/multilingual-modeling#42 · Filter by author yongzx opened on Jul 7, 2022

• [Lower Priority] Implement "Fisher Induced Sparse uncHanging (FISH) Mask"

  bigscience-workshop/multilingual-modeling#38 · Filter by author haileyschoelkopf opened on Jul 1, 2022

• [Low Prio] Implement Diff-Pruning

  bigscience-workshop/multilingual-modeling#37 · Filter by author yongzx opened on Jul 1, 2022

• Trainable Parameters in madx_clm_run.py are incorrect for extend strategy.

  bigscience-workshop/multilingual-modeling#36 · Filter by author yongzx opened on Jul 1, 2022

• Implement Ladder Side-Tuning

  bigscience-workshop/multilingual-modeling#35 · Filter by author haileyschoelkopf opened on Jun 29, 2022
  1

• Implement "Adaptable Adapters"

  bigscience-workshop/multilingual-modeling#34 · Filter by author haileyschoelkopf opened on Jun 29, 2022
  2

• Implement Prefix-Tuning.

  bigscience-workshop/multilingual-modeling#30 · Filter by author yongzx opened on Jun 28, 2022
  

• Package the code up neatly to pass to JZ.

  bigscience-workshop/multilingual-modeling#29 · Filter by author yongzx opened on Jun 28, 2022
  1

• Make generation task (XLSUM) work for decoder-only model and follow the original setting.

  bigscience-workshop/multilingual-modeling#26 · Filter by author yongzx opened on Jun 26, 2022
  

• Control Extra Params (use Adapter 16x reduction size as control)

  bigscience-workshop/multilingual-modeling#25 · Filter by author yongzx opened on Jun 26, 2022

• Implement “Composable Sparse Fine-tuning for Cross-Lingual Transfer”

  bigscience-workshop/multilingual-modeling#23 · Filter by author haileyschoelkopf opened on Jun 26, 2022

• Inconsistent Evaluation Results

  bigscience-workshop/multilingual-modeling#17 · Filter by author yongzx opened on Jun 22, 2022

• Adding Language specific validation sets to deepspeed

  bigscience-workshop/multilingual-modeling#1 · Filter by author hadyelsahar opened on Sep 8, 2021
  4