Multi-Surrogate-Objective Optimization for Topic Models

Setup

1. Install the required libraries:

   numpy==1.26.4
   torch_kmeans==0.2.0
   pytorch==2.2.0
   sentence_transformers==2.2.2
   scipy==1.10
   bertopic==0.16.0
   gensim==4.2.0

2. Install Java and download this JAR file to ./evaluations/palmetto.jar.
3. Download and extract this Wikipedia corpus to ./datasets/wikipedia/.

Usage

To run the model, use the command:

python main.py --model <MODEL_NAME> --dataset <DATASET_NAME> --num_topics 50 --beta_temp 0.1 --num_groups 20 --weight_ECR 5 --alpha_ECR 20  --weight_GR 2  --alpha_GR 5  --weight_InfoNCE 50  --theta_temp 1.0  --DT_alpha 3.0 --TW_alpha 2.0 --epochs 500 --device cuda --lr 0.002 --use_pretrainWE --use_MOO <USE_MSOO> --MOO_name <MOO_METHOD> --learn 0 --coef_ 0.5

Options:

• Models: ECRTM, NeuroMax, FASTopic
• Datasets: AGNews, YahooAnswers, 20NG
• To not use MSOO, set use_MOO=0.
• To use MSOO, set use_MOO=1 and specify MOO_name from: MGDA, PCGrad, IMTL, ExcessMTL, FairGrad
• To use MSOO-A set learn=1

Acknowledgement

Some part of this implementation is based on TopMost. We also utilizes Palmetto for the evaluation of topic coherence.