PyTorch Training getting keyword error - IMDB Tutorial Example

When I run through the IMDB tutorial on a DL1 instance (Gaudi-tutorials/distilbert_imdb.ipynb at main · HabanaAI/Gaudi-tutorials · GitHub), I get a keyword error when trying to start up the TrainingArguments:

The version of transformers I am using is from the HabanaAI GitHub (following installation instructions in the tutorial):

!git clone https://github.com/HabanaAI/Model-References.git
!sudo pip install Model-References/PyTorch/nlp/finetuning/huggingface/bert/transformers/.
transformers.__version__
'4.19.2'
from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments
training_args = TrainingArguments(
    use_habana=True,
    use_lazy_mode=True,
    use_fused_adam=True,
    use_fused_clip_norm=True,
    hmp=True,
    hmp_bf16='./ops_bf16_distilbert_pt.txt',
    hmp_fp32='./ops_fp32_distilbert_pt.txt',
    hmp_verbose=False,
    output_dir='./results',          # output directory
    num_train_epochs=3,              # total number of training epochs
    per_device_train_batch_size=16,  # batch size per device during training
    per_device_eval_batch_size=64,   # batch size for evaluation
    warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    logging_dir='./logs',            # directory for storing logs
    logging_steps=10,
)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [28], in <cell line: 1>()
----> 1 training_args = TrainingArguments(
      2     use_habana=True,
      3     use_lazy_mode=True,
      4     use_fused_adam=True,
      5     use_fused_clip_norm=True,
      6     hmp=True,
      7     hmp_bf16='./ops_bf16_distilbert_pt.txt',
      8     hmp_fp32='./ops_fp32_distilbert_pt.txt',
      9     hmp_verbose=False,
     10     output_dir='./results',          # output directory
     11     num_train_epochs=3,              # total number of training epochs
     12     per_device_train_batch_size=16,  # batch size per device during training
     13     per_device_eval_batch_size=64,   # batch size for evaluation
     14     warmup_steps=500,                # number of warmup steps for learning rate scheduler
     15     weight_decay=0.01,               # strength of weight decay
     16     logging_dir='./logs',            # directory for storing logs
     17     logging_steps=10,
     18 )

TypeError: __init__() got an unexpected keyword argument 'use_habana'

Would appreciate your help. Thanks!

1 Like

Hi,
Thanks for the post. It seems that between 1.5 and 1.6 the argument changed its name from use_habana to use_hpu

1.5 argument here
1.6 argument here

Could you please try use_hpu

1 Like

@silvos just to be sure, are you using 1.5 or 1.6?