When I run through the IMDB tutorial on a DL1 instance (Gaudi-tutorials/distilbert_imdb.ipynb at main · HabanaAI/Gaudi-tutorials · GitHub), I get a keyword error when trying to start up the TrainingArguments:
The version of transformers I am using is from the HabanaAI GitHub (following installation instructions in the tutorial):
!git clone https://github.com/HabanaAI/Model-References.git
!sudo pip install Model-References/PyTorch/nlp/finetuning/huggingface/bert/transformers/.
transformers.__version__
'4.19.2'
from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments
training_args = TrainingArguments(
use_habana=True,
use_lazy_mode=True,
use_fused_adam=True,
use_fused_clip_norm=True,
hmp=True,
hmp_bf16='./ops_bf16_distilbert_pt.txt',
hmp_fp32='./ops_fp32_distilbert_pt.txt',
hmp_verbose=False,
output_dir='./results', # output directory
num_train_epochs=3, # total number of training epochs
per_device_train_batch_size=16, # batch size per device during training
per_device_eval_batch_size=64, # batch size for evaluation
warmup_steps=500, # number of warmup steps for learning rate scheduler
weight_decay=0.01, # strength of weight decay
logging_dir='./logs', # directory for storing logs
logging_steps=10,
)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [28], in <cell line: 1>()
----> 1 training_args = TrainingArguments(
2 use_habana=True,
3 use_lazy_mode=True,
4 use_fused_adam=True,
5 use_fused_clip_norm=True,
6 hmp=True,
7 hmp_bf16='./ops_bf16_distilbert_pt.txt',
8 hmp_fp32='./ops_fp32_distilbert_pt.txt',
9 hmp_verbose=False,
10 output_dir='./results', # output directory
11 num_train_epochs=3, # total number of training epochs
12 per_device_train_batch_size=16, # batch size per device during training
13 per_device_eval_batch_size=64, # batch size for evaluation
14 warmup_steps=500, # number of warmup steps for learning rate scheduler
15 weight_decay=0.01, # strength of weight decay
16 logging_dir='./logs', # directory for storing logs
17 logging_steps=10,
18 )
TypeError: __init__() got an unexpected keyword argument 'use_habana'
Would appreciate your help. Thanks!