Can I train DistilBERT with Gaudi?

Can I train DistilBERT with Gaudi? How?

1 Like

Hi Jonathan,
Distilbert for PyTorch is under progress. You can track Distilbert (and other models) progress in our roadmap here. As for Distilbert for Tensorflow, you can follow the instructions on the migration guide here and port your Distilbert model to Gaudi. If you run into issues, please reach out to us.

2 Likes