Docker: Error response from daemon: Unknown runtime specified habana

System Setup
1

Hello,

I am getting this error messages after I try to execute docker run command

system12@system12-X12DPG-OA6-GD2:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 5a81c4b8502e 3 weeks ago 77.8MB
vault.habana.ai/gaudi-docker/1.10.0/ubuntu20.04/habanalabs/pytorch-installer-2.0.1 latest ecb628a5566b 4 weeks ago 4.31GB
hello-world latest 9c7a54a9a43c 2 months ago 13.3kB
system12@system12-X12DPG-OA6-GD2:~$ docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host -v $HOME/shared:/root/shared vault.habana.ai/gaudi-docker/1.10.0/ubuntu20.04/habanalabs/pytorch-installer-2.0.1
docker: Error response from daemon: Unknown runtime specified habana.
See ‘docker run --help’.

I did a quick check, if i remove the runtime=habana, it can execute the docker image… have you guys encounter his before?

Thanks.

2

I am able to execute the docker command you posted:

docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host -v $HOME/shared:/root/shared vault.habana.ai/gaudi-docker/1.10.0/ubuntu20.04/habanalabs/pytorch-installer-2.0.1

Sometimes there may be invisible special characters instead of space (especially if you copy from somewhere). Can you try typing out the " --runtime=habana " part by hand instead of copying it

3

Hello,

The issue has been resolved. Since this is a fresh server, we need to install the container runtime first

https://docs.habana.ai/en/latest/Installation_Guide/Bare_Metal_Fresh_OS.html#driver-fw-install-bare

Thanks.

4

Hi,

Weird, I have no issue executing this command in the previous Habana Gaudi 1 server, but in this fresh Habana 2 Gaudi server, seems like I am having issue with --runtime=habana. Is the docker version matters? I am using Ubuntu 20.04.2. I retype everything and I am pretty sure there is no special character or spacing for --runtime=habana command.

system12@system12-X12DPG-OA6-GD2:~$ docker --version
Docker version 20.10.21, build 20.10.21-0ubuntu1~20.04.2

Thanks.

5

This error can be from the following reasons:

  1. Package habanalabs-container-runtime wasn’t installed
  2. File /etc/docker/daemon.json wasn’t created or has wrong format (after file created, docker daemon need to be restarted)
  3. Form some reason you don’t have permissions to execute /usr/bin/habana-container-runtime