Preview - /dev/net/tun

Guys.

I have one last issue with my self hosted in that docker cannot start due to a mapping issue with /dev/net/tun. I am running k3s latest on Centos Stream.

I am happy to dig into it but from what I read it looks like you need to pass additional privs for it when it starts.

Can you shed any light on it. There is a issue on github already but does not seem to be getting much traction.

I am really good with the backend stuff…

I’m in a similar situation. I just deployed 0.7.0-beta1 on kubernetes 1.19. If I disable Feature Preview, I can create workspaces. If I turn on Feature Preview, workspace pods fail to start with MountVolume.SetUp failed for volume "dev-net-tun" : hostPath type check failed: /dev/net/tun is not a file, but /dev/net/tun exists on the host running the pod.

$ ls -ld /dev/net/tun
crw-rw-rw- 1 root root 10, 200 Feb 19 19:41 /dev/net/tun

@johnktims which distribution are you running on?

The underlying issue you’re experiencing is that we try to mount /dev/net/tun as file, which fails on some distributions. We should mknod the device instead.

Also, please beware that “Feature Preview” currently incurs a hard Ubuntu (>= 18.04) dependency. There is work underway to change that.

Hello.

I assume you mean in the container. I have custom containers and can test it. Other than this issue, Gitpod is working on Centos Stream and I am happy to test things for you…

I run Ubuntu 18.4 with shiftfs module loaded. I hope this resolves the dependency.
I replaced the char device tun with a simple file. The pod can mount it an tries to pull the image. But, this fails with

Failed to pull image "reg.domain.com:3000/remote/c30738c1-1007-474c-9e42-9c712d13449e": rpc error: code = Unknown desc = Error response from daemon: mediaType in manifest should be 'application/vnd.docker.distribution.manifest.v2+json' not ''

@csweichel I’m using Ubuntu 20.04. I recreated /dev/net/tun using mknod /dev/net/tun c 10 200 and I’m still getting MountVolume.SetUp failed for volume "dev-net-tun" : hostPath type check failed: /dev/net/tun is not a file and Unable to attach or mount volumes: unmounted volumes=[dev-net-tun], unattached volumes=[vol-this-workspace dev-net-tun daemon-mount]: timed out waiting for the condition.

Since /dev/net/tun appears to be a special character device, should https://github.com/gitpod-io/gitpod/blob/945041dc6845926bddcbe299f644698cb1e8c003/components/ws-manager/pkg/manager/create.go#L372 be changed from HostPathFile to HostPathCharDev?

@johnktims

hostPath type check failed: /dev/net/tun is not a file

Yes, that’s expected, for the reason you discovered. as @csweichel posted we’re planning to resolve the issue generically by creating one via mknod for every workspace.

Thanks @geropl. Do you have any idea when the fix might be released?

Hello all. So I got a moment to create a replica of full-workspace with the mknod.
I am no longer getting the hostpath check but the pod is not starting either.

Events:
Type Reason Age From Message


Warning FailedScheduling 24s workspace-scheduler no suitable node found: zero nodes available

My guess on this one is I am missing a label or toleration. Any ideas?

No, but here’s the issue to upvote/follow: https://github.com/gitpod-io/gitpod/issues/2706

I am getting this error on 0.9.0-alpha1 - using ubuntu 18.04, k3s (v1.20.6+k3s1).

Is there something extra I need to install on my nodes for this to go away?