Gitpod stuck "intializing workspace 3/3" but ws pod running

Hello,

I tried to run gitpod on k8s vanilla, so using nginx ingress. Everythings look ok but it looks like the web browser is not aware than the workspace is ready. He got this message “initilizing workspace 3/3” but the workspace looks like its running:

k logs ws-6a60b602-a51f-48e5-a77d-d815d67ebc9c
{"component":"workspace","severity":"info","time":"2020-09-28T06:45:35.835Z","message":"Deploying VS Code extension \"ms-python.python@2019.11.50794\" from \"/tmp/vscode-extensions/ms-python.python@2019.11.50794/extension/out/client/extension\"","payload":{"name":"root","params":[]}}

Do you have any idea?

Best regards,

Hi,

that is no error message. There should be more in the log. Please check also ws-manger and ws-sync for error messages.
Have you used one of the deployment methods using Terraform or the docker-installer?

Best regards,
Wulf

Hello,
I deployed vanilla k8s then I used gitpod’s helm chart.

I tried to start a new workspace, workspace’s logs sounds ok:

k logs ws-0157ae16-5a39-41c3-8ab8-a3bca0f7612a
{“component”:“workspace”,“severity”:“info”,“time”:“2020-09-29T06:11:40.330Z”,“message”:“Deploying VS Code extension “vscode.typescript-language-features@0.2.1” from “/tmp/vscode-extensions/vscode.@theia!vscode-builtin-typescript-language-features@0.2.1/package/dist/extension””,“payload”:{“name”:“root”,“params”:[]}}
{“component”:“workspace”,“severity”:“info”,“time”:“2020-09-29T06:11:40.332Z”,“message”:“Deploying VS Code extension “ms-vscode.go@0.11.4” from “/tmp/vscode-extensions/ms-vscode.Go@0.11.4/extension/out/src/goMain””,“payload”:{“name”:“root”,“params”:[]}}
{“component”:“workspace”,“severity”:“info”,“time”:“2020-09-29T06:11:40.337Z”,“message”:“Deploying VS Code extension “ms-python.python@2019.11.50794” from “/tmp/vscode-extensions/ms-python.python@2019.11.50794/extension/out/client/extension””,“payload”:{“name”:“root”,“params”:[]}}

Below ws-manager log, i got a 400 status and it seems ws-manager didnt receive any ok message from workspace?

k logs ws-manager-68b8c7dcf4-b6vzw
{“instanceId”:“0157ae16-5a39-41c3-8ab8-a3bca0f7612a”,“message”:“workspace did not respond to ready probe with OK status”,“serviceContext”:{“service”:“ws-manager”,“version”:""},“severity”:“debug”,“status”:400,“time”:“2020-09-29T06:15:17Z”,“url”:“https://bf68e324-c014-4bca-bf5e-cf1293a1a0b6.ws.gitpod.univ-lyon1.fr/gitpod/ready",“userId”:"",“workspaceId”:"”}

Then below ws-sync, it looks like ok, because git cloned my repository:

{“message”:“:petri_dish: ws-sync is up and running. Stop with SIGINT or CTRL+C”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“info”,“time”:“2020-09-23T09:06:33Z”}
{“bucketName”:“gitpod-user-5888fcd2-b9dd-4a0d-aaf5-fc9f421fa7df”,“message”:“Creating bucket”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“time”:“2020-09-23T09:10:17Z”}
{“location”:"/mnt/workingarea/c9faec61-11f3-4e53-8d1b-ffc27b5aea4f",“message”:“using Git initializer”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“time”:“2020-09-23T09:10:17Z”}
{“location”:"/mnt/workingarea/c9faec61-11f3-4e53-8d1b-ffc27b5aea4f/asker",“message”:“Running git clone on workspace”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“stage”:“init”,“time”:“2020-09-23T09:10:17Z”}
{“location”:"/mnt/workingarea/c9faec61-11f3-4e53-8d1b-ffc27b5aea4f/asker",“message”:“Git operations complete”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“info”,“stage”:“init”,“time”:“2020-09-23T09:10:18Z”}
{“instanceId”:“c9faec61-11f3-4e53-8d1b-ffc27b5aea4f”,“message”:“DisposeWorkspace called”,“req”:“id:“c9faec61-11f3-4e53-8d1b-ffc27b5aea4f” “,“serviceContext”:{“service”:“ws-sync”,“version”:””},“severity”:“debug”,“time”:“2020-09-23T09:13:17Z”,“userId”:"",“workspaceId”:""}
{“location”:"/mnt/workingarea/557eb522-3b57-4bc4-ade7-afa80d7e810e",“message”:“using Git initializer”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“time”:“2020-09-28T06:22:50Z”}
{“location”:"/mnt/workingarea/557eb522-3b57-4bc4-ade7-afa80d7e810e/asker",“message”:“Running git clone on workspace”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“stage”:“init”,“time”:“2020-09-28T06:22:50Z”}
{“location”:"/mnt/workingarea/557eb522-3b57-4bc4-ade7-afa80d7e810e/asker",“message”:“Git operations complete”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“info”,“stage”:“init”,“time”:“2020-09-28T06:22:52Z”}
{“location”:"/mnt/workingarea/31ba5cdc-16ff-43a9-8948-bfbd7d609740",“message”:“using Git initializer”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“time”:“2020-09-28T06:39:02Z”}
{“location”:"/mnt/workingarea/31ba5cdc-16ff-43a9-8948-bfbd7d609740/asker",“message”:“Running git clone on workspace”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“stage”:“init”,“time”:“2020-09-28T06:39:02Z”}
{“location”:"/mnt/workingarea/31ba5cdc-16ff-43a9-8948-bfbd7d609740/asker",“message”:“Git operations complete”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“info”,“stage”:“init”,“time”:“2020-09-28T06:39:02Z”}
{“instanceId”:“557eb522-3b57-4bc4-ade7-afa80d7e810e”,“message”:“DisposeWorkspace called”,“req”:“id:“557eb522-3b57-4bc4-ade7-afa80d7e810e” “,“serviceContext”:{“service”:“ws-sync”,“version”:””},“severity”:“debug”,“time”:“2020-09-28T06:39:07Z”,“userId”:"",“workspaceId”:""}
{“location”:"/mnt/workingarea/6a60b602-a51f-48e5-a77d-d815d67ebc9c",“message”:“using Git initializer”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“time”:“2020-09-28T06:45:21Z”}
{“location”:"/mnt/workingarea/6a60b602-a51f-48e5-a77d-d815d67ebc9c/asker",“message”:“Running git clone on workspace”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“stage”:“init”,“time”:“2020-09-28T06:45:21Z”}
{“location”:"/mnt/workingarea/6a60b602-a51f-48e5-a77d-d815d67ebc9c/asker",“message”:“Git operations complete”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“info”,“stage”:“init”,“time”:“2020-09-28T06:45:22Z”}
{“instanceId”:“31ba5cdc-16ff-43a9-8948-bfbd7d609740”,“message”:“DisposeWorkspace called”,“req”:“id:“31ba5cdc-16ff-43a9-8948-bfbd7d609740” “,“serviceContext”:{“service”:“ws-sync”,“version”:””},“severity”:“debug”,“time”:“2020-09-28T06:45:27Z”,“userId”:"",“workspaceId”:""}
{“instanceId”:“6a60b602-a51f-48e5-a77d-d815d67ebc9c”,“message”:“DisposeWorkspace called”,“req”:“id:“6a60b602-a51f-48e5-a77d-d815d67ebc9c” “,“serviceContext”:{“service”:“ws-sync”,“version”:””},“severity”:“debug”,“time”:“2020-09-28T07:15:37Z”,“userId”:"",“workspaceId”:""}
{“location”:"/mnt/workingarea/0157ae16-5a39-41c3-8ab8-a3bca0f7612a",“message”:“using Git initializer”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“time”:“2020-09-29T06:11:25Z”}
{“location”:"/mnt/workingarea/0157ae16-5a39-41c3-8ab8-a3bca0f7612a/asker",“message”:“Running git clone on workspace”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“debug”,“stage”:“init”,“time”:“2020-09-29T06:11:25Z”}
{“location”:"/mnt/workingarea/0157ae16-5a39-41c3-8ab8-a3bca0f7612a/asker",“message”:“Git operations complete”,“serviceContext”:{“service”:“ws-sync”,“version”:""},“severity”:“info”,“stage”:“init”,“time”:“2020-09-29T06:11:27Z”}

Best regards,

Fixed by Self-hosted Kubernetes - cannot connect to ws-sync