Cannot initialize workspace (Self-Hosted GKE)

Gitpod Self-Hosted
#1

Hello Team,

I have installed self-hosted gitpod in kubernetes cluster on google cloud.
Few of user are able to access new repositories and able to create and run the workspace but few users are facing below issue:

image
image1107×328 25.4 KB

Can you please help me with the reason and workaround for this?

#2

Hi @sumitrathi95,

I have installed self-hosted gitpod in kubernetes cluster on google cloud.

cool. Which version did you install?

The error suggests you configured custom storage provider but somehow the credentials are not valid. Could you share your config YAML?

Cheers

1 Like
#3

I have installed gitpod-0.4.0
Which config YAML shall I share with you? (Can you please provide the filename)

#4

FYI: We recently released version 0.6.0 and updated the docs.

Which config YAML shall I share with you? (Can you please provide the filename)

Judging from the error message there’s sth wrong with GCP authentication. Could you share the related files (please remove secrets before).

#5

I am sending the values.yaml and values/gcp/buckets.yaml

    values.yaml
    gitpod:
      # disabling minio
      minio:
         disabled: true
            
          # This field must be set to your domain name. Leaving it set to its default value will result in
          # a non-functional installation.
          hostname: hostname

          # If you have a static IP that your domain resolves to, set it here.
          # Leaving this field set to its default value is fine. Kubernetes will assign you an IP address
          # during deployment.
          components:
            proxy:
              loadBalancerIP: IP_addr
            wsSync:
              kind: gcloud
              gcloud:
                credentialsFile: file.json
                projectId: gitpod-proj
            imageBuilder:
              registryCerts: []
              registry:
                # name must not end with a "/"
                name: gcr.io/gitpod-project123
                secretName: image-builder-registry-secret
                path: secrets/account.json
            workspace:
              pullSecret:
                secretName: image-builder-registry-secret

          docker-registry:
            enabled: false

          # Gitpod needs at least one auth provider to allow users to log in.
          # The auth providers below are examples only. Please change/remove them to fit your installation.
          authProviders:
          - id: "Gitlab"
            host: "gitlab_url"
            protocol: "https"
            type: "GitLab"
            oauth:
              clientId: "clientId"
              clientSecret: "clientSecret"
              callBackUrl: "callBackUrl"
              settingsUrl: "settingsUrl"

          # RBAC is enabled by default. If your cluster does not use RBAC, set this flag to false so that
          # we do not attempt to install PodSecurityPolicies and the likes.
          installPodSecurityPolicies: true
          
        gitpod_selfhosted:
          variants:
            customRegistry: true


        ------

        buckets.yml

        # Use GCP Buckets for workspace backups.
        # There is only one value you need to change in this file: projectId

        gitpod:
          components:
            wsSync:
              volumes:
              - name: gcloud-creds
                secret:
                  secretName: gcp-ws-sync-key
              - name: gcloud-tmp
                hostPath:
                  path: /mnt/disks/ssd0/sync-tmp
                  type: DirectoryOrCreate
              volumeMounts:
              - mountPath: /credentials
                name: gcloud-creds
              - mountPath: /mnt/sync-tmp
                name: gcloud-tmp
              remoteStorage:
                kind: gcloud
                gcloud:
                  # You need to set your GCP project ID here.
                  # Beware: the name of your project is not the same as its ID. You can find the project ID under the "Home" page of your GCP project.
                  projectId: some-gcp-project-id
                  # The GCP region you want the workspace content to be stored in. This should ideally be in the same region as your cluster.
                  region: some-gcp-region
                  # You shouldn't have to change the values below if you're using the templates that ship with this chart.
                  credentialsFile: /credentials/gitpod-workspace-syncer-key.json
                  tmpdir: /mnt/sync-tmp
                  parallelUpload: 6
            server:
              storage:
                secretName: gcp-server-storage-key
                keyFilePath: secrets/gitpod-workspace-syncer-key.json

          minio:
            enabled: false
#6

Hi @sumitrathi95,

could you please check if gitpod-workspace-syncer-key.json is correctly imported.

#7

Hi @wulfthimm,

Yes! I have checked the configuration of gcp-ws-sync-key it is correct and imported properly.

#8

Hello @wulfthimm,

I have checked the permission to service account, they are also the same as provided while installing gitpod self-hosted. Right now, old users are able to connect and work on workspaces but new users are facing the issue.

#9

could you please check if gitpod-workspace-syncer-key.json is correctly imported.

I have checked the configuration of gcp-ws-sync-key it is correct and imported properly.

Here is definitely a mismatch: Both server and wsSync need the same secret to access the storage.

Right now, old users are able to connect and work on workspaces but new users are facing the issue.

I wonder how this might happen: wsSync creates those buckets on first usage. So it seems there have been old credentials that worked and new ones that doesn’t.