General questions & caching packages/artifacts

Hi,

I’ve been trying out GitPod for a few days now and it looks really really promising!

There are a few things I’d like to know/hear more about.

  • Do you have some sort of roadmap or list of features you are working on? Maybe having a public https://canny.io account where the community can vote on issues and see the progress being made on them?

  • Is it possible to set environment variables during the init phase? e.g. for installing private npm or composer packages. Currently the environment variables set in the account are only available in the command cmd

  • Currently only a single Dockerfile or image can be provided. Are there plans for supporting multiple services like docker-compose? or docker-in-docker as an alternative?

  • Is there a more (technical) in depth documentation how the build phase works? Which paths are persistent, which paths are mounted, etc

  • Is it possible to have some sort of caching or artifact mechanism? Prebuilds work really nice, but it still needs to run processes like npm install or composer install without any caching and this can still take a long time to complete. This could be a problem when you work in a team where multiple people are working on the same branches and builds are triggered often when there are a lot of code pushes.

  • How can you externally connect to the workspace? e.g. I’m using a Sequel Pro as my database tool. I’d like to be able to ssh into the workspace and connect to the MySQL instance

Thanks!
Rick

Hi Rick,

Those are really great questions. Please find my answers inline below:

  • Do you have some sort of roadmap or list of features you are working on? Maybe having a public https://canny.io account where the community can vote on issues and see the progress being made on them?

We do not have that at the moment. The current approach is to use GitHub issues, where we ask everyone to add their use cases or some other kind of signal (e.g. :+1:) that a certain feature/bug is important to them. But we are not happy with this and want to get to a clear monthly cadence with a roadmap listing the most important features we are working on. We’ll have a look at canny.io as well. Thanks for the suggestion.

  • Is it possible to set environment variables during the init phase? e.g. for installing private npm or composer packages. Currently the environment variables set in the account are only available in the command cmd

The init phase runs during prebuilds and the outcome is used by everyone who has access to that repository and starts a workspace. ATM prebuilds are running as the user who configured them (i.e. installed the GH app, resp. installed teh webhook on GitLab/Bitbucket). We will change that by introducing organizations. On that level you will be able to configure your org’s projects and also set env variables for prebuilds.

  • Currently only a single Dockerfile or image can be provided. Are there plans for supporting multiple services like docker-compose? or docker-in-docker as an alternative?

We will soon allow root and docker in docker. With that you will also be able run docker compose within your workspaces. ETA for the Beta is December.

  • Is there a more (technical) in depth documentation how the build phase works? Which paths are persistent, which paths are mounted, etc

It is somewhat spreaded through the docs. The path question is answered here: https://www.gitpod.io/docs/life-of-workspace/#changes-are-saved
That said we have already implemented something we call “full-workspace backup”, which as the name suggests backs up everything. That feature is currently hidden behind a feature flag and will be rolled out in early January.

  • Is it possible to have some sort of caching or artifact mechanism? Prebuilds work really nice, but it still needs to run processes like npm install or composer install without any caching and this can still take a long time to complete. This could be a problem when you work in a team where multiple people are working on the same branches and builds are triggered often when there are a lot of code pushes.

Another great question that we will address soon. The feature we are working to mitigate this is called ‘updateable prebuilds’ internally. It will allow base a prebuild on a previous prebuild, hence run an incremental build on top. We thought about an explicit caching machanism similar to what CI tools have, but believe leveraging the mechanisms that are built into all the build tools already is simpler and more reliable.

  • How can you externally connect to the workspace? e.g. I’m using a Sequel Pro as my database tool. I’d like to be able to ssh into the workspace and connect to the MySQL instance

An SSH mode is pretty high on the roadmap and will land early next year. For now you can use https://github.com/jpillora/chisel and create an SSH tunnel.

1 Like

Hi @svenefftinge, thanks for your excellent reply!

I really look forward to using all the features you mentioned. When all of those are implemented GitPod would fit perfectly within our workflow as an agency with a large team and lots of projects :smiley:

3 Likes

@svenefftinge could you tell me a bit more about the available resources per workspace when using the hosted version of GitPod? I’m running a rather large Drupal site on it now, but it still feels quite slow compared to my local Docker (and docker for mac can be quite slow when using large file mounts :stuck_out_tongue:)