April 30, 2020

Gitpod

Thanks to everyone who was able to make it to the Go Remote Meetup today! I made a mistake with the recording, so we don’t have video :disappointed: but I did type up my meeting notes, so we do have a complete record of the presentation and discussion.

An addition copy of these notes has been shared as a snippet as well.

As a side note, I’m using gitpod while creating this post.

Presentation Summary

Speaker: Christian Weichel (@csweichel) from TypeFox

Lede: Go powers the cloud. Yet, most Go code is still written offline. With the advent of a new generation of Cloud IDEs, such as Eclipse Theia, this is changing.

Who are TypeFox?

TypeFox develops Eclipse Theia, an extensible platform to develop multi-language Cloud & Desktop IDEs with state-of-the-art web technologies, Gitpod, which launches ready-to-code dev environments for your GitHub or GitLab project with a single click, and Xtext, a framework for development of programming languages and domain-specific languages.

What is Gitpod?

Gitpod operates like a Theia hosting service. While developing the Theia project, a local deployment using tools like minikube can be extremely resource intensive. These local deployment and testing environment often are far-removed from a production build and deploy workflow.

To relieve these issues, Chris and the team at TypeFox created Gitpod as a way to offload those resource requirements from the local development environment, bringing the workflow closer to a cloud environment resembling a production environment. They used Docker, Kubernetes, and the debugging proxy telepresence to create Gitpod.

The Gitpod codebase is an open source project; contributions are welcome. Users can self-host Gitpod, use it in the cloud for free on open source projects, or pay a subscription for private, heavy-use, parallel, or enterprise workspaces.

How does it work?

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/4d7a5bd5-e95d-46ba-b1f2-60437fb90bfc/Untitled.png

Gitpod creates a “workspace” which downloads repository project and builds a container to develop and run the project. Theia provides the IDE, Docker the container image, Kubernetes the deployment, and Telepresence the proxy into the workspace. Athens provides a go modules proxy.

What were some challenges? How were these challenges addressed?

Given that the vision is to make Gitpod a fast, closer-to-production environment for development, the team pays particular attention to the build time for workspaces. They looked at (and achieved!) reducing workspace build-time from around one hour down to under 5 minutes.

One approach early on was to optimize using Docker multistage builds, relying on the caching behavior to improve the build-time. They also made use of the Jenkins Kubernetes plugin for the deployment of the workspace container into the Kubernetes hosting cluster.

To improve the total timeline from workspace launch to ready-for-development, Chris and his team ended up rolling their own build process. This allowed them to parallelize the build, making further use of Docker multi-stage capabilities. The added in Athens for fetching the go packages, and leveraged YARN’s capabilities for TypeScript dependencies. These steps, alongside the introduction of Werft as a CI tool and heavy use of image caching, achieved their goals and ultimately reduced overall build-time to about 2 minutes.

The “key ingredient” in making Gitpod successful as an IDE is the Telepresence proxy for remote debugging. With Telepresence, the workspace can move the minikube experience smoothly into a GKE cluster, secured via TLS connection, giving a remote debugging experience almost identical to a local experience.

Discussion Q&A

What are the repository requirements to launch into Gitpod?

The hosted service supports Github and Gitlab natively; as an open source project, self-hosted deployments may make use of other git hosting repositories. Users can further configure their workspace by using parameters in a .gitpod.yml file.

Are there any particular issues moving around large files for testing?

Since the hosted workspace ultimately runs in GKE, the reliability of Google Storage makes it possible to move large amounts of data.

There are some challenges in developing and/or compiling against specialized hardware, architectures, etc. For instance, some tinygo targets may be tricky to test and debug. Similarly with code targeting things like FPGAs.

Does the debugging proxy use the delve protocol?

The Telepresence proxy can run with or without delve. Both ends of the proxy run locally, neither end actually runs in the target cluster. The communication channel is secured via TLS, and a debugger tool like delve can attach to the client end.

(NOTE: I’m not sure I followed this part of the answer correctly. -Jared)

What’s next for Gitpod?

Chris is looking forward to a few features coming out soon:

  • sudo privilages inside the workspace
  • continuous improvement to startup and build-time
  • evolution in Theia, especially OpenVSX, allowing add-on compatibility with VS Code

As an open source project, contributions are welcome! TypeFox has a contributors program which rewards developers with a free license.

How can developers engage in remote pairing sessions?

Gitpod does have an ability to share workspaces. There’s two ways to do this: (1) sharing the running workspace and (2) sharing a snapshot of the workspace.

When sharing a running workspace, it’s important to note that the behavior is equivalent to sharing a laptop — everything about the environment can be accessed by the other developers with the shared link. Crucially, the git repository credentials are shared as part of the environment; although obscured, it’s possible for someone to dig around and find the credentials. So share responsibly and only with people you trust!

When sharing a snapshot, Gitpod creates a copy of the workspace and provides that copy. No credentials get copied, making this a much safer alternative.

There aren’t any features supporting concurrent editing (eg. as seen in VS Code Live Share), but it’s been discussed as a future endeavor.

In what ways did you leverage go in creating Gitpod?

TypeFox is largely a TypeScript shop, and so the earlier development did start with TypeScript. This worked well enough, but the amount of work that interfaces with large go projects (eg. Docker, Kubernetes) meant choosing more tooling and writing more code with go.

In general, says Chris, go services are in many ways much easier to operate. Utilities such as pprof makes the task of diagnosing runtimes issues very productive. go IDE integration with Theia is very good. Creating cli tooling with go is a breeze. Building cloud services that are readable and efficient also a strong point in favor of the language.

Can Telepresence be used in a Quality Assurance environment?

Telepresence needs a deployment, and it can only work in a single container.

But this must be done prior to launching the workspace — it’s not possible, for example, to fire up telepresence and mix in a debugging tool. Telepresence only replaces within a namespace.

The workspace configuration does allow developers to use a debug container. Basically, this uses a Docker image layer with the delve binary pre-installed, a trick often used by teams deploying into development, quality, or (oh no!) production environments for live troubleshooting.

Follow Up Questions

I just noticed in the image above a stage “Study README.me”… I wonder what that is?


Disclaimer: I (Jared Davis) am not an employee or direct investor in TypeFox, Docker, or any of the other companies developing the tools described in this summary.

Content by © Jared Davis 2019-2020

Powered by Hugo & Kiss.