Building a platform on Kubernetes in a Windows based software development shop

On the eve of KubeCon North America 2020, I am reflecting on my journey to become a platform product owner. I am grateful and amazed my team created a working alpha version of this internal developer platform in only three months. While I thought adapting to a Microsoft Software Development shop would be a challenge it ended up offering the greatest opportunity to work with talented people to create a product that will supercharge development at a reasonable price. You might wonder why a Windows shop would ever consider migrating to Kubernetes but the cross-platform nature of .NET core makes it irresistible.

How does the platform work?

Imagine a self-service portal where you can create a new API at the click of a button. Code generation kicks off and creates your new project with the latest and greatest updates. You receive a link to your git repository and can clone it down or edit your project straight away in the browser. As you make a commit the already made CICD pipeline starts building and immediately deploys your project to the development environment. You can navigate to the live running version immediately.

For our alpha product GitLab allowed us to rapidly iterate on the idea. We created a prototype in 3 days to demonstrate what is possible. Over the course of three months, we developed and released the alpha version where we learned a lot more about how to make the whole effort simple to automate along with setting up Kubernetes in our non-production environments.

The power and versatility of GitLab granted us a lot of speed to iterate on our idea. Over time we refined a .gitlab-ci.yml to do the majority of work we need without developer interaction. While we do plan on moving some things out of CICD for now this is what we have.

  • Run .NET project unit tests
  • Creates namespaces per project in all environments
  • Automate versioning of the application using go sem rel
  • Use buildah to create the container and push it to the GitLab container registry
  • Push new helm chart to our Helm chart repository
  • Helm install/upgrade to Kubernetes cluster environments
  • Various Clean up tasks as required

What this offers is the ability for a software developer to not think at all about how Kubernetes works. We ship with every project appsettings.json that is automatically applied per environment. Along with a hook for the application to retrieve its Hashicorp Vault secrets automatically. We also ship a Values.yml file for developers that are adventurous and want to toggle Kubernetes specific settings via this Helm specific file.

What are the risks of the rest of this journey?

We still have many significant challenges ahead of us and I have no idea how we are going to solve many of them. Visual Studio Code has taken over the world but Visual Studio IDE still is used by the majority of the developers I work with. VS Code does not outclass C# development, yet. Debugging C# .NET code locally and remotely on Kubernetes is still a significant hurdle. We’ve played with Bridge to Kubernetes and Okteto and it is unclear what technology will be a permanent and lasting solution for .NET container debugging. Until role-based access control is solved I am hesitant to ship a kubeconfig and kubectl to our developers. I am confident we will find a solution but understanding how our automation works illustrate how much we abstract from software teams. So far as of this writing Okteto is providing the best .NET rapid development and container debugging experience.

What this enables

This new abstraction enables a centralized location to separate a lot of the tedium that developers did in maintenance. For growing businesses, there is a trade-off between the creation of new software and the maintenance of old software. Negotiating time across 16+ teams is possible but incredibly wasteful and time-consuming. In some cases it must be done but if you can create a platform that can centralize that work you have an incredible opportunity to make the most of your business.