Infrastructure 2017: Configuring CoreOS and Kubernetes

With the new infrastructure, I wanted the entire setup to be reproducible from a set of configuration files and scripts. This would mean that I can restore a fresh state anytime in the future, and provide a way to track changes in configuration. This also means that I can quickly spawn another Kubernetes setup to test new features safely.

Before I created installation automation scripts, I spent a while learning about how Kubernetes works by manually running the generic multi-node scripts Kubernetes provides, and failing repeatedly. A while back Sudharshan gave me one of his old OEM desktops, and it became really useful for testing Kubernetes installs. CoreOS is best installed using a Ignition configuration file. It's a JSON file with a specific format that CoreOS would read on the first boot and install the appropriate files or configurations specified. Usually, it is preferred to write these files in YML and then transpile them to JSON with a special tool. A great thing about CoreOS is it's ISO images boot from RAM (and the ISO image itself can read Ignition configuration), and they include a tool that would download the image directly from the web and write it to a disk together with the Ignition configuration.

I spent time playing with the CoreOS configuration options. Testing it, however, is a slow and painful process, but I learnt a lot from it. My scripts are on GitHub but are mostly not ready.

It's been months since I last experimented with CoreOS since school responsibilities took over. A lot has changed, including the deprecation of the coreos-kubernetes repository in favour of Tectonic. In the follow-up, I'll describe the final setup I will be using on my production cluster.