All Content

  • The Homelab Show Episode 81: Building Lab Templates and Images

    When building a new virtual or physical server, doing everything manually is a waste of time. With an image or template, you can have some of your recurring tasks already set up on the image – and then every instance you create from that template will have all of those tweaks already made. But how far should you go with this default setup? What are some of the caveats that you might run into? In this episode, Jay and Tom discuss creating server images/templates and the related shenanigans you may run into while doing so.

    YouTube player
  • Ubuntu 22.04 and Kubernetes recently Broke Compatibility with Each Other (and how to work around it)

    Here’s another blog post today, that I’m creating for the same reason as the previous one. It took me a bit longer than I’d like to admit to figure this out, and if anyone else out there is wondering why their automated Kubernetes builds on Ubuntu 22.04 started failing on them suddenly for no apparent reason. Specifically, your Kubernetes cluster builds started failing on December 9th. (You literally can’t make this stuff up). So, after troubleshooting for countless hours I finally figured it out. I mentioned it to Jeff Geerling (yes THAT Jeff Geerling) and he mentioned I should write a blog post, in case it may help someone else. I figured that his suggestion was logical 🖖, so here it is.

    What’s the problem I’m referring to? If you’re attempting to initialize a Kubernetes cluster on Ubuntu 22.04 and you see error messages that include output such as this:

    CRI v1 runtime API is not implemented for endpoint

    Or maybe even this:

    unknown service runtime.v1.RuntimeService

    Continue reading and I’ll let you know what the issue is, and how to fix it. I’ll also sneak in a quantum science reference and it’s going to be a good time.

    (more…)
  • Quick Fix: “CrashLoopBackoff” While Building a Kubernetes Cluster with Ubuntu 22.04 on the Raspberry Pi

    Normally, I like to make videos for Linux and Container-related things – it’s my favorite way to teach and inform. But some things don’t translate as well to videos, and this is one of them. Recently, I ran into an issue that I’ve been trying to solve for some time, where initializing a Kubernetes cluster on Ubuntu 22.04 seems to fail, specifically on the Raspberry Pi. Although I did ultimately find something while searching the web that led to my resolving this, search results were generally unhelpful, and neither was Stack Overflow. So what I wanted to do, was to create this article just so I could make sure that anyone else Googling for answers does find something.

    Here’s the issue I ran into. For some reason, initializing Kubernetes clusters on Ubuntu 22.04 fails on the Raspberry Pi. More specifically, Flannel doesn’t seem to launch, with it stuck on “CrashLoopBackoff” most of the time. And not only that, the coredns pods will tend to stay stuck on “ContainerCreating” for eternity. Another symptom is that you’ll find errors in /var/log/syslog complaining about /run/flannel/subnet.env not being found. Oh, and another symptom. RPC errors will be mentioned in the syslog as well, and also complains about not being unable to create the sandbox.

    Blogs and howto’s mention many “fixes” for this issue, but for me I didn’t have any luck there. Some of them will have you create the /run/flannel/subnet.env file manually – but you shouldn’t have to do that, it’s not your job. (That file should be created automatically). I’ve seen at least one blog post mention the order matters regarding when you apply Flannel (nonsense) and not to be outdone, the classic “remove the directory and reboot” trick that also never seemed to work for me.

    So, what’s the fix?

    Are you ready?

    Here you go…

    sudo apt install linux-modules-extra-raspi

    Yeah, that’s literally it. I’m not kidding. See for yourself. You’re all set. Profit. Enjoy. I know, right?!

    Thanks to this completely unrelated bug report that gave me the idea to install this package and ultimately led to my writing this article. It’s my sincere hope that this blog post finds its way into the Google searches of whoever out there needs this.