Mark BetzExhausting conntrack table space crippled our k8s clusterOriginally published at https://www.markbetz.net/2023/12/12/exhausting-conntrack-table-space-crippled-our-k8s-cluster/. Reposted here with…Dec 21, 2023Dec 21, 2023
Mark BetzSo you want Windows to show 24-hour time?I spend a large part of every day shelled into cloud servers, viewing logs, checking alerts in slack channels, looking at pages on my…Sep 21, 20202Sep 21, 20202
Mark BetzThe day a chat message blew up prodI don’t often write “in the trenches” stories about production issues, although I enjoy reading them. One of my favorite sources for that…Aug 28, 20201Aug 28, 20201
Mark BetzThe cost of tailing logs in kubernetesLogging is one of those plumbing things that often gets attention only when it’s broken. That’s not necessarily a criticism. Nobody makes…Jul 10, 20201Jul 10, 20201
Mark BetzBehind the front lines of the pandemicI’m a software engineer and so I usually fill this space with software and systems engineering topics. It’s what I do and love, and I…Apr 17, 20201Apr 17, 20201
Mark BetzPulling shared docker tags is badLast night we migrated a key service to a new environment. Everything went smoothly and we concluded the maintenance window early…Mar 6, 20201Mar 6, 20201
Mark BetzAfter my disk crashedA couple of years ago I lost all of what I would have considered, up to that point, my intellectual life, not to mention a lot of…Sep 2, 20191Sep 2, 20191
Mark BetzWhy I’ve turned off private notesThis morning I disabled private notes on my stories, for a couple of reasons. Philosophically I just don’t like them. If we’re editing a…Aug 21, 2019Aug 21, 2019
Mark BetzinGoogle Cloud - CommunityUpgrading a large cluster on GKEAt Olark we’ve been running production workloads on kubernetes in GKE since early 2017. In the beginning our clusters were small and…May 26, 2019May 26, 2019
Mark BetzinGoogle Cloud - CommunityIngress load balancing issues on Google’s GKEUsually my posts here are about some thing I think I might have figured out and want to share. Today’s post is about a thing I’m pretty…Apr 12, 20196Apr 12, 20196