A handy configuration snippet that I am using with the nginx ingress controller

One of the most common ways to implement Ingress on Kubernetes is the nginx ingress controller. The nginx ingress controller is configured via annotations that modify the default behavior of the controller. That way for example by using the configuration snipper you can add to the controller nginx directives that would go to a location block on a normal nginx.

In fact whenever I am spinning up an nginx ingress I now always add the following annotation:

nginx.ingress.kubernetes.io/configuration-snippet: #deny all;

Whenever I need for some emergency reason or whatever to block incoming traffic to the served site, I can do it immediately with kubectl edit ingress and simply uncommenting the hash, rather than googling that time for the specific annotation name.

PS: If you want to define a whitelist properly, it is best that you use nginx.ingress.kubernetes.io/whitelist-source-range.


Happy Sysadmin Day.

Well, it is that day of the year again. I remembered that I used to run a greek translation of the site once upon a time. And I just saw that someone else has picked up the torch. Good.

I did some Rancher2 work today. Somewhat fulfilling I can say. That magic when things you do fall into place and you keep your people happy.

We work on top of layers, on top of layer, on top of yet more layers of abstraction. The machine is so far away that we may not even need to know the details of it to make it work (“The line is a dot to you” like Joey said). Nor do our horror stories look like this. Yet while they are equally epic, they’re not just as legendary.

[Each year I keep saying to myself I won’t write a blog post, but always the next day I do, I write one and change the date to match the Day… ]

Sometimes you need JDK8 in your docker image …

… and depending the base image this may not be available, even from the ppa repositories. Moreover, you cannot download it directly from Oracle without a username / password anymore, so this may break automation, if you do not host your own repositories. Have no fear, sdkman can come to the rescue and fetch for you a supported JDK8 version. You need to pay some extra care with JAVA_HOME and PATH though:

FROM python:3.6-slim
RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y curl zip unzip
RUN curl -fsSL https://get.sdkman.io -o get-sdkman.sh
RUN sh ./get-sdkman.sh
RUN bash -c "source /root/.sdkman/bin/sdkman-init.sh && sdk install java 8.0.212-amzn"
ENV JAVA_HOME=/root/.sdkman/candidates/java/current
CMD bash

Since curl | sudo bash is a topic of contention, do your homework and decide whether you’re going to use it as displayed in the Dockerfile above, or employ some extra verification mechanism of your choice.

minikube instead of docker-machine

docker-machine is a cool project that allows you to work with different docker environments, especially when your machine or OS (like Windows 10 Home) does not allow for native docker. My primary use case for docker-machine was spinning up a VirtualBox host and using it to run the docker service.

It seems that docker-machine is not in so much active development any more and I decided to switch to another project for my particular use case: Windows 10 Home with VirtualBox. One solution is minikube:

minikube start
minikube docker-env | Invoke-Expression # Windows PowerShell
eval $(minikube docker-env) # bash shell
docker pull python:3
docker images

and you’re set to go.

Sometimes n2n is good enough

You are not always in a position to use one of the big five cloud providers (BTW, I think Watson’s prediction was kind of true; serverless is making sure of that).

So when you’re working in different than usual public cloud environments, sometimes you miss features that are a given, like a VPC. A VPC is a pretty cool abstraction that allows you to have an isolated network of machines (and sometimes services) within your cloud provider and allows for easier management of things like security groups, routing traffic between machines and the like.

So what do you do when you do not have a VPC available? You need some kind of overlay networking. When deploying Kubernetes for example, you need to deploy an overlay network (there are many solutions to choose from) and you let it deal with iptables and routing hell. But, you may need to temporarily scale services that are not container orchestrated for whatever the reason (I, for example abhor running databases with Kubernetes). Still you may need an autoscaling solution like EC2 does. IPSec would be a cool solution, but deploying it in my current workplace would be too complex. Something simpler was needed. And I found it here, despite the shortcomings reported: N2N from ntop.org.

N2N was in a development hiatus, but now is back on active development. It utilizes TUN/TAP and allows you to build a VPC over the interface that the client you run on your machine creates. It comprises of two components: a supernode, which is actually a directory server that informs the members of the (let’s call it) VPC of the actual IP addresses of the members, and a client program (called edge) that creates the interface on each VM and contacts the supernode to register with it and query for needed routing information. The supernode itself is not routing any packets. It was a single point of failure, but current versions of N2N/edge support two supernodes, so your network is in a better position.

In my case I needed to autoscale a certain service for a limited amount of time, but did not have any prior knowledge of the IP addresses of the VMs that were to be created. So I had them spun off using an image that was starting edge, registering to the supernode and then routing a certain kind of traffic through a node that was also registered in the same network and acting as a NAT server. Hence, I simplified some of the iptables hell that I was dealing with until we deploy a better solution.

N2N supports encrypted traffic, and requires the equivalent of a username / password combination (common to all machines that are members of the VPC, but not known to supernode apriori).

So where else might you want to use N2N? Maybe you need a common IP address space between two cloud providers? You may be in a cloud provider that allows for VPCs but does not make it easy to route traffic from a VPC in one region to a VPC in another region? Or in cases when solutions like DMVPN are expensive and your own BGP solution an overkill? Stuff like that.

So how do the machines acquire an address in that VPC? You have two choices (a) DHCP and if that is not working (b) a static address. In the second case with you need to implement a poor man’s DHCP by having the machine assign an IP address to itself with a low probability of collision. To this end, let’s assign a /16 to that VPC and have the following entry in /etc/rc.local (yes I am still a fun of rc.local for limited usage) like:

edge -l supernode:port -c community-string -k password-string -s -a static:172.31.$(shuf -i 1-251 -n 1).$(shuf -i 1-251 -n 1). 

The probability of a collision is (1/63001) so you get to decide at how many temporary instances you need to have a better hack at that. Plus you get to use for static machinery within the VPC (like a NAT gateway for example).

Not a perfect solution, but definitely an easy, fast one.


Figlet, a program to make large letters out of ordinary text was featured in Lobsters and it reminded me of cursive, a similar program that we used back when ASCII art was the only decoration one could add to their email signature. And since it seems that cursive is not carried by any Linux distribution, I set out to find the sources. Which I did thanks to FreeBSD.

I downloaded the source and tried to compile it, and it required xstr. Yet another old program! I think the last time one can see it, is OpenBSD-5.5 manual. I mean even the 5.6 changelog writes:

“mkstr was intended for the limited architecture of the PDP 11 family.” Time moves on, memory gets cheaper. There’s no need for mkstr or xstr.

Hacks like xstr and mkstr might be of interest to embedded, or otherwise deprived, systems people as cool hacks and peculiarities, but really no actual need to maintain them.

So I set out to make the cursive source compilable with xstr again (it compiles without it just fine, just type make lcursive instead). And the minimally changed source code, to eradicate compiler warnings, of a program that can happily sign your email since 1985 is now on Github:

  /  `
 /--   ____  o ____  ,
(___, / / (_/_(_) (_/___
           /       /
         -'       '

Shoutouts to my good friend Panagiotis C. who was the person who showed me cursive and figlet some 25 years ago or so.