terraform, route53 and lots of records

At work we try to manage as much as we can with terraform. This also includes Route53 for zones and records. In a certain situation we had about 14 zones and 1476 records managed in a single state file.

As it happened I needed a zone recreated (but not erased) and this affected about 409 records. Well deleting them with terraform apply took ages. To the point that the temporary STS token expired and botched the process.  So after a little facepalming, I decided to cleanup the zone from the AWS console and then issue a batch of terraform state rm to reconcile the state. Happily, after that, apply took its time (but reasonably) and all was well.

I am thinking that next time I am faced with such a situation, to lock the state file in Dynamo, copy it over from S3, manipulate it locally, unlock and run a plan to see how it all plays out. Or, wherever I can, use a state per zone instead of a state file encompassing a set of zones.

Advertisements

When Alexa does not connect to the net, it might be your fault sometimes

I moved to a new flat today, and after unpacking and general housekeeping, it was time to connect the Echo to the network.  Unfortunately, it refused to play ball.

So I reset it to factory defaults. No luck. The process was hanging when it tried to connect to the net. The progress bar stopped at some point after being halfway through. Factory defaults again. No luck. But the old mother of all evil came into mind:

Everything is a DNS problem.

Could it be so? Of course it could. I am running dnscrypt-proxy so my DNS server is always set to 127.0.0.1 and not whatever the DHCP proxy serves. So, let’s get the default from the network:

networksetup -setdnsservers Wi-Fi empty

I then pointed my browser to alexa.amazon.com (yes I am not using the app) and the configuration completed without a hassle! I switched back to using OpenDNS FamilyShield by:

networksetup -setdnsservers Wi-Fi 127.0.0.1

For anyone interested I brew install dnscrypt-proxy and my dnscrypt-proxy.conf has:

ResolverName cisco-familyshield

addressing

I’ve started reading John Day’s Patterns in Network Architecture and during the first pages it makes strong references to Saltzer’s 1982 paper. Why would I bring this up? Well, I just heard Surprisingly Awesome‘s episode on Postal codes where they deal with two countries (Lebanon and Mongolia) with almost non-existent addressing plans. Here is what an addressing plan should give you:

  • a name identifies what you want,
  • an address identifies where it is, and
  • a route identifies a way to get there

Day makes the case that we usually use that address of a network element in the same way that we use its name also which is an error, since by moving an element elsewhere in the network, we need to change its name also.  You on the other hand do not change your name when you change your home address. You used to change your phone number, but even that has become equally portable.

In places where no stable addressing system exists the courier is required to build a mental representation of the routes in their area of delivery, based on landmarks, trees, neon signs, whatever can help to make the delivery. In Mongolia this is solved differently: When something arrives at the post office, they call you back and you go and pick it up.

Enter the NAC. What is it exactly? It is an effort to map longitude and latitude to a more memorable representation using the base 30 number system using digits and capital letters. Borrowing from Wikipedia, the NAC for the centre of the city of Brussels is HBV6R RG77T. Compact, accurate, but not quite memorable.

what3words seems to be a service set to solve this since with their solution a unique combination of just 3 words identifies a 3m x 3m square anywhere on the planet. For example, roughly the same place as above is described as october.donor.outlined. I admit, this is much easier to type in a GPS (or tell Siri).

However, I am still surprised that nobody ever thought of using IPv6 for this (maybe somebody has? Please tell me). Given the abundance that the 128bits give us, we could have indexed every square meter on the surface of the planet and make it addressable. Oh, the directories we could have built on top of that. But I have no fear. It is quite probable that much of the inhabited First World’s surface will be pingable in the foreseeable future. The IoT will make sure of that.

 

Funny OpenDNS, VirtualBox and Debian weirdness

I was trying to install a virtual machine using the latest VirtualBox on a Windows 7 Host. The host was also running OpenDNS DNSCrypt 0.0.6 client. The guest operating system should be Debian/LXDE. Installation went fine until the installer tried to contact Debian mirrors to fetch missing packages.

It couldn’t find them. Like the common system administration mantra says:

Everything is a DNS problem.

So at the OpenDNS DNSCrypt client dashboard I (temporarily) disabled the DNS over TCP option and the installation continued smoothly. The same thing does not happen with OS X Mavericks as the host operating system. After the installation is finished, you can reenable DNS over TCP for DNSCrypt. The guest operating system’s resolver sees no issues with this.

I am posting this short note because it may bite others out there.

ISOC Perspectives on Domain Name System (DNS) Filtering

The Internet Society (ISOC) posted its views on DNS filtering. They are excellently summed up by the ISOC in a single phrase:

The real solution is international cooperation.

The reality though is that DNS filtering is here to stay. And it is here to stay because its initial deployment is far more easier than attacking the problem to its source via international cooperation.

So until DNS filtering (and supporting users mainly) starts costing Service Providers a lot, as in costing that much that it makes international cooperation cost less (even with the bureaucracy involved) it is a fact of everyday life that we have to get used to. Just imagine debugging not being able to access a single site, while at the same time all antivirus vendors run their own private, and allowed to be queried only by machines running their products (a “value added service”), resolvers.

Sad but true.

Please do not mix CNAME and MX RRs

From time to time I observe the following email setups, from web hosting providers mostly:

$ host -t mx example.com
example.com mail is handled by 5 mail.example.com.

$ host mail.example.com
mail.example.com is an alias for www.example.com.
www.example.com has address 192.0.2.2

In other words this is a single server that provides web and mail services, The devil is in the details though: mail.example.com is an alias for http://www.example.com. This is a mistake as when something is declared as a CNAME, it cannot have other resource records bound with it. I copy from DNS for Rocket Scientists:

CNAME RRs cannot have any other RRs with the same name, for example, a TXT – well that was true until DNSSEC came along and in this case RRSIG, NSEC and certain KEY RRs can now occupy the same name.

So the above setup is wrong. The correct setup would be the following:

$ host -t mx example.com
example.com mail is handled by 5 mail.example.com.

$ host mail.example.com
mail.example.com has address 192.0.2.2

$ host www.example.com
www.example.com is an alias for mail.example.com.
mail.example.com has address 192.0.2.2

That is if you want to use a CNAME at all. Personally I am using A RRs instead of CNAMEs whenever possible. But why cannot a CNAME carry any other information? I copy from RFC1034 (section 3.6.2):

A CNAME RR identifies its owner name as an alias, and specifies the corresponding canonical name in the RDATA section of the RR. If a CNAME RR is present at a node, no other data should be present; this ensures that the data for a canonical name and its aliases cannot be different. This rule also insures that a cached CNAME can be used without checking with an authoritative server for other RR types.

So please people, correct your defaults. Your clients will benefit from that.