I make it no secret that one of the most impactful essays on me the last 20 years has been Bob Lucky‘s essay about the last electrical engineer. And I have seen other computer scientists taking it into account too.
Ever since cloud computing caught on, it comes more and more frequently to mind, because I see that the abundance of computing power makes people think that they can get away without really thinking about the problem. After all, computing has become cheap, right? The prototype you’ve just rigged up costs $10 / month to host. Well scale on the other hand does not come cheap and it will bite you.
Now that “cloud” has become a commodity word, “serverless” is coming to dominate. Because you really do not need to buy your infrastructure since, you know, you can have a data center as a tab in your browser for half the cost. And you really now do not need old-school sysadmins, since you make people all work together and deploy stuff n-times a day (DevOps). But hey, even that is complex, because you just want to write the damn stuff and not think on where it runs, or how the hell it is supposed to run on scale. You only want profit on scale.
So let’s go serverless. And in the process, let’s make the prophecy about the world needing 5 computers true. Only they are cloud / serverless providers (just count the big players) where for some engineers, paraphrasing a bit Bob Lucky, life will be:
Projecting the current trends, future applications will be serverless. No one will have the foggiest idea what is on that cloud. Somewhere in the basement of the serverless provider or its successor will be a huge computer file with the listing of that infrastructure. The last engineer will sit beside the file, handcuffed to the disk drive like a scene out of “Ben Hur.” That engineer will be extremely well paid, and his or her every demand will be immediately satisfied. That engineer will be the last keeper of the secret of the universe: E = IR.
Bootcamps and other vocational training institutions will turn out lots of serverless programmers, so prepare at least to be able to tick the Steinmetz chalk mark.
I got You are a writer by Jeff Goins on a day it was given for free years ago. I got to read it over the weekend. I thought then, and I still do no now, that the book offers advice to people who write code too. Maybe because sometimes I believe that code is like a poem. So I will try to rehash advice I found useful here, since this seems to be one of the acceptable self-help books that have come my way.
While anyone can connect with anyone and put stuff out there, how does one gets noticed? By helping people. By relieving their pain. I’ve advocated answering one question per day on ServerFault (or StackOverflow, or whatever clicks to you), although I do not always practice this. I used to, though, years ago when mailing lists and newsgroups were the thing.
You need to build a platform. This blog is a platform. I post stuff here. Others do it with more success, for example John Sonmez who uses a variety of platforms (blog, YouTube channel, podcast, book) to publish his work (and he gives 90% of his work for free he said on Ruby Rogues). You make the platform and you establish a brand. Without a brand you are forgettable.
And you know what helps you not being forgotten? Networking does. Meaningful relationships do. Relationships that are mutual, matter to both parties and give you the opportunity to make friends and find mentors.
But nothing of the above makes any sense unless you are prepared to answer the following questions:
- Am I serious?
- Am I committed?
- Am I prepared to be challenged?
Please take the time to sincerely answer these questions. And while you are at it, understand that you probably are not the best coder in the world yet. But you can become one. There is room at the top here.
Like one of the main characters in The Phoenix Project said, mastery comes through practice. And here Goins argues that the best kind of practice is done publicly. Interestingly GiHub seems to be our public place of practice.
While you’re at it, learn to estimate. Underpromise and overdeliver. Especially when you are part of a team. Like Goins writes, you have a gift. Someone is willing to work with you.
Go on! Practice.
I cannot even remember how many disks Windows 95 was, fourteen, fifteen? Something like that. And how many hours installing it to machines around the lab.
That was the day I said goodbye to FTP software’s PC/TCP and Trumpet Winsock.
(only your weekend)
After numerous installs since the Windows 3.11 era I should have known better and not tried the upgrade from Windows 7 to 10. But I do not have a spare machine for a clean install and therefore when the popup assured me that I could upgrade the laptop, I broke my rule:
You never change (upgrade or downgrade) the operating system that came with your laptop. If you do, at least make sure you have a spare machine to do actual work.
That is because laptops come with hardware that is not necessarily recognised by anything else other than drivers supplied by the manufacturer. So if you do change the OS, or even install a “clean” version of it (without all the bloatware that manufacturers package with it) be prepared for stuff not working. Yes, that fingerprint sensor, that 3G modem, the winmodem in the old days and before the linuxant drivers for example.
So anyway I felt brave and went along and tried it. It seemed to go fine. But at 76% the process stalled. And then it failed with DRIVER_POWER_STATE_FAILURE. Following the suggestion of a friend, I tried the online downloading tool that Microsoft gives to download and / or start the upgrade. It failed to even start. I run it again and built an ISO DVD to have around just in case. But still I was facing the same problem.
I gave it some thought after browsing the Net for a bit and thought that it could be the speed setting switch which I almost always have set to “stamina” and not “speed”. Again at 76% the procedure stalled like forever. I shut down the machine and when I powered it up again the procedure continued. And finished. Yes!
So I browsed around to get a feeling of the system. And then Skype was not working. Fair I thought since my login account was not linked to a Hotmail.com account and maybe after the upgrade this is a mandatory requirement. In the mean time OneDrive installed and update. So let’s reboot and see what happens. And it never rebooted. It stuck on the Windows 10 logo with the balls doing their circles.
So I shut down the machine again. I power it on and it is OK. I backup my data, shut down the VAIO and press the magic “assist” button. Windows 7 with all the bloat that Sony has chosen gets installed on the machine again. About 1h later I start uninstalling stuff. Adobe, McAfee, Office (Trial), SQL Server, Norton go by. SP1 gets installed. 214 updates after SP1 too.
Like the upgrade procedure said: This won’t take long
[ This draft has been gathering dust for a few years, so I am posting it because what the heck, I’ve written it ]
I found out about dotplots while reading “Algorithms on Strings” (thanks to Bill Gasarch) an impressive book by the way, that easily makes it to the top-5 CS books I’ve read. It may well be one of the most complete works on the subject fulfilling simultaneously the needs of those who want to read the theory, see the proofs and/or write code (for it contains well understood pseudocode). But I digress.
Dotplots are tables that can be used to compare sequences (and therefore strings) and can reveal hidden patterns that may not be observable with other methods. They are in use for decades and known to people who work on computational biology (see for example “A high speed, high capacity homology matrix: zooming through SV40 and polyoma“)
Definition: Given two strings x and y with lengths m and n respectively we define a table Dot (the dotplot) of size m×n such that Dot[i][j] is true if x[i] equals y[j] and false otherwise.
Using the book example sequences ACGT and ATGCTACG we get the dotplot:
ATGCTACG A*....*.. C...*..*. G..*....* T.*..*...
To interpret a dotplot we observe that diagonals express similarities and antidiagonals reveal the existence of a substring in reverse order. In a similar fashion horizontal lines represent insertions and vertical lines deletions that lead from one (sub)string to the other. One can therefore think of a dotplot as a kind of visual grep.
Dotplots find usage outside computational biology. They can be used for text analysis and even translated plagiarism! However, the screen real estate is not big enough to use for comparisons that involve for example the source code of the X Window System and techniques mentioned in “Dotplot: a Program for Exploring Self-Similarity in Millions of Lines of Text and Code” and “Dotplot Patterns: A Literal Look at Pattern Languages” can be used. Both papers contain really interesting applications of dotplots and explanations of the visual patterns that occur. See for example:
As shown by the ASCII example above, it is very easy to create proof-of-concept dotplots. But character based dotplots are limited by screen space. In fact I set out to create my own version of a dotplot program using gnuplot for the graphical stuff. Building a basic dotplotter is really easy once you read the techniques presented in Helfman’s papers but the result might not be as “good looking” as the ones that Helfman has done. To build cool dotplots, it may be needed to customize your software appropriately. For example, to dotplot source code, your program might need to have parsing capabilities for the language in question (for example two for loops might be similar to another when you look at it, but whitespace might make them look different to a simplistic dotplot builder).
[ There is not going to be a part2. Originally I wanted to experiment with dotplots and spam, but I am not a postmaster anymore, so I’ll leave it to the next interested postmaster. ]
A bit more of a week has passed, so let’s put my notes in order before memory and other notes hide them away.
From the very beginning this seemed it was going to be a good conference. And why not since by entering the venue I bumped into a close friend from the 1990s whom I had not seen for a few years due to unusual circumstances.
The day began with Matz giving a keynote. And a keynote it was. It contained memories from his first Sharp computer to switching to Pascal and then to Lisp and explaining why Ruby was influenced by it. Sapir-Worf hypothesis was also included since Matz as a person who loves programming languages tried to make us all design our own domain specific language. He made the (to the point) remark that every program that we create is in fact its own programming language with which we communicate with the system and direct it to do anything we want within its abilities. Matz urged us to become language designers (remember that every interface to a program is a language) by starting with a limited domain, locate patterns, solve problems in this domain and then expand. Perfection is the enemy of the designer he said, so he also found room to squeeze the Pareto principle in his keynote, just to make sure that one is not lost in details instead of delivering a result.
Picking from the keynote and the Lisp legacy Pat Shaughnessy continued with functional aspects in Ruby and his adventures of learning Haskell in order to make a comparison and learn something new in the process. Best lines for the talk were that “memoization is just another name for caching” and that in Ruby lazy evaluation is in fact lazy enumeration and one has to keep this in mind.
Konstantin Tennhard continued with his work with Ruby and NLP. He covered a lot of aspects of NLP and how it is done and work he did with the help of JRuby and OpenNLP which I had not head of until then. He also described a gem he has built and gave examples of its usage. I guess NLP people will find this work interesting.
One of the best presentations I’ve ever watched that include Mathematics and Programming was done by Xavier Noria. How common is it that you get to hear about a proof that the square root of 2 is irrational, about the continuum hypothesis, about set theory and stuff like cardinality of a set in a Ruby talk? By his sharp remark that “programmers are so used to integers and floats that they forget the use of rationals” he went on and gave a real life example demonstrating this as an application for a betting site. Arbitrary precision and no cent lost. One pointer though. Next time that you have ZF in your slides at such a conference, translate it.
Arguably the best presentation of Euruko 2013 for me was the one by Grzegorz Witek and Simon Kroeger. They came with a real life example that Unicorn could not handle due to high load, need for many deployments per day and requirements for the fastest user experience, showed alternative plans (Puma, Reainbow, ZBattery, Thin) and why these were not follwoed thtough and how they reached to picking the commercial version of Passenger Phusion to do the job and how well it did it. A “We did it and this is how” talk where you can only admire someone with a working production setup that brings money home.
Chris Kelly walked us through parts of the source code (and macros needed to understand how Ruby works) for MRI and how garbage collection works in general and especially in Ruby 2.0. Frequent mentions to “Ruby under a microscope” were made.
Hey did you know that an ISO standard for Ruby exists? It is ISO/IEC 30170:2012 and it is a hybrid between 1.8.x. and 1.9.x. This I learned from Koichi Sasada who proved live that when you have something to say, not knowing English well is not a barrier because the audience is willing to hear.
Rubinius was the subject of Dirkjan Bussink. He explained the design behind it and that it is a good choice when number crunching is in order. Rubinius people aim for concurrency, multicore and optimization for the common case so that the programmer does not have to deal with much outside the environment of his program. He closed his presentation with a few programming mantras that are always worth mentioning like “Be nice; write type stable code”, “Write small and simple methods” and “Always benchmark your code”.
Benjamin Smith talked about rails engines and how locating patterns in the codebase and implementing them as engines helped refactoring the code for a client with a small team while the main team was working on the product. If you are into locating patterns (and you should be) when the video is put online you must watch this. As he put it “In the long term engines work out” as long as you avoid circular dependencies between engines. You can read a lot about this stuff at the rails Archives.
Many times I’ve said that Redis is the kind of software I wish I had written. So when Javier Ramirez talked about it I could not miss it. It was a general introduction for Redis to the audience and I think everyone left the room with a clear understanding of what it can and cannot do (including pubsub).
Unfortunately I missed Steve Klabnik‘s talk on Functional Reactive Ruby but from the sound of clapping it must have been a hit. I did not watch a few more presentations and the lightining talks due to other engagements and am waiting for the videos to upload so as to complete the experience.
All in all the experience of the conference was excellent, the Jazz concert great, the volunteers did a great job and in the chosen Eurovision style of presenting the conference: 12 points guys.
PS: For those interested, EuRuKo 2014 will be held in Kiev.
A twitter conversation I followed minutes ago sparked this memory:
“the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.”
I think the first time I used the above argument from Dijkstra was when huku and @_argp made their excellent presentation about memory allocators back in OWASP AppSecEU 2012.