Why I’m not using Linux on my servers anymore

“No Linux anymore? Isn’t Linux on my servers the best choice still?”. That’s what I heard from my friends and colleagues so many times. There seems to be a big preoccupation out the in field, favoring Linux. “What else are you using? Windows?” Don’t be silly.

No more Linux on my servers

My journey for finding the “right” server OS began back in 2011, when one of my Linux servers was hacked. That made me try out FreeBSD because the BSDs have a reputation for greater security. So I went through my FreeBSD phase and discovered many benefits:

1. Stable, production-grade ZFS on root.

2. Jails.

3. Better licensing.

ZFS

At the time, I was not searching for a new files system. FreeBSD was just offering me ZFS. Since then, I wonder how I will ever be able to live without it. There’s many good articles out there that lay out ZFS’s advantages. Well, what strikes me most are basically three things:

1. Reliability

2. Completeness of features

3. Ease of use

Reliability is the killer feature. You might also call it fault tolerance. ZFS is designed from the ground up to prevent a corrupted file system or otherwise inconsistent data on the disk. Even when the there’s a power fail. Raid-Like features, called RAID-Z, are an integral part of ZFS. After having had some ugly experiences with (hardware) RAID, I absolutely love the reliability of RAID-Z. After having made my first, pleasurable experiences with ZFS and RAID-Z under FreeBSD, I asked myself: Is reliability a feature that I desire in a server? Well, unless we have a quick-and-dirty, scratch-type, experimental server, the answer is a big and enthusiastic YES. Things are happening on a server, people are using it creating important data, they are used for storage purposes etc. We want to trust our servers to keep all that in a safe place. ZFS gives us exactly that. Be aware that in order to get the real deal regarding reliability, better use ECC-RAM rather than non-ECC RAM. Yes, ZFS is designed in a way that it never looses or erroneously changes data, but if you have a bit error in RAM (and these do happen), and that bit error then gets written to disk, all of the ZFS feature won’t help to prevent it. A chain is only as strong as its weakest link. Be wise and use ECC RAM.

Jails, zones, containers – or VMs?

FreeBSD has a concept called Jails since 2004. It’s a fantastic means of virtualization. The confusing part is terminology. Why the heck is a virtual server called a jail? Well, a virtual server (in the case of jails, which is also called operating system level virtualization) is nothing more than a “compartment” of the overall operating system that is completely shut off against the rest of the OS. A little like the good old “chroot”, which stands for ‘change root’, i.e. run commands in an environment where a specific subdirectory is made by the ‘root’ directory of the commands running inside the chroot environment. While the compartments in chroots have some holes likes Swiss cheese, jails fixed that problem. Their isolation is safe. That’s why they’re called jails: You can’t break out. Well, no rule without exception, with real jails as well as with FreeBSD jails.

But anyway, you get the idea. So how do jails differ from “real” virtual machines, a.k.a. VMs? Well, VMs usually make use of CPU-level virtualization. Each VM gets their very own operating system, including the kernel. This adds a significant overhead, in CPU throughout, memory usage, I/O throughput and more. Jails, in contrast basically offer the full bare-metal performance in each and every jail. Jails save resources, including electricity and heat. Jails are green. Jails are easy to use as well. Just a simple command (using tools like ezjail), and I have my own, virtualized server. No need to pay extra money for extra hardware, no need to pay Amazon or digitalocean any extra money. And I can do in my jails whatever I want. And I can create as many as I want.

For many years, there was no comparable equivalent of jails in Linux. Yes, there was OpenVZ, but it needed a patched Linux kernel. Things changed a bit with LXC (Linux containers, containers being Linux lingo for jails), but I never found them as easy and compelling as jails. And then came docker, popularizing containers. But docker isn’t made for “virtual servers”, it’s just for “packaging” software. I find docker fundamentally flawed in some aspects, but that’s a different story, and I’ll write about it in a different post. Unfortunately, the “emulation” of a full operating that jails provide is not complete. There are some rare cases of a piece of software that you can install in the main OS, but not inside a jails. One example is the namespace for IPC, to mention a specific case for the experts. In search for a better solution, I found SmartOS. It is a descendant of Solaris, Sun’s flavor of Unix. While Oracle took Solaris closed-source after they acquired Sun, SmartOS is based on the open source version of Solaris, and SmartOS is still open source today. It’s maintained by Joyent, and Joyent makes a case of offering hosting services based on SmartOS. Long story short, SmartOS is the best of all worlds:

1. Zones, the SmartOS (or Solaris) variant of jails, is indeed complete and offers full and safe OS level virtualization at bare-metal performance. Fast, green VMs.

2. ZFS is deeply engrained in the genes of SmartOS. Through its Solaris pedigree, SmartOS offers enterprise-grade robustness, including ZFS and the fault tolerance it brings with it. On the other hand, as of the day of this writing, ZFSonLinux is still not production ready.

3. SMF, SmartOS’ Service Management Facility, is all the “service” feature in debain/Ubuntu ever wanted to be, and more.

4. Since 2014, we have a fascinating thing in SmartOS. It’s called Linux branded zones. Normally, each zone runs SmartOS, not Linux. Since LX branded zones are around, a SmartOS zone offers a complete Linux environment, without the need for CPU-level virtualization using KVM, Xen or others. The result: We can have SmartOS zones (which are sufficient most of the time) AND Linux zones (which are needed for some software that just runs in a Linux environment). Both run fast, with no overhead. Green VMs in two flavors. And by the way, we can install Ubuntu, CentOS, Debian and others in LX branded zones.

5. KVM is still available in SmartOS. So in rare cases where the LX branded zone is not enough and we need a “real” Linux with its own kernel, we can create a KVM-based VM. We can run docker in these VMs if we want to. Also, we can run *BSD or even Windows under a SmartOS KVM-based VM. Why would I ever base my servers on Linux again?

Licensing

GNU/Linux, in any of its more recent flavors, is broadly based on GPL V3. The Free Software Foundation, and its founder Richard Stallman, consider that “free software“. Well, there is lots of controversy about that. Personally, I am strongly opinionated on the ”other” side, not the one of the FSF. I believe the GPL, and more than anything else the GPL V3, is much more about forcing something on me rather than giving me freedom. Richard, you want me to do everything as you dictate, rather than giving me freedom to do what I want. In that sense, my view on the GPL V3 is closer to that of Linus Thorvalds than to that of Richard Stallman. I tend to stay away from the GPL, especially GPL V3, wherever I can. In the case of SmartOS, I can, and I even have a product that is lightyears ahead of any flavor of GNU/Linux.

Questions? Ask in the Antsle forum.