About
Hyperscale cloud orchestration platforms were once the purview for large companies with thousands of talented engineers (Google, Netflix, Facebook, Twitter). This led to a proliferation of non-standard bespoke tools built and developed for the same company. This is no longer the case any more. The merging of development and operations called "DevOps" are now driving development of standardised hyperscale orchestration tools that can be leveraged and are accessible by individual developers or small teams.
Orchestration encompasses the entire range of problems that DevOps practitioners tackle. This means there are many tools which do many different things, with many overlapping features.
The goal is to create a quality alphabetical list of orchestration platforms with a collection of articles, blog posts, slides, and videos about them. After reading the linked articles, you should have a good idea about: the pros and cons of each tool, a basic understanding of how the tool works, and what each tool is trying to achieve. Basically, you should have all the information you need to decide which tool will best fit your needs. Do note that some of these tools are still under development, and the information on how it works will change rapidly.
The whole site is open source and is updated quite frequently. If you find any mistakes or see something missing, feel free to post an issue or send a pull request. All contributions are welcome! Thanks.
Platforms
Atlas
Compatibilty: Linux, Docker, VM
Atlas by Hashicorp is a tool that facilitates a single workflow from development to production. Atlas unifies Hashicorp's other tools: Vagrant, Packer, Terraform and Consul into a single platform. Vagrant provides reproducible virtual development environments, Packer facilitates immutable golden image deploys, Terraform configures and deploys Packer images into production, and Consul enables service discovery.
Resources
CoreOS
Compatibility: Linux, Docker, Rocket
Core OS is a minimal Linux distro that provides infrastructure primitives focused on deploying and running Docker services in production. It's 2 core primitives are etcd and fleet. etcd is a distributed key-value store that facilitates service discovery and configuration. fleet abstracts distributed Docker deployments by accepting service configuration files that will be queued for execution into a CoreOS cluster.
Resources
Docker
Compatibility: Linux, Docker
Docker is the new standard for portable Linux services and applications. It relies on core Linux kernel container primitives, and abstracts them into a user friendly API. It allows you to create an isolated container of your service, which acts like a lightweight virtual machine that shares the same kernel as the host operating system. This container can be turned into an image, and shared with anybody running Linux. The Docker company also provides orchestration tools on top of Docker such as Machine, Compose and Swarm.
Resources
Flynn
Compatibilty: Linux, Docker
Flynn is a multi-host open source PAAS built around Docker. Flynn is built in 2 layers: Layer 0 is a low level scheduling and service discovery framework, Layer 1 is a set of higher level components that make it easier to deploy and maintain both stateful services and 12 factor applications.
Resources
HybridCluster
Compatibility: FreeBSD, Jails
HybridCluster is an cloud infrastructure stack focused on hosting large scale web applications with built in high availability, auto scaling and ZFS storage. It currently supports running unmodified LAMP applications. HybridCluster is built up of AwesomeProxy, Flex Containers, ZFS and Control & Management tools that facilitates heartbeats and replication across HybridCluster machines.
Resources
Hyper
Compatibility: Linux, Mac OS X
Hyper is a set of Linux kernel, init process and management tools, able to virtualise containers to improve their isolation and management in case of multi-tenant applications, eliminating the need of guest OS and all the components it brings. Hyper provides isolated environments (virtual machines) on which portable environments (containers) can be easily scheduled.
Resources
- Hyper, a Hypervisor-Agnostic Docker Engine
- Docker, Hyper and the end of Guest OS
- The CaaS Revolution
Kubernetes
Compatibility: Linux, Docker
Kubernetes is both a configuration and scheduling tool for Docker services. It provides you a way to configure Docker containers into clusters of interacting services, and then it will intelligently schedule this configuration into a cluster of machines, taking into account resource density, replication and service grouping. It can run on Google Container Engine, or be installed on existing Linux distros.
Resources
Matrix AI
Compatibility: Linux, Docker
Matrix AI is an cloud & IoT (Internet of Things) operating system that seeks to be self-healing, self-organising, and self-adaptable. It models infrastructure as a functionally reactive topology that can be explicitly controlled through an infrastructural type system provided by the Architect language.
Resources
Mesosphere
Compatibility: Linux, Docker
Mesosphere DCOS (Data Center Operating System) leverages Mesos an open source distributed cluster scheduler and a variety of data center services to organise all of your machines into single pool of shared resources.
Resources
Mirage OS
Compatibility: Xen, OCaml
Mirage OS is a library operating system that allows you to construct networked services as UniKernels that can run on the Xen hypervisor. A UniKernel operates similarly to a Docker container, except that it is more secure, and does not require a host OS, because it already has its own kernel.
Resources
- Unikernels: Library Operating Systems for the Cloud
- Are Cloud Operating Systems the Next Big Thing?
- Unikernels: Rise of the Virtual Library Operating System
NixOS
Compatibility: Linux
NixOS is a purely functional Linux distro. It provides a declarative pure functional way of specifying operating system state, which allows you to atomically specify changes to the operating system. The configuration language called Nix replaces configuration management tools such as Puppet, Chef, Ansible and CFEngine. Distributed management of NixOS clusters can be done through NixOps or Disnix.
Resources
Project Atomic
Compatibility: Linux, Docker
Project Atomic is a set of tools that is focused on running Docker services on Redhat distros (Fedora, CentOS, RHEL). It's advantages include atomic updates and secure containers using SELinux.
Resources
RancherOS
http://rancher.com/rancher-os/
Compatibilty: Linux, Docker
RancherOS is probably best compared to CoreOS, the main difference is that it uses Docker to run everything except the kernel.
Resources
- RancherOS: A tiny Linux distribution ideal for running Docker
- A new superminimal Linux in the vein of CoreOS, RancherOS uses Docker to run everything outside of the kernel itself
SmartOS
Compatibilty: Open Solaris, KVM, Zones
SmartOS is a OpenSolaris based hypervisor suitable for building high scalable cloud platforms. It combines ZFS, DTrace, Zones and KVM. Zones is the OpenSolaris analogue of Docker containers.
Resources
Snappy Ubuntu
https://developer.ubuntu.com/en/snappy/
Compatibility: Linux, Docker
Snappy Ubuntu Core is a new rendition of Ubuntu with transactional updates - a minimal server image with the same libraries as today’s Ubuntu, but applications are provided through a simpler mechanism.
Resources
Vmware-Photon
https://vmware.github.io/photon/
Compatibility: Linux, Docker
Photon is a technology preview of a minimal Linux container host. It is designed to have a small footprint and boot extremely quickly on VMware platforms. Photon™ is intended to invite collaboration around running containerized applications in a virtualized environment.
Resources