Otto
Sep 28 2015 Mitchell Hashimoto otto

Today we announce Otto — the successor to Vagrant. Otto is the single solution to develop and deploy any application, with first class support for microservices. It is the most powerful tool we've built yet.

Otto automatically builds development environments without any configuration; it can detect your project type and has built-in knowledge of industry-standard tools to setup a development environment that is ready to go. When you’re ready to deploy, otto builds and manages an infrastructure, sets up servers, builds, and deploys the application.

With the growing trend of microservices, Otto knows how to install and configure service dependencies for development and deployment. It automatically exposes these dependencies via DNS for your application to consume.

Vagrant brought simplicity and power to development, and we believe we've brought that same elegant user experience to both development and deployment with Otto.

Read on to learn more.

The industry has needed this kind of abstraction for a long time, and it’s finally here with Otto.

Ben McRae, Head of Technical Operations at Conde Nast Commerce

The Successor to Vagrant

Vagrant does a lot right. Vagrant downloads have continued to accelerate and grow to this day. There are more Vagrant downloads per day today than ever before. But we think we can do even better.

We've learned a lot about developers and development environments over the past six years:

  • Development environments are similar: All Ruby development environments look alike, all PHP development environments look alike, etc. There isn't much deviation between two development environments for the same language or framework. The Vagrantfile requires users to configure these environments for every project, rather than having it abstracted away.

  • Developers want to deploy: After developing an application, the next step is to deploy! "vagrant up" to production has been a feature request for years. Production environments look very different from development: load balancers, firewalls, routing concerns, new configurations, etc. Unfortunately, the Vagrantfile isn’t a good format to describe this information.

  • Microservices are difficult: More and more applications are being written in a service oriented way. Modeling these services for development and deploy is difficult. The Vagrantfile puts the burden of installing/configuring every service onto the end user.

Otto solves all of these issues.

Vagrant is a mature, healthy project that is continuing to grow every day. We are committed to supporting Vagrant for the foreseeable future and will continue to release new versions. Otto is our vision for the next generation and will be developed alongside Vagrant.

Appfile

We created a new configuration format for Otto that is expressive enough to solve all the mentioned shortcomings of Vagrant: the Appfile.

As the name suggests, the Appfile is a higher-level description focusing on the application. A complete example is shown below:

application {
  name = “my-app”
  type = “ruby”

  dependency {
     source = “github.com/hashicorp/otto/examples/postgresql”
  }
}

customization “ruby” {
  ruby_version = “2.1”
}

Notice that the Appfile makes no mention of OS, memory, disk space, etc. Otto has built-in knowledge of best practices and picks smart defaults for you. In addition to this, you can see how dependencies are first-class features. Otto will use this information to automatically install and configure dependencies in your development and deployment environments.

The Appfile is the simplest configuration format we’ve ever made. Actually, everything in the Appfile (including the existence of the Appfile itself) is completely optional. You can run Otto in any directory and it will attempt to discover the application type automatically.

On top of this simplicity, customization blocks are a way to customize Otto's behavior. These are also fairly high level, but let you dig deeper down. This is where things such as manually tuning memory would go.

Centralization of Knowledge

The simplicity and purposeful lack of detail in the Appfile embodies one of the core ideas behind Otto: the centralization of knowledge.

Otto is smart. When you tell Otto the type of application you're developing, Otto knows how to develop and deploy that application using industry best practices. Otto knows how to configure and manage systems such as Terraform, Consul, and others.

Over time, Otto will get smarter, and adopt new best practices to improve your environments. For example, Otto will soon be able to configure and bootstrap Vault to automatically store and provide secrets to applications securely.

Imagine this scenario: a critical security update is released for your OS. The average developer or operator doesn't always keep up with these, but someone who does will contribute a change up to Otto to handle it. When Otto is updated, it'll automatically perform a rolling upgrade of this OS on your deployed servers, and secure your infrastructure using the latest best practices.

This scenario isn't fiction, it is what Otto was designed to do.

Development

To create a development environment, you run otto dev.

This will build a virtualized local development environment tailored to your application. For example, using the Appfile above, Otto will automatically install Ruby, Bundler, and configure various OS settings for Ruby development (directories and so on). Otto outputs application-specific instructions after creating the development environment to guide you along:

$ otto dev
...

A development environment has been created for writing a generic Ruby-based
application. For this development environment, Ruby is pre-installed. To
work on your project, edit files locally on your own machine. The file changes
will be synced to the development environment.

When you’re ready to build your project, run 'otto dev ssh' to enter
the development environment. You’ll be placed directly into the working
directory where you can run ‘bundle install’ and ‘ruby’ as you normally would.

With little to no configuration, Otto knows how to build development environments for many popular languages and frameworks. You can use customizations in the Appfile to change the environments slightly as needed, but the goal is for Otto to build a best practices development environment for every common application type.

Once the development is up and running, commands exist such as otto dev ssh to SSH into the environment, otto dev address to get a static address for accessing the environment from your host, and so on. This should be familiar for any Vagrant users.

Deployment

Otto deploys applications using industry best practices in just a few simple commands. Otto manages your infrastructure, builds your application, and then deploys the application.

The Appfile is cloud-agnostic: Otto is built to support deploying any application to any cloud environment with no special configuration. Otto 0.1 ships with AWS support only, but we’ll be adding new cloud targets rapidly.

In just a few commands: otto infra, otto build, and otto deploy, Otto launches a complete infrastructure, builds your application, and deploys it.

Infrastructure can be shared across multiple Appfiles. This enables multiple applications to be deployed to the same infrastructure. You only need to run otto infra once to build infrastructure.

Microservices

Developing and deploying microservices has historically been a challenging, tedious process. Otto makes it a simple, enjoyable experience by supporting microservices as a first class feature.

Service dependencies are defined in the Appfile. Below, we show an Appfile from HashiCorp internally that depends on two internal services that we have: Binstore and Storagelocker.

application {
  ...

  dependency {
    source = "github.com/hashicorp/binstore"
  }

  dependency {
    source = "github.com/hashicorp/storagelocker"
  }
}

Dependencies are declared using the dependency block and point to the dependency with a URL. This URL can point to a file path, Git, Mercurial, HTTP, and more.

This is all you need to tell Otto about your dependencies. Otto fetches and loads the Appfiles from these dependencies to know how to install and configure them. Otto will also automatically fetch dependencies of dependencies; you don’t need to specify these manually.

In development, Otto configures and starts these dependencies all within the same virtual machine. They are then available via DNS as they normally would be in production.

When deployed, Otto ensures that only one of each service is deployed, and configures your application to be able to find and communicate with these services over DNS. Future versions of Otto will allow for multiple instances of the same service.

HashiCorp Built

At HashiCorp, we build solutions to DevOps problems that are a joy to use. We approach difficult problems and solve them elegantly for the user.

Otto is the eighth open source tool we’ve built. We’ve also built Vagrant, Packer, Serf, Consul, Terraform, Vault, and Nomad. Otto will use and build on top of these tools to develop and deploy applications in a secure, scalable, and modern way.

Otto will integrate with our commercial tool Atlas to help developers and operators collaborate, monitor, and maintain applications and infrastructure across an organization. Otto on its own can't prevent simultaneous deploys from different team members, a deployment history, etc. Atlas will do this.

We're proud of Otto and are excited for its future. As a disclaimer: Otto is very young and while we tout it as a successor to Vagrant, it’ll take time to reach the maturity level that Vagrant is already at. We don’t recommend dropping Vagrant today, but look to Otto for the future..

Learn More

To learn more about Otto, please visit the Otto website. The following pages in particular are good next steps:

  • Intro - The intro section explains in more detail what Otto is, how it works, what use cases it has, and includes a brief getting started guide so you can start experimenting with Otto and learn about all of its features.

  • Comparison to other software - If you'd like to know how Otto is different from other options out there, take a look at this page and we go into detail on the differences.

  • GitHub - The source code for Otto is hosted on GitHub if you want to dive right in. We recommend reading the documentation first, since an understanding of how Otto works will help greatly in understanding the implementation.

More posts