Building a testable Go web app

This was originally a talk given by Quinn Slack at Pivotal Labs in their weekly series. Many thanks to Pivotal for inviting us! Check out the slides here (source).

Nearly every programmer agrees that testing is important. But testing can be difficult in a variety of ways that discourage people from writing them. They can be slow to run, they can involve a lot of repetitive boilerplate, or they might test too much at once, making it difficult to identify the root cause of a test failure.

In this post, we'll talk about how we've designed the unit tests for Sourcegraph to be easy to write, easy to maintain, quick to run, and usable by others. We hope that some of the patterns we mention here will be useful to others writing Go web apps and also welcome suggestions for improving the way we test. Before get into testing, though, here's a quick overview of our architecture.

Architecture

Like many web apps, our site has three layers:

• the web frontend, which serves HTML;
• the HTTP API, which returns JSON; and
• the data store, which runs SQL queries against our database and returns the results as Go structs or slices.

When a user requests a page on Sourcegraph, the frontend receives the HTTP request for the page and issues a set of HTTP requests to the API server. The API server then queries the data store. The data store returns the data to the API server, which then marshals it as JSON that's returned to the frontend web server. The frontend then uses the Go html/template package to display and format this data as HTML.

Here's what it looks like: (For more details about our architecture, check out the recap of our Google I/O talk about building a large-scale code search engine in Go.)

Tests v0

When we first started building Sourcegraph, we wrote our tests in a manner that was easiest to get up and running. Each test would populate the database with fixtures and then issues HTTP GET requests to the API endpoints being tested. The test would parse the HTTP response body and check this against expected data. A typical v0 test looked something like this:

func TestListRepositories(t *testing.T) {
  tests := []struct { url string; insert []interface{}; want []*Repo }{
    {"/repos", []*Repo{{Name: "foo"}}, []*Repo{{Name: "foo"}}},
    {"/repos?lang=Go", []*Repo{{Lang: "Python"}}, nil},
    {"/repos?lang=Go", []*Repo{{Lang: "Go"}}, []*Repo{{Lang: "Go"}}},
  }
  db.Connect()
  s := http.NewServeMux()
  s.Handle("/", router)
  for _, test := range tests {
    func() {
      req, _ := http.NewRequest("GET", test.url, nil)
      tx, _ := db.DB.DbMap.Begin()
      defer tx.Rollback()
      tx.Insert(test.data...)
      rw := httptest.NewRecorder()
      rw.Body = new(bytes.Buffer)
      s.ServeHTTP(rw, req)
      var got []*Repo
      json.NewDecoder(rw.Body).Decode(&got)
      if !reflect.DeepEqual(got, want) {
        t.Errorf("%s: got %v, want %v", test.url, got, test.want)
      }
    }()
  }
}

Writing tests this way was easy and simple at first, but became the source of a lot of pain as our app evolved. Over time, we added more features. More features led to more tests, and the tests took longer to run, slowing down our dev cycle. More features also required changing and adding new URL routes (there are now about 75 of them), many of which are fairly complex. Each layer of Sourcegraph also became more complex internally, so we wanted to test them in isolation from the other layers.

Here are some of the testing-related problems we ran into:

1. The tests were slow, because they required interacting with the actual database—inserting the test fixtures, issuing queries, and rolling back the transaction for each test. Each test ran on the order of 100s of milliseconds, and this added up over time as we added more tests.
2. The tests were hard to refactor. The tests specified HTTP routes and query parameters as hardcoded strings. This meant that if we wanted to make a change to a URL path or set of query parameters, we would have to manually update the URLs specified in the tests, too. This pain grew with the complexity and quantity of our URL routes.
3. There was a lot of messy and fragile boilerplate. Setting up each test required ensuring the database was in the correct state and had the right data. This code was very repetitive in many cases, but different enough to introduce bugs in the setup code. We found ourselves spending a lot of time debugging our tests instead of our actual app code.
4. Test failures were hard to diagnose. As our app became more complex, it became hard to diagnose the root cause of a test failure, because each test hit all three layers of our app. Our tests were more like integration tests than unit tests.

Finally, another need that popped up was that we wanted to make an API client that we could release publicly. And we wanted that API to be easily mockable so that consumers of our API could write well-tested code, too.

High-level test goals

As our app evolved, we realized we needed tests that met these high level requirements:

• Targeted: We needed to test each layer of our app separately.
• Comprehensive: All three layers of our app should be tested.
• Fast: Tests should run very quickly. This implied eliminating interaction with the DB.
• DRY: Even though each layer in our app is different, they share many common data structures. The tests should take advantage of this to eliminate repetitive boilerplate.
• Mockable: The patterns we use for testing internally should also be available to external consumers of our API. People building on top of our API should be able to easily write good tests for their own projects. After all, our web frontend is not special—it's just another API consumer.

How we rebuilt our tests

Writing good, maintainable tests goes hand-in-hand with writing good, maintainable application code. Re-structuring our application code enabled us to vastly improve our test code. Here are the steps we took to improve our tests.

1. Build a Go HTTP API client

The first step to simplifying our tests was writing a high-quality client for our API in Go. Previously, our site had been an AngularJS app, but because we were primarily serving static content, we decided to move the frontend HTML generation to the server. After doing this, it made sense to write an API client in Go that our new frontend could use to communicate with the API server. The client, go-sourcegraph, is open-source and its design was heavily influenced by the awesome go-github library. The code for the client (specifically, the endpoint for fetching repository data) looks something like this:

func NewClient() *Client {
  c := &Client{BaseURL:DefaultBaseURL}
  c.Repositories = &repoService{c}
  return c
}

type repoService struct{ c *Client }

func (c *repoService) Get(name string) (*Repo, error) {
    resp, err := http.Get(fmt.Sprintf("%s/api/repos/%s", c.BaseURL, name))
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    var repo Repo
    return &repo, json.NewDecoder(resp.Body).Decode(&repo)
}

Previously, our v0 API tests had hardcoded a lot of URL routes and constructed HTTP requests in an ad-hoc manner. Now, they could use this API client to construct and issue requests.

2. Unify the interface of the HTTP API client and datastore

Next, we unified the interfaces of our HTTP API client and datastore. Previously our API http.Handlers were issuing SQL queries directly. Now our API http.Handlers just needed to parse the http.Request and then just call our datastore, which implemented the same interface as the HTTP API client.

Mirroring the (*repoService).Get method from the HTTP API client above, we now also had (*repoStore).Get:

func NewDatastore(dbh modl.SqlExecutor) *Datastore {
  s := &Datastore{dbh: dbh}
  s.Repositories = &repoStore{s}
  return s
}

type repoStore struct{ *Datastore }

func (s *repoStore) Get(name string) (*Repo, error) {
    var repo *Repo
    return repo, s.db.Select(&repo, "SELECT * FROM repo WHERE name=$1", name)
}

Unifying these interfaces put the description of the behavior of our web app in one place, making it easier to understand and reason about. Moreover, we could reuse the same data types and parameter structs in both the API client and datastore.

3. Centralize URL route definitions

Previously, we had to redefine URL routes in multiple layers of our application. In our API client, we had code like

resp, err := http.Get(fmt.Sprintf("%s/api/repos/%s", c.BaseURL, name))

This was very error prone, because we had over 75 route definitions, many of which were complex. Centralizing the URL route definitions meant refactoring the routes into a new package, separate from the API server. The definitions of routes were declared in the route package:

const RepoGetRoute = "repo"

func NewAPIRouter() *mux.Router {
    m := mux.NewRouter()
    // define the routes
    m.Path("/api/repos/{Name:.*}").Name(RepoGetRoute)
    return m
}

while the http.Handlers were actually mounted in the API server package:

func init() {
    m := NewAPIRouter()
    // mount handlers
    m.Get(RepoGetRoute).HandlerFunc(handleRepoGet)
    http.Handle("/api/", m)
}

Now, in the API client, we could use the route package to generate URLs, instead of having to hardcode them. The (*repoService).Get method is now:

var apiRouter = NewAPIRouter()

func (s *repoService) Get(name string) (*Repo, error) {
    url, _ := apiRouter.Get(RepoGetRoute).URL("name", name)
    resp, err := http.Get(s.baseURL + url.String())
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    var repo []Repo
    return repo, json.NewDecoder(resp.Body).Decode(&repo)
}

4. Create mocks of the unified interfaces

Our v0 tests tested the router, HTTP handlers, SQL generation, and DB querying at the same time. Test failures were hard to diagnose, and the tests were very slow.

Now, we have separate tests for each layer and we mock the functionality of the adjacent layers. Because each layer of our application implements the same interface, we can use the same mock implementation of that interface in all 3 layers!

The implementation of the mock is simply a struct of stub functions, which can be specified in each test:

type MockRepoService struct {
    Get_ func(name string) (*Repo, error)
}

var _ RepoInterface = MockRepoService{}

func (s MockRepoService) Get(name string) (*Repo, error) {
    if s.Get_ == nil {
        return nil, nil
    }
    return s.Get_(name)
}

func NewMockClient() *Client { return &Client{&MockRepoService{}} }

And here it is in use in a test. We mock the datastore's RepoService and use our HTTP API client to test the API http.Handler. (This snippet uses all the stuff we talked about above!)

func TestRepoGet(t *testing.T) {
   setup()
   defer teardown()

   var fetchedRepo bool
   mockDatastore.Repo.(*MockRepoService).Get_ = func(name string) (*Repo, error) {
       if name != "foo" {
           t.Errorf("want Get %q, got %q", "foo", repo.URI)
       }
       fetchedRepo = true
       return &Repo{name}, nil
   }

   repo, err := mockAPIClient.Repositories.Get("foo")
   if err != nil { t.Fatal(err) }

   if !fetchedRepo { t.Errorf("!fetchedRepo") }
}

High-level test goals revisited

Using the patterns mentioned above, we've achieved our goals for testing. Our test code is:

• Targeted: One layer is tested at a time.
• Comprehensive: All three layers are tested.
• Fast: Tests run very quickly.
• DRY: We've consolidated the common interface to the three layers of our application, and reuse them not only in application code but also in our tests.
• Mockable: One mock implementation can be used in all three layers of our application. And it can also be used by external API consumers who want to test their own libraries built on top of Sourcegraph.

So there you have it—the story of how we rebuilt and improved Sourcegraph's tests. The patterns and examples we've covered have worked well for us, and we hope they'll be useful to others in the Go community. But obviously they may not be right for every scenario, and we're sure there's room for improvement. We're constantly trying to improve the way we do things, so we'd love to hear your suggestions and feedback—tell us about your own experiences writing tests in Go!

Questions/comments?

Follow @srcgraph on Twitter, and email us at hi@sourcegraph.com. And try Sourcegraph now!

Also, we're hiring! Help us make programming faster, more accessible, and more enjoyable while building a strong open-source community around creating semantically aware tools for programmers. Email us and introduce yourself.