Go: 90% Perfect, 100% of the time.

"GoCon Tokyo"

31 May 2014

Brad Fitzpatrick

"60% of the time, it works every time...."

Starting a new project, picking a language...

Disclaimer

Personal experience only
My opinions
Fake, fun numbers (e.g. "90% Perfect, 100% of the time.")
Maybe it's 91%.
All languages are both wonderful and terrible
<3

Idea: Camlistore

All my life's data from everywhere,
Import, backup, serve, browse, search, share, index
Designed to last 100+ years
Don't trust any company
In 2010: an idea
In 2014: busy, fun, active project

Which language to use for Camlistore?

Before Go

Starting a new project from 1994 ~ 2010...
Perl or C?
Rarely but occasionally Java.

Perl: The Good

Fun
Expressive
Get things done, quickly

Python, Ruby, Javascript: The Good

Same

Perl: The Bad

Not fast like C
Sometimes need to write in C
... or write in XS (Perl-C glue language)
Perl-C, C-Perl calls are slow
Single-threaded, single-core, callback hell

Python, Ruby, Javascript: The Bad

Same, but even slower :)
JavaScript is faster now, but not as fast as C

Summary

Perlythonubyscript: slow for computer, fast & fun for human
C: slow for human, fast for computer
C++: little more fun than C, but more complexity
Java: tedious (verbose), slow start-up, easy to find programmers
PHP: if somebody's paying you enough, maybe

Maybe mix of two languages?

Fun, slow parts in Perlythonubyscript?
Important, fast parts in C?

But...

You'll want to share code.
Calling from C to X or X to C is slow.
The C-X glue language is the least fun language
You'll be lazy, and write in the fun language
Or be lazy and waste time, and write too much stuff in C++ (like Google)

Threads

C, C++, Java, ...
Threads? Heavy. Lots of memory per thread for stacks. Be careful!

Executor pool = new ScheduledThreadPoolExecutor(/* ???? guess a number */);

Events

Perlythonubyscript: events. no choice. Callback hell.
C, C++: libevent, etc. Callback hell.

C#: await async await async await async await async noise

It's really hard to pick a language!

Go: 90% Perfect, 100% of the time.

Go is very good at lots of things
High-level code when you want
Low-level code when you want
Start high, profile, go low-level when needed
Static typing without too much keyboard typing

Go's Concurrency

Built-in! Lightweight threads, no callback hell
Readable, top-down code
So easy to write servers

for {
  conn, err := listener.Accept()
  // check err
  go serve(conn)
}

Goroutine-per-request can scale

Go is showing up for all sorts of tasks

Web frameworks

Typically a job for scripting languages: Node.js, Python, Ruby, Perl, PHP, etc.
net/http in the standard library
Other http frameworks on top: gorilla, martini, Revel etc...
No need for nginx: standard library's http server scales

Image processing

no libpng, imagemagick, etc
pure Go PNG, JPEG, GIF encoding & decoding in the standard library
slower, but safer
getting faster

Crypto

no OpenSSL, no GnuTLS
pure Go AES, TLS (SSL), OpenPGP, etc
slower, but safer
getting faster

Replacing shell scripts

Instead of little Perl & shell scripts, I now write in Go
os/exec package & goroutines makes subprocess management easier than shell

Camlistore

web handlers
using third-party web APIs
image thumbnails: decode, resize, encode
EXIF, ID3 parsing
database drivers
crypto, image decoding/resizing/encoding
search, indexing: tight memory layout control
WebSockets, HTTPS
all pure Go!

Controlling flying drones

www.godrone.io

"Isn’t Go unsuitable for real-time applications like this?"
"However, for all practical purposes the GC just needs to keep up with the stabilization loop which runs at 200 Hz. This means that GC pauses below 5ms have no impact on performance. Longer pauses will degrade stabilization performance, but the tolerance threshold may be up to a second ..." (small heap)

Disassembler, linker, compiler.

llgo compiler (Go -> LLVM IR -> ...)
Go 1.3's disassembler
Go 1.4's linker
Go 1.4 or Go 1.5's compiler

Mobile

Go runs on ARM
minux's iOS port of Go
Camlistore child process, goandroid

Audio synthesis

https://github.com/nf/sigourney

Cloud infrastructure

Go: the emerging language of cloud infrastructure

Docker, Packer
CoreOS’s etcd and fleet
Ubuntu Juju, Mozilla Heka, Apcera’s NATS, gnatsd

Load balancers & servers

YouTube's vitess: MySQL
net/http/httputil.ReverseProxy
dl.google.com
etc

What other language is used for all these?

web apps
scripts
system administration
image processing
load balancers, servers
crypto
hardware

Go!

Go built-in tools

testing
benchmarking
profiling (CPU, memory, blocking)
huge standard library
"go get"
godoc
gofmt, goimports
race detector
static binaries

Generics

No generics
It sucks sometimes, but rarely
Use maps, slices, interfaces
Occasionally a gross interface{}
No algorithms in stdlib
No great proposal yet

No generics (Simon Peyton Jones: "Haskell is useless")

Data races can happen

shared, mutable state is possible
use channels
be careful
conventions and documentation help
runtime race detector helps too

$ go test -race

Code generation

The default Go compiler often generates pretty dumb code
Especially on ARM
But getting better
In Go 1.4, 1.5+: Go compiler in Go, refactor, add SSA, more optimizations

gccgo

gccgo generates very good code
... but lacks escape analysis: kills performance with many small allocs + garbage
... GC isn't precise. Bad for 32-bit.

Compiling to JavaScript isn't yet great

Would be nice to write frontend & backend in the same language, share code
go/types + go/ssa enables many things:

TARDIS Go Compiler

Limited Mobile Support

No built-in Android support (but goandroid, child processes...)
No built-in iOS support (but unmerged port...)

Embedding Go in C/C++/Java/etc

Can't embed Go in other languages, environments.
e.g. Android, iOS, C/C++ programs
problems with memory setup, GC, signals, ...
need embedding API design
In Go 1.4?

Shared libraries

no shared libraries
can't load Go code at runtime

Garbage collector

Pauses: faster each release, good people working on it.
Precision: heap in 1.2, most stacks in 1.3, more in 1.4.
4 versions of memcached: Perl, C, C++, Go

Hot stack splits

Almost entirely fixed in Go 1.3's contiguous stacks
Rest should be in Go 1.4.

But...

Problems are fixable.
It all keeps getting better with each release.

Go for everything

Since mid-2010, I prefer 90%-perfect Go for all my hacking:

Go is flexible, fun, readable
Go is fast (for computers & humans)
Go is good for working with others
Go for everything!

Go: 90% Perfect, 100% of the time.

"GoCon Tokyo"

31 May 2014

Hello!

"60% of the time, it works every time...."

"That makes no sense"

Starting a new project, picking a language...

Disclaimer

2010

Idea: Camlistore

Which language to use for Camlistore?

Before Go

Perl: The Good

Python, Ruby, Javascript: The Good

Perl: The Bad

Python, Ruby, Javascript: The Bad

Summary

Fun vs. fast

Maybe mix of two languages?

But...

Server? Threads or events?

Threads

Events

Concurrency and performance

It's really hard to pick a language!

2010. Let's try Go!

Go: 90% Perfect, 100% of the time.

Go: 90% Perfect, 100% of the time.

Before Go

After Go

Go's Concurrency

Concurrency before Go

After Go

Go is showing up for all sorts of tasks

Web frameworks

Image processing

Crypto

Replacing shell scripts

Camlistore

Controlling flying drones

Emulators

Disassembler, linker, compiler.

Mobile

Audio synthesis

Cloud infrastructure

Load balancers & servers

Raspberry Pi GPIO

What other language is used for all these?

And even better...

Go built-in tools

So why isn't Go perfect?

Generics

No generics (Simon Peyton Jones: "Haskell is useless")

Data races can happen

Code generation

gccgo

Compiling to JavaScript isn't yet great

Limited Mobile Support

Embedding Go in C/C++/Java/etc

Shared libraries

Garbage collector

Hot stack splits

But...

Go for everything

Thank you