See, that’s what the app is perfect for.

The Core Web team at Tumblr is proud to announce the release of Laphs (Live Anywhere Photos - LAPhs; get it?), an open source JavaScript library for implementing Apple’s Live Photos on the web.

We use Laphs to support Live Photos on the web at Tumblr and now you can too! Check it out on github and npm and let us know what you think.

Happy coding!

At Tumblr, we’re always looking for new ways to improve the performance of the site. This means things like adding caching to heavily used codepaths, testing out new CDN configurations, or upgrading underlying software.

Recently, in a cross-team effort, we upgraded our full web server fleet from PHP 5 to PHP 7. The whole upgrade was a fun project with some very cool results, so we wanted to share it with you.

Timeline

It all started as a hackday project in the fall of 2015. @oli and @trav got Tumblr running on one of the PHP 7 release candidates. At this point in time, quite a few PHP extensions did not have support for version 7 yet, but there were unofficial forks floating around with (very) experimental support. Nevertheless, it actually ran!

This spring, things were starting to get more stable and we decided it was time to start looking in to upgrading more closely. One of the first things we did was package the new version up so that installation would be easy and consistent. In parallel, we ported our in-house PHP extensions to the new version so everything would be ready and available from the get-go.

A small script was written that would upgrade (or downgrade) a developer’s server. Then, during the late spring and the summer, tests were run (more on this below), PHP package builds iterated on and performance measured and evaluated. As things stabilized we started roping in more developers to do their day-to-day work on PHP 7-enabled machines.

Finally, in the end of August we felt confident in our testing and rolled PHP 7 out to a small percentage of our production servers. Two weeks later, after incrementally ramping up, every server responding to user requests was updated!

Testing

When doing upgrades like this it’s of course very important to test everything to make sure that the code behaves in the same way, and we had a couple of approaches to this.

Phan. In this project, we used it to find code in our codebase that would be incompatible with PHP 7. It made it very easy to find the low-hanging fruit and fix those issues.

We also have a suite of unit and integration tests that helped a lot in identifying what wasn’t working the way it used to. And since normal development continued alongside this project, we needed to make sure no new code was added that wasn’t PHP 7-proof, so we set up our CI tasks to run all tests on both PHP 5 and PHP 7.

Results

So at the end of this rollout, what were the final results? Well, two things stand out as big improvements for us; performance and language features.

Performance

When we rolled PHP 7 out to the first batch of servers we obviously kept a very close eye at the various graphs we have to make sure things are running smoothly. As we mentioned above, we were looking for performance improvements, but the real-world result was striking. Almost immediately saw the latency drop by half, and the CPU load on the servers decrease at least 50%, often more. Not only were our servers serving pages twice as fast, they were doing it using half the amount of CPU resources.

These are graphs from one of the servers that handle our API. As you can see, the latency dropped to less than half, and the load average at peak is now lower than it’s previous lowest point!

Language features

PHP 7 also brings a lot of fun new features that can make the life of the developers at Tumblr a bit easier. Some highlights are:

• Scalar type hints: PHP has historically been fairly poor for type safety, PHP 7 introduces scalar type hints which ensures values passed around conform to specific types (string, bool, int, float, etc).
• Return type declarations: Now, with PHP 7, functions can have explicit return types that the language will enforce. This reduces the need for some boilerplate code and manually checking the return values from functions.
• Anonymous classes: Much like anonymous functions (closures), anonymous classes are constructed at runtime and can simulate a class, conforming to interfaces and even extending other classes. These are great for utility objects like logging classes and useful in unit tests.
• Various security & performance enhancements across the board.

Summary

PHP 7 is pretty rad!

We’ve been looking at improving the posting and reblogging experience in our mobile apps for a long time. As many of our power users and public API consumers are aware, posts on Tumblr are stored and served in a sanitized HTML format. This choice made the most sense when Tumblr was originally built, when using Tumblr meant visiting via a web browser on your computer on the information superhighway back in 2007.

Storing post content primarily as HTML has remained our standard for ten years; there are a significant number of assumptions in our codebase about posts being primarily HTML. To compound this, when we want to change something about how posts are made or stored, we have to think in terms of the 150 billion posts on Tumblr and the billion new posts made every month. We have to spend a lot of time thinking about that scale whenever we consider how to make posting on Tumblr a better experience.

Over a year ago, Tumblr Engineering came up with a very ambitious idea: ditch HTML entirely and move to a brand new format for how posts are created and stored. HTML is fine, but its scope is limited as it was intended for the browser, long before the concept of mobile apps existed. Conversely, the JSON standard has been heavily favored by APIs and mobile development for years, and feels much cleaner and more flexible than HTML. We can apply an extensible format and schema with JSON easier than we can with HTML.

With this in mind, we’ve chosen to write a brand new JSON-based specification for post content. We’re calling it the Tumblr Neue Post Format, or simply NPF. The NPF specification breaks apart post content into discreet content blocks, each defined by a type field. All of our existing post content easily fits into this kind of specification, affording backwards-compatibility with the billions of posts on Tumblr.

For example, right now when you add text to a post, we store and serve:

<p>Some text in a post!<p>

With NPF, the same thing is created and served this way:

{
  "type": "text",
  "text": "Some text in a post!"
}

Those two representations are fully interchangeable, but we begin to gain advantages with JSON for things HTML cannot do well, providing flexibility and extensibility for future integrations. The power of NPF really becomes critical when we want to build content blocks for Tumblr that cannot be easily represented with HTML, such as a physical location:

{
  "type": "location",
  "latitude": 40.739485,
  "longitude": -73.988402,
  "map_style": "quirky"
}

This new JSON specification also gives us the benefit of not having to worry as much about potential security risks in malicious HTML payloads in post content. Moving from HTML to JSON allows us to have safer, more injection-proof defaults, and prevents us from having to do heavy DOM parsing at runtime, which means improved performance of our backend and mobile apps. With NPF, posting and viewing posts on Tumblr should be considerably faster and safer.

Our work so far with the NPF specification has been to reach feature parity with the rich text editor available to Tumblr users on the web, as well as extend those basic options with new ones, such as fun new text styles:

{
  "type": "text",
  "text": "Oh, worm?",
  "subtype": "quirky"
}

Our initial release includes support for text blocks (with inline formatting), GIF search blocks, and image upload blocks. All of these options are available in our mobile apps via the Text, Quote, and Chat post forms, as well as when you reblog a post. Yes, you can now upload images in a reblog on mobile.

Future releases of the mobile apps will continue to close the gap with our other post options as we build NPF support for link blocks, video upload blocks, third-party video and audio blocks, and more. We also plan on allowing third-party API consumers to view and create posts using the new specification sometime in the future.

- me (@cyle) and @noisysocks (with love for @keithmcknight who started the original NPF spec)