So you want to decentralize your website with IPFS

Last time we discussed decentralization, I was creating a dat version of this website using Beaker Browser. I want to try some other dex systems, and the next on my list is IPFS.

IPFS and dat have similar aims but differ in many respects:

A potential point of confusion is that the ‘decentralized web’ is low on dat’s stated priorities, but it’s the top priority for IPFS. But dat has a working, easily-installable web browser, and IPFS doesn’t. So, ironically, it’s currently much easier to create a website with dat than with IPFS.

As I write this, the buzz surrounding an IPFS-adjacent project, Filecoin, is heating up. It’s IPFS’s plan for distributed storage, and is launching in the flashy 2017 style of an Initial Coin Offering. If you’re very wealthy and brave, you can invest in it. This is not investment advice.

To compare the end goals of the Filecoin & dat approaches:

• In the land of Filecoin, you offer to pay for storage, and the most efficient, capable, and cheap hard drives win the bid and they host your files.
• In the land of dat, you serve as the first host of your website and then pay specific entities, like hashbase, to also continuously host your website. You could eventually pay multiple services as hosts, so you have automatic redundancy in the event of bankruptcy or network outage.

Anyway: this is not meant to be a buyers guide or technical deep-dive. I’m going to use IPFS and see how it works.

Installing IPFS

Okay: so, there’s no Beaker Browser-like downloadable desktop application for IPFS. There’s a command-line tool called ipfs. Because the reference implementation of IPFS is written in Go, they can distribute this command-line tool as one file that you can download and that doesn’t have any dependencies (you don’t have to install Go).

The user experience of downloading IPFS is acceptable but not great. From the homepage, you click ‘Try it’, ‘Install IPFS now’, ‘Install IPFS for your Platform’, and then ‘download go-ipfs’. The flow has some kinks:

• it refers to OS X in one spot, and then as Darwin elsewhere. Pedantically, it’s macOS ever since 2016.
• You can probably grok from the link that you want to install go-ipfs, but the path until that point talks about installing IPFS, not go-ipfs.
• The download page links to a README file for documentation. It doesn’t link to the significantly less-intimidating Getting Started page.

Okay, so I’ve got go-ipfs_v0.4.10_darwin-amd64.tar.gz downloaded, so I unpack it and move ipfs to /usr/local/bin so that typing ipfs on the command line works. I run through IPFS’s first run experience:

~/Downloads/go-ipfs〉ipfs init
initializing IPFS node at /Users/tmcw/.ipfs
generating 2048-bit RSA keypair...done
peer identity: QmVf5UwKt2E5foja4y9gsXXCPhb3bnsR9wTR6PejbtYdFT
to get started, enter:

	ipfs cat /ipfs/QmVLDAhCY3X9P2uRudKAryuQFPM5zqA3Yij1dY8FpGbL7T/readme

~/Downloads/go-ipfs〉ipfs cat /ipfs/QmVLDAhCY3X9P2uRudKAryuQFPM5zqA3Yij1dY8FpGbL7T/readme
Hello and Welcome to IPFS!

██╗██████╗ ███████╗███████╗
██║██╔══██╗██╔════╝██╔════╝
██║██████╔╝█████╗  ███████╗
██║██╔═══╝ ██╔══╝  ╚════██║
██║██║     ██║     ███████║
╚═╝╚═╝     ╚═╝     ╚══════╝

If you're seeing this, you have successfully installed
IPFS and are now interfacing with the ipfs merkledag!

 -------------------------------------------------------
| Warning:                                              |
|   This is alpha software. Use at your own discretion! |
|   Much is missing or lacking polish. There are bugs.  |
|   Not yet secure. Read the security notes for more.   |
 -------------------------------------------------------

Check out some of the other files in this directory:

  ./about
  ./help
  ./quick-start     <-- usage examples
  ./readme          <-- this file
  ./security-notes

Quick notes here:

• When I run ipfs init, it refers to initializing IPFS node at /Users/tmcw/.ipfs. In Getting Started, this is referred to as a ‘global local object repository’. Is it a node or a repository? Also, how can global local be a compound adjective? 🤔
• Is it IPFS or ipfs? It’s an acronym, but is referred to in lowercase in several contexts.
• merkledag is a cool and fun word, but is it necessary to use it here? In the IPFS endgame, do indie bands setting up their websites say merkledag to each other?
• Running ipfs says ‘Global p2p merkle-dag filesystem.’ and the readme says ‘interfacing with the ipfs merkledag!’. Is it merkle-dag or merkledag? The IPLD spec also refers to ‘merkle dag’.

Apology for the pedantry of this post. That said, IPFS and the decentralized web is introducing a lot of new lingo, as they replace an ever-increasing portion of web technology. Technical writing is difficult. Words are work. If we’re going to have new words, clean and consistent usage will make things a lot easier for everyone involved.

Okay, so I’m running ipfs daemon, which has the following output:

Initializing daemon...
Swarm listening on /ip4/10.6.117.62/tcp/4001
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/172.125.81.133/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

There’s a pattern in IPFS that differs from dat: where dat pretty early on claimed dat:// URLs for its own, IPFS has been slower on the uptake and instead refers to IPFS resources like a filesystem, like /ipfs/QmW2WQi7j6c7UgJTarActp7tDNikE4B2qXtFCfLPdsgaTQ/cat.jpg. Referring to objects on IPFS seems fine, but I’m not sold on the choice to refer to TCP/IP addresses in the same filesystem-path style: in this example output, /ip4/127.0.0.1/tcp/8080 is equivalent to 127.0.0.1:8080, but is such a needlessly confusing way to spell out an IP address and port.

Adding my website to IPFS

So, from Getting Started to Onward to more Examples to IPFS for Websites, I see that to add a website to IPFS, I run ipfs add -r mysite.

So, I use jekyll build to build my website using Jekyll, just like my dat-centric post, then go to _site and run ipfs add -r . That outputs a long list of hashes with content to the CLI:

added QmZKTxWvJZADMXTRfYJKKgci1oGq38f6Tyb1K3onv4AQxL _site/projects
added QmU6REJYTGSXogDYzB6k2MmVzSDC5fdtrEhcdNSLTzENFV _site/resources
added Qmc4LtYiBWRsQnanVTHHSbSq6jm6c1hkCqrbRhfB2jZAeD _site/resume
added QmSFpHibaTzYUNjZjQ5qh1WJrWrR2r28Uhsoftayef1DMy _site/simple-statistics
added QmbvADibAvBgusnaaNTVsqrhwteZuRc2XngyrK7KC2tBjm _site/swift
added QmXHHgNSQVf2KjdAsxxXeVG9fN582CLFhDxdwvJyjMPVDP _site/talks
added QmZmEkw6srCD2mCLQV4SpPbqFpKES4kX3d8wW2tRBHeuHf _site/topics
added QmR96iiRBEroYAe955my8cdHkK4kAu6SApGC2eS5kSzHwt _site

I have the IPFS daemon running and the way to access one of these files is to put it in the $HASH part of http://localhost:8080/ipfs/$HASH/, so I open http://localhost:8080/ipfs/QmR96iiRBEroYAe955my8cdHkK4kAu6SApGC2eS5kSzHwt/

Anyone who maintained a website in the old bad days of subdirectories can see what comes next: the homepage works, but clicking a link like

<li><a title='08-01-2017' href='/2017/08/01/recently.html'>Recently</a></li>

Goes to http://localhost:8080/2017/08/01/recently.html, and 404s. And even if I used the base tag to re-route that link so that it points

http://localhost:8080/ipfs/QmR96iiRBEroYAe955my8cdHkK4kAu6SApGC2eS5kSzHwt/2017/08/01/recently.html

It wouldn’t work: the correct URL for that Recently post is, instead,

http://localhost:8080/ipfs/QmTbJ6RSLZDmVYy8dgdoeQLCtKya7UrNTCqaw93yZCpH5T/

So IPFS content links are fully content-addressed. I suppose to make my site fully IPFS, I’d have to build each individual page and then construct a home page that linked to the generated hashes for those pages. That leaves an open question: how could two pages link to each other? Adding a link from one page to the other would change its hash, so wouldn’t it be impossible for pages to reference each other? This might be a lack-of-coffee problem on my part.

Visiting someone else’s IPFS site

So, I published my site to IPFS and was able to visit it, but couldn’t figure out how links could work. Now I’d like to visit another website with IPFS.

For that, there’s NeoCities, Kyle Drake’s indie effort to create a GeoCities-like website host in the modern world. Kyle enabled IPFS for NeoCities, instantly enlarging the IPFS web by 140,000 websites.

From a NeoCities page about an awesome 8-bit Dragon Quest game, I go to archives, and then hack that into a URL that asks my local IPFS daemon for it: http://localhost:8080/ipfs/QmSjtF5z9XfGTjdGAM6e6dFR3LE8WH7Tnwwncs8xPou8Jb/

That works. But note that this way of visiting IPFS websites doesn’t follow the paranoid security standards of Beaker Browser - that request to that Dragon Quest game required jQuery over HTTP. If there’s a way for IPFS websites to be as secure than normal websites, I haven’t found it.

Fin

This is all alpha and beta software. Even the most optimistic observer would say that decentralized web tech has another 2-5 years until it might be mainstream. Protocol Labs is, like the dat team, focusing on low-level fundamentals first, rather than user experience testing.

From an end-user perspective, IPFS is significantly more intimidating and challenging than dat right now. You need to be comfortable with reading URLs, using the command-line, and digesting lots of new technical language. There’s little encapsulation, little abstraction on top of the protocol.

I expect a lot of that will change, but still wonder which terms and concepts will be exposed to eventual users. Will a Jane Everywoman say merkledag, and will immutable & content-addressable hashes become a user-facing feature?

The IPFS is useful here and now section of the homepage has me a little wary: is it really useful right now, with this flow? Trying to host a website with IPFS seems like a hack right now - the web security model isn’t there, the technical requirements for someone to browse an IPFS website are outside the reach of non-programmers.

Of course, all of this is going to change. Filecoin will likely make persistent web hosting much, much easier. Browser support for IPFS could be on the horizon. Documentation will hopefully improve. Protocol Labs’s papers and presentations are impressive, and they’ve garnered a lot of respect from adjacent communities. Using the tech as it stands today is fun, but not easy. It’s promising, but it’s hard to figure out what it’ll become.

Also, some of my experiences here may be simply mistakes or misunderstandings on my part. If so, absolutely let me know: my email’s on the /about page.

Notes

• I linked dnslink above and it’s a blank repo. But it’s what you get when you Google ‘IPFS dnslink’ and I can’t find any better reference material about it. I’m happy to update that link to something more descriptive but as it stands it’s kind of an unfortunate example of a spec that’s widely implemented but doesn’t seem to be described anywhere.
• Redditors were quite dismayed about the fact that I didn’t mention ZeroNet last time, at all. I didn’t mention it because I hadn’t heard of it. I’ve now heard of it, and might try it out. That said, I continue to be completely disinterested in infighting and smack-talk between these projects. Politically, disadvantaged groups benefit from big-tent approaches and unifying voices, not trolls.
• Correction from earlier version of this post: dat has raised $1,370,000. Previously this post said $730,000, which didn’t include $640,000 from the Sloan Foundation in 2015.