Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

[Posted April 14, 2016 by jake]

Over at the Freedom to Tinker blog, guest poster Vitaly Shmatikov, who is a professor at Cornell Tech, writes about his study [PDF] of what URL shortening means for the security and privacy of cloud services. "TL;DR: short URLs produced by bit.ly, goo.gl, and similar services are so short that they can be scanned by brute force. Our scan discovered a large number of Microsoft OneDrive accounts with private documents. Many of these accounts are unlocked and allow anyone to inject malware that will be automatically downloaded to users’ devices. We also discovered many driving directions that reveal sensitive information for identifiable individuals, including their visits to specialized medical facilities, prisons, and adult establishments."

(Log in to post comments)

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 14, 2016 20:29 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

Heh. I've always disliked shorteners. Seems like a good reason to continue to not use them. Also fun is finding the shorteners which need JS to actually work…

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 10:14 UTC (Fri) by jezuch (subscriber, #52988) [Link]

> Heh. I've always disliked shorteners.

I was going to say that! :) Yeah, they are opaque and very unhelpful when I want to quickly and efficiently determine whether I want to go there at all. You never know where you're going to land. This alone has some security implications...

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 11:30 UTC (Fri) by pboddie (guest, #50784) [Link]

Careful: you'll get shouted down by the "everybody else does it" brigade. It gets even more annoying when people use these obfuscated URLs in communications where the original URL need not be short, like normal HTML where it is the link text that gets displayed.

Sadly, 25 years on and the exotic wizardry of hypertext remains barely understood by the people whose job involves communicating with others on the Web, as their communications gradually degrade into a string of hash- and at-prefixed keywords mixed with opaque references that depend on a handful of proprietary services for their correct interpretation, making those utterances even less comprehensible when reviewed in 25 years' time.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 12:55 UTC (Fri) by ballombe (subscriber, #9523) [Link]

I agree with you but CMS that generate ridiculously long and non copy-pastable URL are to blame fo the invention of shorteners.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 15:02 UTC (Fri) by drag (guest, #31333) [Link]

> I agree with you but CMS that generate ridiculously long and non copy-pastable URL are to blame fo the invention of shorteners.

The reply to that is obviously:

> Sadly, 25 years on and the exotic wizardry of hypertext remains barely understood by the people whose job involves communicating with others on the Web...

If you are puking your internal data structures into the URL then you are doing something wrong, I figure. URL shorteners are just a symptom of a bigger problem.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 26, 2016 12:10 UTC (Tue) by robbe (guest, #16131) [Link]

> If you are puking your internal data structures into the URL

If you’re CMS puts the article "Gone In Six Characters: Short URLs Considered Harmful for Cloud Services" under https://example.org/Gone-In-Six-Characters-Short-URLs-Con... it is NOT because it somehow shows its innards (that’s much more the case, if the article in question is at https://lwn.net/Articles/683880/)

The reason for these verbose URLs seems to be search engine "optimisation" (newspeak for tricking). I don’t know if Google (are there other engines these SEOers and their customers care about?) still gives more weight to keywords in the URL than in the text, or if it ever did.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 26, 2016 12:50 UTC (Tue) by itvirta (guest, #49997) [Link]

Including the title in the URL is actually _useful_ too, if you happen to only have the URLs saved.
It's rather annoying to pick the correct one amongst many that differ only by an opaque number.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 26, 2016 18:04 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

The best patterns I've seen are "https://host/path/id/slug" where the slug doesn't matter (so if the link gets word-wrapped or truncated, it still resolves properly), but is still useful when searching history or whatever. But that doesn't really work for non-static websites.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 15:24 UTC (Fri) by ledow (guest, #11753) [Link]

That's a problem with the CMS, not the URL.

And every CMS I've ever used has a "friendly URL" option which basically just puts the logical location (e.g. fred.com/section/subsection/page) as the URL string.

There's nothing worse than copy-pasting an Amazon string, even and discovering a pile of unnecessary junk on the end of it.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 25, 2016 14:18 UTC (Mon) by JFlorian (guest, #49650) [Link]

> There's nothing worse than copy-pasting an Amazon string, even and discovering a pile of unnecessary junk on the end of it.

I oft wonder if my soul is part of that unnecessary junk and if I've made some sort of deal with the devil if I don't trim it.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 14, 2016 20:34 UTC (Thu) by niner (subscriber, #26151) [Link]

A quick test shows that even with manual trial and error one can find working URLs quite easily. A "take me to some random page" feature would be very simple to build...

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 14, 2016 20:59 UTC (Thu) by noahm (subscriber, #40155) [Link]

The difference in tone of the responses from Google and Microsoft, when informed of this problem with their services, is really interesting. One of these companies either didn't comprehend the significance of the problem or didn't take seriously the threat to their users. The other fixed the problem promptly, and is working on additional defenses for the future.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 0:12 UTC (Fri) by dirtyepic (guest, #30178) [Link]

Which is which?

Just kidding, we all know the answer.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 6:10 UTC (Fri) by eru (subscriber, #2753) [Link]

In one sense MS is right: the attack works because the URLs are short, and making short URLs is the whole point of URL shorteners. I suspect you would need something like 20 characters to make brute-forcing infeasible today, and the minimum length would grow over time.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 14:28 UTC (Fri) by khim (subscriber, #9252) [Link]

I don't really see why. Note that we are NOT talking about some arbitrary functions which you could calculate locally. Rather we talk about something you need to ask remote server about!

Which means that if server responds fast enough to make human reader happy but not fast enough to make brute-force attack feasible... then that's it: fast computers and ASICs wouldn't change anything for that equations.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 6:51 UTC (Fri) by epa (subscriber, #39769) [Link]

Hasn't it always been a tenet of the Web that relying on keeping a URI 'secret' is doomed?
If you have sensitive information to protect, don't rely on others not being able to guess the URI; protect it with a password or other authentication mechanism instead.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 7:26 UTC (Fri) by tao (subscriber, #17563) [Link]

Indeed. If there really are services that rely on the URI for security, then those services are flawed -- URI shortening or not.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 7:28 UTC (Fri) by DOT (guest, #58786) [Link]

The URI is a red herring in this case, since you can consider it the password. The real problem is that the password was too short.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 9:06 UTC (Fri) by epa (subscriber, #39769) [Link]

That's what I am saying. Treating the URI as a password and trying to keep it 'secret' is a flawed approach. Accept that the URI can be found out by anyone who does a bit of digging, and if you have sensitive information to protect, use a password or other authentication to protect it.

A longer URI would be harder to find by brute force, but that is only a sticking plaster for the problem, since it will still turn up in Referer: headers and so on.

That said, it is certainly true that the insecure approach is usually the more convenient one, and passing around URIs which don't require any further login details is always going to beat any other approach in convenience. So a sticking plaster may be the best we can do at the moment. If everyone in the world had a Google account then it would be trivial to 'share these driving directions with the following users...' but, thank goodness, the world is more messy than that.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 11:10 UTC (Fri) by alonz (subscriber, #815) [Link]

It actually is quite common to have a URI act as a password: just look e.g. at the URIs for Google photos.

And there is good reason for this practice—it enables the photo owner to share it with friends without them having to sign in. So sure, it's limited (if the URI leaks, it's usable by anyone) but it is a valid trade-off.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 10:43 UTC (Fri) by niner (subscriber, #26151) [Link]

So how exactly can I password protect the Google Maps route I'm sending to someone?

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 11:25 UTC (Fri) by NAR (guest, #1313) [Link]

I think there's an option to share it with only specific Google Maps users (i.e. with those who have a Google account).

What I don't quite understand is how do they know who created that map?

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 19:33 UTC (Fri) by cwitty (subscriber, #4600) [Link]

"What I don't quite understand is how do they know who created that map?"

If you're talking about the geocacher map, the researchers created the map as a summary of hundreds of sets of driving directions, all starting at one particular residential address. So it seems reasonable to assume that the person who requested all of those driving directions lives at that address.

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 17, 2016 20:41 UTC (Sun) by pr1268 (subscriber, #24648) [Link]

So it seems reasonable to assume that the person who requested all of those driving directions lives at that address.

Either that, or the researchers stumbled upon someone's malicious prank to inundate said address with dozens of unwanted visitors.

Okay, I'm being a little facetious here, but it could happen!

Gone In Six Characters: Short URLs Considered Harmful for Cloud Services (Freedom to Tinker)

Posted Apr 15, 2016 12:05 UTC (Fri) by lmb (subscriber, #39048) [Link]

Not to mention that these various URL redirection services amass tons of data on who accesses the links from where.

It's not uncommon nowadays to be shuffled through three or more layers of indirection, allowing an exact mapping of the social media graph it got passed through.