Compiler Explorer and the Promise of URLs That Last Forever

The history of Compiler Explorer's URL system is a tale of evolution, compromise, and the quest for a promise that seemed impossible at first: URLs that last forever.

In the early days of 2012, we stored the entire state of the compiler in the URL. It quickly became unwieldy, leading us to add support for Google's link shortener goo.gl in March 2014. This change allowed us to shorten links into a manageable form: goo.gl/abc123. Clicking on these short links would eventually redirect you to the full URL link on our site, and we'd decode the state from the URL.

In 2016, Stack Overflow banned link shorteners due to their vulnerability to abuse, affecting our own Compiler Explorer links. We had no intention of storing any user data at the time, so we came up with a hack: we still used goo.gl, but rewrote the link we handed out to be godbolt.org/g/abc123 (where the abc123 was the goo.gl unique ID). We then redirected any hits to /g/abc123 to goo.gl/abc123, which ultimately redirected back to godbolt.org with the appropriate state in the URL.

This was a three-redirect solution, and we were committed to making things complicated. Later, we used Google's API to avoid this redirection dance.

However, by 2018, the limitations of storing state in the URL began to bite. We needed a better solution, so we implemented our own storage system: hashing the input, saving the state as a JSON document on S3 under the hash, and providing a shortened form of the hash as a godbolt.org/z/hashbit URL.

We use DynamoDB to store the mapping of shortened hashes to full paths, accounting for partial collisions. We even check the short link's hash for rude words and add deliberate extra information into the document until we no longer get a rude word.

Despite Google's solemn promise that "all existing links will continue to redirect to the intended destination," their URL shortener went read-only a few years back, and is now finally sunsetting in August 2025. This means we'll no longer be able to resolve goo.gl-based links!

This has left me with a dilemma: I want Compiler Explorer links to last forever. While we can't do anything about the really legacy actual goo.gl links, we can create our own database of links and redirect them in-house.

A Digital Archaeological Dig

Over the last few days, I've been scraping everywhere I could think of, collating the links I found out in the wild, and compiling my own database of links. So far, I've found 12,000 links from scraping: 12,298 rescued links and counting – not bad for a few days of digital archaeology.

We're now using this database in preference to goo.gl internally, so I'm also keeping an eye on new "g" links that we don't yet have. If you have a secret cache of godbolt.org/g/abc123 links lying around, now's the time to visit each of them! That will ensure they're in my web logs and I'll add them to the database.

A Lesson in Critical Infrastructure

This whole saga reinforces why I'm skeptical of relying on third-party services for critical infrastructure. Google's URL shortener was supposed to be permanent, but we soon learned that even the best-laid plans can go awry.

The redirect chains we built were clever workarounds that bought us time, but ultimately, the only way to truly keep a promise of "URLs that last forever" is to own the entire stack. It's been a fascinating archaeological dig through the internet, hunting down these legacy links like some sort of digital Indiana Jones, except instead of ancient artifacts I'm rescuing compiler flags and optimization examples.

Each one represents someone's attempt to share knowledge, ask a question, or demonstrate a concept. Preserving them feels like preserving a small piece of programming history.

A Promise Kept

If you've got old Compiler Explorer links bookmarked somewhere, dust them off and give them a click. You'll be helping preserve a little corner of the internet's shared knowledge – and keeping a promise I made back in 2012.

And hey, at least this time I'm in control of the infrastructure. What could possibly go wrong?