In 2002, I grew annoyed with not finding the obscure technical information I was looking for, so I started Gmane, the mailing list archive. All technical discussion took place on mailing lists those days, and archiving those were, at best, spotty and with horrible web interfaces.
The past few weeks, the Gmane machines (and more importantly, the company I work for, who are graciously hosting the servers) have been the target of a number of distributed denial of service attacks. Our upstream have been good about helping us filter out the DDoS traffic, but it’s meant serious downtime where we’ve been completely off the Internet.
Of course, there are ways to try to mitigate all this: I’m moving the Gmane servers off of my employer’s net, and I’m putting Cloudflare in front of the Gmane web servers.
But I ask myself: Is this fun any more?
Running a mailing list archive means, of course, that people want stuff removed from the archive, so I’ve apparently been sued in India (along with Google and Yahoo) (and I’m never going there: I might be sentenced. I don’t know). And I’m the Internet Help Desk, which is nice, but confusing. And all the threats of “legal action” are, well, something.
And now the DDoS stuff, which I have no idea why is happening, but I can only assume that somebody is angry about something.
Probably me being a wise ass.
So… it’s been 14 years… I’m old now. I almost threw up earlier tonight because I’m so stressed about the situation. I should retire and read comic books and watch films. Oh, and the day job. Work, work, work. Oh, and Gnus.
I’m thinking about ending Gmane, at least as a web site. Perhaps continue running the SMTP-to-NNTP bridge? Perhaps not? I don’t want to make 20-30K mailing lists start having bouncing addresses, but I could just funnel all incoming mail to /dev/null, I guess…
The nice thing about a mailing list archive (with NNTP and HTTP interfaces) is that it enables software maintainers to say (whenever somebody suggests using Spiffy Collaboration Tool of the Month instead of yucky mailing lists) is “well, just read the stuff on Gmane, then”. I feel like I’m letting down a generation here. And despite what I rambled about in that paragraph up there, I’ve had many fun interactions with people because of Gmane. And lots and lots and lots of appreciative feedback over the years.
*sigh*
But there’s The Mail Archive. Those guys are doing a good job. If The Mail Archive had been as good in 2002 as it is now, I probably wouldn’t have started Gmane.
I’m open to ideas here. If somebody else wants to take over the concept, I can FedEx you a disk containing the archive (as an NNTP spool). I’ve written a lot of software for Gmane, but it’s all quite site specific and un-documented. And the web interface was written in, like, 2004, so it’s way way way un-Web 2.0-ey and shiny. You’re probably better off implementing this stuff from scratch.
Oh, and along with the spool you’ll get the gmane.conf file which has the mailing list->NNTP mapping.
I can’t really recommend the job, though. It sucks.
[Update: See this comment.]
FWIW, I’ve only really interested in the mail-to-news gateway. And very much in it. Would the burden become acceptable if only this survived?
Me too. I mean it is lovely to be able to point to particular thread on http://thread.gmane.org/ but I care a way more about mail2news gateway. And of course, if worst comes to worst I have my own small one which I can run locally (https://gitlab.com/mcepl/pyg), but fun is in that?
Please, please don’t shut down. GMane. As far as I’m concerned, it’s the mail archive that “just works”. The thread view in the frame-based UI is just nice, because even if I wasn’t subscribed to a list in the past I can read a thread easily. And not just in the heavily broken Mailman web views most MLs still show. I agree that a rewrite of the frame UI with HTML5 might be nice, but I don’t really care. This works.
If you need help, maybe ask for that? Like „would somebody be interested in co-hosting GMane?“ or „can somebody do a rewrite of the UI“, etc. If you need donations to keep the service up, ask for them more openly.
In conclusion I really hope GMane stays up or at this point: comes back up again.
This is really sad. Even though gmane is not web-2.0 it is good at what is important: being fast. It is miles ahead of the stuff companies like Microsoft put out.
Perhaps it could be made to work as a team effort? That way you wouldn’t have to deal with all the mail and the threats on your own.
Gmane is better than the alternative websites thanks to the threaded view.
And the possibility to download mbox is a killer feature as well. With that it’s possible to just reply to messages even when subscribing after the fact.
Granted, the whole gmane website could be improved a little bit by switching from web 1.0 + iframes to web 2.0.
But the alternatives are uglier web 1.0 only websites.
Google Groups could be the best of the alternatives, but it’s still not as good as gmane for the UI, and has quite inferior download (have to download each email).
This is sad to read😦 gmane has been my copilot for years, the UI is perfect, the archive has all the ML I’m interested in.
Thanks for all the years giving this service and if your health is being affected by this, then you certainly know what to do, no one can blame you.
PS: These days people don’t appreciate volunteering work, they demand as if you were Google.
I love gmane! And I think the UI is flipping awesome. None of that fancy unnecessary stuff, just the text ma’am!
I, for one, will miss Gmane if it’s gone. 😦
Gnus on Gmane groups is how I manage to keep track of a load of mailing lists.
This means that what I would miss most, and what Mail Archive and friends won’t supply, is the NNTP gateway. I would also miss Gnus search on Gmane if it’s gone.
I won’t be happy if the web interface disappears, but it won’t affect my daily workflow in the same way as if the NNTP interface disappears.
(I hope you will give this a good nights sleep before you decide, and I hope it will all seem a little less dark when you wake…?)
If it helps, I am willing to pay for a subscription. Gmane is a valuable service and I hope that a new home can be found if you no longer want to maintain it. Thank you for providing such an amazing service for free all this time.
The end of Gmane would be sad but I can’t blame you at all for feeling this way. Instead I would like to thank you and celebrate your years of service. You have made many people’s lives easier and better. Thank you so much.
How much storage and what kind of hardware specs are you using? Gmane is a really good resource. I’m tentatively interested in hosting it.
Noticed today that gmane.org was down, and then read this😦
I have had loads of help of Gmane over the years easily finding threads on MLs such as net-dev and others.
I operate an ISP as a side business and I would love to give back to the community by operating mirrors, donating hosting services, etc. for you, if needed. Drop me an e-mail if you’d like… =)
How many $ to send to Seattle, USA? How big is the data?
Might archive.org be interested in helping? I doubt it, but might be an interesting project.
I have been heavily dependent on gmane smtp/nntp gateway for many years. This is a tremendous community service. Think of all the wasted space and bandwidth eliminated by viewing mailing lists via nntp interface! And it’s far more efficient use of our time to browse via nntp instead of e.g., gmail.
I’ve found answers to questions in gmane web archives many times over the years (so, thanks), but I didn’t even know you ran a smtp/nntp gateway. Neat.
Hi,
I would be very sorry to see Gmane go. It is a very useful service, and I *like* the clear and well-structured HTML interface – nothing fancy, nothing sliding, blinking, soft-scrolling at me, just presenting useful information in a clear way.
We’re over at OpenVPN use gmane URLs in our git repo to document why changes went in (“there is the patch, and the discussion that came with it”) so it’s valuable for open source transparency.
I understand the feeling of “why am I doing this? since 10+ years?” (been building Internet infrastructure for 23 years now…) but I hope it will be possible to support you in some way so the burden is easier – like monetary donations, or … dunno. (I can’t offer *time*, as that is all sunk in non-paid projects on my side already…)
On behalf of all the people who use gmane to access the mailing lists that I manage, I just wanted to say thank you for the hard work you have put in over the years, regardless of your decision (and if the thought of stopping makes you feel happier/better, then please do stop for your own well-being).
Thank you for running Gmane, it’s much appreciated. I can understand how you feel and wish you all the best, which ever way you decide to go from here.
please don’t shut it down. I would be a much worse engineer if it wouldn’t be for Gmane. It is sad enough that so much discussion happens behind walled gardens these days. If Gmane goes, it would be so sad.
Thanks for your hard work on gmane!
Thanks for Gmane, it is awesome and an amazing amount of (mostly) thankless work. DDOS is just random idiot most likely. If you choose to shut down I will be very sad but I will always wear the T shirt and dream about the hidden, civilized parts of the net.
Thank you, Thank You, THANK YOU! Thanks for the 1,000s of hours you have donated to all of us who have benefitted from your hard work.
I heard a podcast a year or so ago by someone at archive.org who talked about archiving large websites and hosting them and how they don’t want sites to just drop off the internet. gmane is definitely one of those websites.
When I hose up a system, I can jump onto gmane via the web
and get help from numerous help/user groups. All that is needed is a web browser, net access and the main url (in memory). Irreplacable, as it’s been a crutch for me for a decade now….
A donation if a new team/organization materializes. Perhaps the best solution is to get gmane back online and then let several sites build small clusters (mesos or openstack) and put a gmane clone up. That way DDos attacks will have a far tougher time defeating the resoure, as others upstream ISP can help filter the DDos and users can have multiple sites to access… Drop me a message, should you need help on the distributed setup.
James
This is really sad.😦 Gmane is one of the services I’ve used nearly every day to read mails from high-traffic lists like linux-kernel and lists I’m not subscribed to. The extent of the archive is also phenomenal, it has helped me out many times when I need to read some old threads. The user interface is fast and keyboard friendly, and a km above the other mail archives when it comes to usability.
I’ve in fact been thinking now and then about this, fearing the day Gmane no longer exists. It is a heritage, in the spirit of archive.org.
I really hope Gmane will continue to exist, and I’ll gladly pay a monthly subscription to keep it running. I’m sure Gmane is valuable enough that a lot of people or organisations will pay a subscription fee to cover the hosting expenses and maybe cover payments to the sysadmin.
Thanks for all you’ve done, Lars, and hopefully I don’t have to say “thanks for all the fish” not quite yet.
Stor og takknemlig hilsen,
Øyvind
I beg you not to end the Gmane web UI. Gmane is absolutely the best web archive for mailing lists. Nothing comes close. I use Gmane every day,
It has unique features that *matter*. Being “web 1.0” and using frames is one of them. Those are a *bonus*. Please read the comments on LWN as well: http://lwn.net/Articles/695695/
Can you perhaps turn this question into one of funding?
I want to emphasize that the “site specific and un-documented” code you wrote, and “the web interface” from “like, 2004” make Gmane Gmane. Those things make Gmane second to none.
Some mailing lists have only been archived by Gmane.
Gmane is irreplaceable.
Lars, I’m Jason Scott with the Internet Archive. I’d like to talk to you about getting a copy of the site and archives on hard drive for permanent archival storage, regardless of the outcome of this phase of your excellent work with the site. I can be reached at jscott@archive.org or through twitter at @textfiles.
I’ve had a couple of hours sleep and managed to eat something, so I’m not quite as despondent as when I wrote the stuff above. And thanks for all the nice comments and feedback, it’s really heartening.
I’ve been talking to ISPs and stuff today, and I’m moving the MTA and the NNTP server to new hosting (and hardware) this weekend. Expect some downtime, but no mail should be lost.
I will not be taking the web servers up again. I’m just too fed up with all that stuff. However, it would be a real shame if all the gazillions of Gmane links out there stopped working. So here’s what I hope somebody could make happen:
I hand over the gmane.org domain to you. You point the MX and the “news” A to my server.
You set up a new web based service based on the Gmane spool, which I will send to you on a disk. (It’s about 2TB, and full of itsy bitsy files.) You’ll also get a feed with new email. You create the web interface the way you like: I think that’s a potentially fun and rewarding project for somebody interested in stuff like that.
All previous permalink.gmane.org (etc.) links continue to work as before, but they’ll look new and spiffy in your new and spiffy web interface. (And you can start small and simple and then expand into awesomeness later. A short downtime would be better than a long downtime.)
You handle any takedown requests from the public and spam detection and stuff.
It would perhaps be preferable if a group of people, or an organisation, were to run this thing, so that it could be a stable resource…
If nobody starts DDoS-ing the MTA or the NNTP server, I could see this being something that could run for many years to come, because it would relieve me of most of the things about Gmane that’s I’m way beyond fed up with.
Any takers?
(Or any other ideas…) Send me an email.
Might you consider posting the current Gmane source code to somewhere like GitHub, and maybe post a subset of the data somewhere? That would let a new set of people start the task of improving/re-building straight away, instead of leaving it all to one individual. It would also mean we wouldn’t have to start entirely from scratch!
The source code is (mostly) on github already. The main bits are the weaverd and weft bits: https://github.com/larsmagne/ But I think starting from scratch is a better idea.
Thanks!
Keeping the NNTP server up would really be appreciated!
I read all my mailing lists (and a few rss feeds) in mutt (via NNTP).
It would be annoying if I had to manually subscribe to all of the individually.
Thanks for all your work over the years.
If the NNTP gateway goes down, I’ll probably have to stop using mailing lists altogether (at least writing to them). There’s just no way I can handle the flood of mail in my inbox. Thank you for providing this invaluable service! I hope it will remain available.
I probably would not miss the web part too much if that one went away, NNTP is the really interesting part.
Hi Lars, just a word to thank you for the superb service you have created and given to so many people over the years. Enjoy your retirement
Thanks SO much for all your work. Gmane has made my life a lot easier. I see two distinct values in Gmane
1) the physical archive of just so many mailing lists; I understand that it totally does not scale and a single person is never going to be able to properly cope with all the legal/takedown crap
2) the COOL threaded web interface with permanent URLs
Wouldn’t it work if you separated these two? Mailing list admins (of live mailing lists) could keep an archive themselves. And you just cache them (be it over NNTP, rsync of a Maildir, whatever). For dead lists or lists with operators unwilling to cooperate, a volunteer would be required to provide the physical archive.
If nobody cares about a mailinglist, you just drop it.
And you get to keep the domain, the precious permalink URLs and the excellent threaded UI – and protect it from whoever wants to Web-2.0-ize it
I’ll do it
Please consider contacting The Internet Archive, I bet they’d love to back up and share a copy of your messages: https://archive.org/about/contact.php