Busy day here, I have a day job, and it's used 100% of my cycles today. I'll summarise the current technical state of affairs with the fruit site however they stand, but tomorrow rather than today. Breaks are important to take.

Real quick before going back to work: (1) Hostslick back in rotation, (2) Polish host HosTeam (AS51290) is new. EU nationals can write to local DPA under GDPR for both (3) VSYS still serving. (4) NICE IT was being used for IPv6 yesterday (but was broken anyways); out for now.

(5) mail dot lolcow dot llc is serving off Terrahost and using Epik NS, but crucially, the domain is registered with Google Domains (Epik appears to not be a dot-LLC registrar); registrar-abuse@google.com should be a pretty easy avenue to deal with that.

Overall though, the prognosis is that the current set of Hostslick, HosTeam, and VSYS are going to be able to keep the site up and stable (e.g. as it's been past 40 hours) until at least one or two of them pull out. I always share honestly, even when I have bad news to share.

I would not expect any dramatic changes for the next week; it is going to be a protracted regulatory siege for the foreseeable future, with limited value to continued blow by blow analysis of what I can determine is happening.

Hosteam has not been seen in rotation for 21 hours according to my monitoring tools; however, it does appear to be up and serving the KF SSL certificate on port 443.

Incognet appears to be serving a majority of the traffic currently, and far more VPSes (or at least loadbalancing IPs) appear to be deployed there than the singular hosts with the other providers.

I have interesting news to share for IncogNET. It was previously serving majority of traffic _but_ had a significant misconfiguration. It was removed from rotation over a day ago. And now... 23.137.250.0/23 is host unreachable, their upstream Worldstream is last hop. Fascinating

Incognet's US points of presence behind US-EAST dot AS210630 dot NET are serving, but 109.236.95.231 (Worldstream) is denying all further traffic to that /23 which is Incognet's Dutch presence. I suspect Incognet are not in Worldstream's good graces right now for proxying KF.

Still host unreachable... so not a fluke/temporary outage.

yup, host unreachable multiple days. folks, this is why proxying KiwiFarms in the EU is bad for your business and will get you cut off by your upstreams.

wait holy shit WHAT 4 hours ago the bot check is removed. it's just serving... in the clear on 103.114.191.55?

surely this can't be right, but all of my tooling is telling me that the network of frontend proxies has been dismantled and it's right back directly on the 1776 hosting netblock with no DDoS protection of any kind?

... OH. OH. OH. bgp.he.net/AS397702 That explains it. He finally got his direct link to Zayo online, with Zayo's paid volumetric DDoS protection services. Well then, there's only one upstream to go after now.

And, now he's directly a paying customer of Zayo, so the argument of "we just provide transit/peering to Fiberhub which provides transit/peering to 1776" doesn't fly any more. The site is a walking, talking violation of t.co/7iifqcSgSc and now is a DIRECT CUSTOMER.

abuse@zayo.com abuse@zayo.com abuse@zayo.com abuse@zayo.com abuse@zayo.com t.co/XMEDyCQsU9 (I do not know for sure whether Zayo passes reports directly to their clients, so use a throwaway if you're paranoid)

also that's really fucking cute, that Josh thinks his only threat model is volumetric L3 DDoS attacks. this new architecture is hilariously vulnerable to L7 attacks now.

this means all the other unsavoury shit is back online as well, including the .cc domain (on .61 in the 1776 Solutions netblock) and action zealandia (shielded by cloudflare, but has a backend in the 1776 Solutions netblock and went down along with KF earlier)

so in addition to reporting to Zayo for hosting KF, you can report 1776 Solutions to Zayo for directly facilitating/shielding AZ.

This also raises interesting questions about how Zayo is being paid. They don't exactly accept cryptocurrency payments so...

Also, _iff_ you are willing to self-doxx under DMCA: If any of your media have been reproduced in their entirety and are being served from no-cookie dot kiwifarms dot net or from www dot kiwifarms dot net, you can contact dmca@zayo.com regarding copyvio. t.co/enpQOuqz55

Unlike other forms of abuse, Zayo _is_ obligated to swiftly act with regard to DMCA and distribution of copyrighted material through its network if a direct client is involved; however, expect your full listed name, address (use a PO box). to be passed to 1776 Hosting / Josh.

other perhaps useful AUP infringing content on 103.114.191.x .11: madattheinternet dot com .55: kiwifarms dot net (et al) .60: riot dot kiwifarms dot net .61: kiwifarms dot cc .63: git dot kiwifarms dot net

something just happened! bgp.tools/as/397702?sh... either someone messed up their routing announcements, or just got dropped! run a traceroute folks, tell me where it cuts out for you :)

oh I see, that's why all the requests are flowing through Telia. Telia is the only one announcing the routes, because of caching/delays, but it's no longer being announced as reachable through Zayo, except Telia still having the old records.

bgp.tools/prefix/103.1... -- this should look like it did earlier here, with 16+ other Tier 1 ISPs announcing the route via Zayo. But it's just lonely Telia now: twitter.com/DropKiwifarm...

so basically all the traffic is being concentrated on Telia net, then is being sent to the edge router between Telia and Zayo, where it's then rejected. explains why it routes to ash-b2-link.ip.twelve99.net and then goes host-unreachable, because it hits Zayo and Zayo says NOPE!

aaaand it's gone. "This prefix is not visible in the DFZ right now" bgp.tools/prefix/103.1...

Reader correction: AS1299 is no longer Telia, it's Arelion, spinoff/sale/rebrand. anyways, they're out of the global routing table. It's over.

here's the state of affairs: 1776 Hosting is in FiberHub/VegasNAP datacentres. FH/VN have 3 upstreams: Path, Zayo, and HE. Path won't carry 1776 traffic and hasn't for years. Zayo just decided not to carry it either. So it's down to just HE, and that's if Josh figures out how.

It's so down you can't even get to it over Tor. Because the origin/backend servers have been yeeted off the internet, because Joshua Moon got too cocky, bet it all on a direct "DDoS proof" link with Zayo, and lost when they were informed exactly what he was doing / who he is.

Josh's only hope is to get Hurricane Electric to carry him, if they don't, then it literally will take FiberHub and/or 1776 paying a new provider (who will put up with them) hundreds of thousands of dollars to backhoe new fiber into the ground to a peering point. He's fucked.

Ahahaha.

Update to the above regarding @IncogNetLLC, whom I owe an apology to. Instead of having their gateway respond, if a host is not provisioned (due to customer withdrawing), the upstream Worldnet responds unreachable, which led to my incorrect conclusion.

Some new developments - it appears that Joshua Moon is abandoning the idea of having 1776 Solutions LLC / Flow Chemical Pty Ltd netblocks be announced from VegasNAP/FiberHub LLC in Las Vegas; instead, he's pursuing another strategy.

He's going all in with VSYS. How do I know this? Well, just a few minutes ago, 103.114.191.0/24 began to announce via AS397702, but with upstream AS29320 (Bimeya Ltd) which only announces one netblock, one of VSYS's: 128.0.104.0/24. bgp.tools/prefix/103.1...