Just a callout that Fastly provides free bot detection, CDN, and other security services for FOSS projects, and has been for 10+ years https://www.fastly.com/fast-forward (disclaimer, I work for Fastly and help with this program)
Without going into too much detail, this tracks with the trends in inquiries we're getting from new programs and existing members. A few years ago, the requests were almost exclusively related to performance, uptime, implementing OWASP rules in a WAF, or more generic volumetric impact. Now, AI scraping is increasingly something that FOSS orgs come to us for help with.
I've been running into bot detection on at least five different websites in the past two months (not even including captcha walls)
Not sure what to tell you but I surely feel quite human
Three of the pages told me to contact customer support and the other two were a hard and useless block wall. Only from Codeberg did I get a useful response, the other two customer supports were the typical "have you tried clearing your cookies" and restart the router advice — which is counterproductive because cookie tracking is often what lets one pass. Support is not prepared to deal with this, which means I can't shop at the stores that have blocking algorithms erroneously going off. I also don't think any normal person would ever contact support, I only do it to help them realise there's a problem and they're blocking legitimate people from using the internet normally
It's not like they say, but it's at least three different implementations and I don't think any were cloudflare because I've been running into those pages for years and they've got captchas (functional or not). One of them was Akamai I think indeed
Yeah, I definitely don't want to pivot this thread into a product pitch, as the important thing is helping the open-source projects, but we can work with the maintainers to tune the systems to be as strict/lax as preferred. I'm sure the other services can too, to be fair.
The underlying issue is that many sites aren't going to get feedback from the real people they've blocked, so their operators won't actually know that tuning is required (also, the more strict the system, the higher percentage of requests will be marked as bots, which might lead an operator to want things to be even more strict...)
I will say -- a higher-end bot detection service should provide paper trails on the block actions they take (this may not be available for freemium tiers, depending on the vendor).
But to your point, the real kicker is the "many sites aren't going to get feedback from the real people they've blocked" since those tools inherently decided that the traffic was not human. You start getting into Westworld "doesn't look like anything to me" territory.
I'm not into westworld so can't speak to the latter paragraph, but as for "high-end" vendors' paper trail: how do log files help uncover false blocks? Any vendor will be able to look up these request IDs printed on the blocking page, but how does it help?
You don't know if each entry in the log is a real customer until they buy products proportional to some fraction of their page load rate, or real people until they submit useful content or whatever your site is about. Many people just read information without contributing to the site itself and that's okay, too. A list of blocked systems won't help; I run a server myself, I see the legit-looking user agent strings doing hundreds of thousands of requests, crawling past every page in sequence, but if there wasn't this inhuman request pattern and I just saw this user agent and IP address and other metadata among a list of blocked access attempts, I'd have no clue if the ban is legit or not
With these protection services, you can't know how much frustration is hiding in that paper trail, so I'm not blocking anyone from my sites; I'm making the system stand up to crawling. You have to do that regardless for search engines and traffic spikes like from HN
Oh my, a Dutch film that actually sounds good?! I get to watch a movie that's originally in my native language for perhaps the second time in my life, thanks for linking this :D
Edit: and it's on YouTube in full! Was wondering which streaming service I'd have to buy for this niche genre of Dutch sci-fi but that makes life easy: https://www.youtube.com/watch?v=4VrLQXR7mKU
Final update: well, that was certainly special. Favorite moment was 10:26–10:36 ^^. Don't think that comes fully across in the baked-in subtitles in English though. Overall it could have been an episode of Dark Mirror, just shorter. Thanks again for the tip :)
I have to assume the Dutch movie industry just isn't too big.
I guess it's a side effect of America's media, but when I went to Europe including the Netherlands almost everyone spoke English at an almost native level.
It almost felt like playing a video game where there is an immersive mode you can just turn off if it gets too difficult ( subtitles in English at all public facilities).
Salesforce development. There's a real learning curve, but Salesforce provides really good teaching academies and certification processes.
For an enterprise company, there is no data more important or highly valued than the CRM. Also at a mature company, sale motions are already dialed in, and comp plans typically only change 1-2 times a year (more than one change a year is bad for org morale). Large overhauls can be a pain, but they're extremely infrequent and ideally scheduled well in advance.
Once the system is operating and the daily, weekly, monthly, and quarterly cadence reports are dialed in, you just need to make sure the system continues to run, and deal with any special odd projects or reporting requirements from the sales execs you're working with.
Your sale exec partners have every incentive to get their processes right the first time, and rarely change them -- change disrupts any team (not just engineers), only with sales teams, that change results in tightly trackable lost revenue.
Its not "zero work", but it fits the "2 hours of active work, then chill" requirement. You'll have a busier end of quarter, but the first 2-3 weeks of the quarter will either be crickets or greenfield work.
Edit: this role can legitimately be fun too. You're at the nerve center for how the company actually makes money, and you get to have a cross sectional view of all other functions as a result. You'll see "how the sausage is made" but you only have to report on it and build the tracking systems -- you're not on the hook for performance or output. Also, sales people know when they have a good CRM process and are some of the most grateful folks in the world to those who help them. If you do a good job for your users, you'll be getting public shoutouts every day.
One answer - It's much, much harder to build and operate a global network infrastructure that it might seem. It's also even harder to invent some sort of "killer feature" or other genuine innovation on the experience. You're likely not using Cloudflare simply for commoditized pipes alone, but for other features or designed experiences in their offering.
A second answer - there are a bunch of bottom-barrel commoditized pipe services. You likely haven't heard of them because they're so generic. They've existed before Cloudflare, and more will be created in the future https://www.citrix.com/products/citrix-intelligent-traffic-m...
What you're proposing is testing two different variables in the same experiment. To dial in nutrition research, it's more traditional to control kcal count and test different nutritional quality, or control quality and vary kcal content.
I'm an amateur olympic-style weightlifter with a similar schedule to yours, and after trying every trick diet in the book I've come to the same conclusion. The only real advantage I can see from diets like paleo, keto, etc. is that they can make the average person feel satisfied with less calories. Also, processed high carb foods tend to be surprisingly high calorie bombs. But, it all comes down to cals in vs cals expended.
The discussion around _quality_ of calories, the mix of macronutrients for performance, etc is an entire separate can of worms.
This rings incredibly true to me and I'm glad the idea is spreading. My diet philosophy pitch to people is this:
(Cribbed from my post above:)
THE BEST DIET to lose weight is about psychology 80% and nutrition 20%. Most of the value you're going to get by keeping your weekly calories low. You can try variations of macronutrients, meal timing and frequency, voodoo, etc. but the best one is the one you stick with, and can stick with long enough to reach your goals.
Maybe you'll need to think about essential minerals and nutrients if you're trying something really really extreme. But with even minimal variety you should be good.
Interesting to get your perspective - when people ask me for advice on a specific diet I view them all as a way to help them eat less calories overall while not requiring quite so much shear will.
On your latter point my wife opened that can of worms in the run up to her first competiton and it got complicated!
I work for an competitor to an AWS product. We've grown rapidly over the past ~7 years in a generally competitive (lots of startups, some really old tough incumbent companies). Without revealing too much on our end, here's some lessons learned:
* AWS is bad at customer service, even for their large or premium customers. If you position yourself, and _seriously_ invest in making your company's culture rooted in exceptional customer service, that's a foothold.
* Don't compete on price. This is hard for most tech startups, as pricing is a very difficult thing to do properly, but resist the urge to drop price to compete. You'll never have the scale, the supply chain masterminds, or the financial modeling to compete with AWS on price, so position yourself as a premium or luxury offering and don't be afraid to price accordingly. If you do the first step properly (a deeply rooted culture of service) you'll be able to justify the price.
* AWS has great uptime, but often the actual operating performance of their service isn't that great, especially when you push the products beyond the 80% use case. They know that for the majority of their customer base, best-in-class performance isn't actually business critical (despite how flashy it sounds). However, there is absolutely a market for people who truly need best in class performance, or product flexibility, or some other best-in-class trait (latency, interaction design, etc.). Find who these people are, and optimize for that ruthlessly. This focus, in combination with the culture of exceptional service and positioning your brand as a premium provider, puts you into a completely different market space than AWS.
Really? I’ve had better customer experience with AWS than Azure or GCE... Even their SDK devs are responsive, I raised a bug for the SDK for their queues and went to bed, woke up and they had published a new SDK with the fix after 7 hours of raising the ticket...
There is a lot to dislike about AWS but from my experience customer server isn’t one of them...
Azure had me on the phone within minutes of filing critical issues. Their engineering department provided updates all night.
Amazon issue resolution could be summarized as "won't fix" or "someday/maybe". Their API inconsistencies might be considered a poor developer experience [0].
It's great you like AWS. I do too for some use cases! I disagree their customer service is "better" though.
To be honest, all of this depends heavily on the support level and company size as well.
While we had gold support at GCP (initial free offering), we were using one of their beta APIs that wasn't working properly — and within minutes we were talking to one of their Zurich SREs who was top notch and filed an issue that was resolved within a day.
When we scaled down to developer support (and even with a 5 digit infrastructure spend per month), we got delegated to an outsourced first-level support team somewhere in Asia with a complex problem and couldn't get through to anyone who didn't ask the equivalent of "have you tried turning it on and off again?", even after several tries. Eventually we gave up on using IAP completely due to severe bugs. Guess we'll upgrade again soon despite the cost, since the difference was night and day.
Just as a finishing line, the support experience at AWS was always equally mediocre, even in business support.
Ah their API is far more consistent than Azure. Azure has the worst API of all. But their services are much faster than AWS.
Aws is much more consistent if you know how to use one API then learning a new feature is simple because it’s more or less the same thing (only basing this on .NET and JavaScript SDKs)
Been on Aws for... 6 years now. I was an Azure insider for like 5 years till they discontinued the program and use it for contract work.
GCE I don’t have a lot of experience in but just try to Ensure I know what’s going on incase we need to switch one day or I move to new job.
We had an instance crash (first time), once we were bestowed the honor of submitting a ticket after an hours of downtime (to which the clock only started an hour after the instance went offline, we were able to solve the issue ourselves, and closed the ticket an hour after submitting it... to which a couple of days later we received a response asking about the issue.
Ah I had that experience on Azure. With aws we have paid support so when we have issues and raise critical tickets they respond in ~20m but azure. When they were still doing sql server as a service I lost access to my database. Support told me there was no problem. The azure insiders forum said others were having issues. The support wouldn’t even acknowledge I couldn’t access the database. 2 days later I finally got access again randomly. I never got a response to why this happened. I’m just Glad this was a non production application for some contract work but they ended up wanting to go aws over the experience.
I Guess at the end of the day we all experience different levels of support from all these providers.
I got some support from Azure a few months ago asking about DTU spikes, more of a shot in the dark than hoping for a good answer, but got an extremely good, knowledgeable woman go through it with me within a day. We only spend a few thousand a year at the moment.
I may hate their admin interface, but their support was top notch in my experience.
> better customer experience with AWS than Azure or GCE
Yeah, they're all awful. A comparison might show one is better than another, but it's all a wash when you look at actually good customer service.
A good host has a phone number, you ring it, somebody answers and then they fix your problem, without pinging you around a call centre. You're in and out within 20 minutes.
For AWS (and their class) you submit a ticket and in 4-48 hours, you twiddle your thumbs while the cheapest labour available to Amazon wakes up on the other side of the planet to investigate your problem (also known as walking you through a script).
AWS-sized hosts have advantages but I put a lot of weight in scaling things back to the RackSpace, Linode and Hetzner size operations. They put so much more effort into their human interaction.
I dunno, that sounds pretty good honestly. The only info I have on AWS's service is testimonials, so I don't have anything concrete.
The industry we obsessively study/studied was hospitality, where service is literally the differentiator between the "pretty good" and the "absolute best in the world," and we've been pretty meticulous about iterating and building upon our service ecosystem (it's way beyond just one team at this point) like one might do with a software product. Most of our customers, from the small ones in the early days to the huge internet juggernauts we've had the privilege to work with use us as the standard to which all other vendors are measured, sometimes quite literally in a very odd way. So, I don't know if we got lucky with some stinker incumbents, or we stumbled on to something completely different, but we've got a good system in place that's built a pretty solid competitive moat.
I guess if folks on the thread are still reading and interested in replicating this at their startups, my advice is: study elite hospitality and restaurants, not tech. I'm very rarely impressed with the service ecosystem in the tech world, though it happens, but I can learn something massive every day from hospitality.
True their support has been quite helpful and pretty amazing. A dev reached out to me days after the issue was resolved for feedback and advice, and it wasn't a cookie cutter template. He actually wrote a 300 word letter to me, including asking for my feedback on a new feature they were working on.
I couldn't believe a company as big as AWS would seek out opinion of some guy just posting on their forum.
Slack is going to have to do a better job of educating new users as they grow. Assuming that these large, traditional enterprises will know how to use the tool without some sort of guardrails or deeper education will be problematic and likely hurt long term adoption.
As options, integrations, and bots in Slack get more and more customizable, in many cases it will end up being the responsibility of individual organizations to provide some sort of training for their users. SOPs and conventions will be unique to each organization.
It kinda gets to be a situation like we have with some of the huge CRM and asset management systems, where the official documentation and training is really more focused on setup and administration, and the specific implementation is really up to the org.
Can confirm, working at a major bank which "uses slack". Our channels are completely dead, everyone still emails each other stupid crap like "Looks good!".
Give it a few more years, big institutions are still full of people who are just getting the hang of email.
Make no mistake, the lifelong servitude of Mandela and Gandhi is so far beyond the work of a businessman, however groundbreaking and amazing Musk is, that it's a societal disgrace to humanity to reference them as equals. I write this as someone who respects and admires Musk as much as anyone in this forum, if not more.
Without going into too much detail, this tracks with the trends in inquiries we're getting from new programs and existing members. A few years ago, the requests were almost exclusively related to performance, uptime, implementing OWASP rules in a WAF, or more generic volumetric impact. Now, AI scraping is increasingly something that FOSS orgs come to us for help with.
reply