Okay developers, time to have a serious talk. As you are probably already aware, this week React, Babel, and a bunch of other high-profile packages on NPM broke. The reason they broke is rather astounding.
A simple NPM package called left-pad that was a dependency of React, Babel, and other packages. One that, at the time of writing this, has 11 stars on GitHub. The entire package is 11 simple lines that implement a basic left-pad string function. In case those links ever die, here is the entire code of left-pad:
module.exports = leftpad; function leftpad (str, len, ch) { str = String(str); var i = -1; if (!ch && ch !== 0) ch = ' '; len = len - str.length; while (++i < len) { str = ch + str; } return str; }
What concerns me here is that so many packages took on a dependency for a simple left padding string function, rather than taking 2 minutes to write such a basic function themselves.
As a result of learning about the left-pad disaster, I started investigating the NPM ecosystem. Here are some things that I observed:
- There’s a package called isArray that has 880,000 downloads a day, and 18 million downloads in February of 2016. It has 72 dependent NPM packages. Here’s it’s entire 1 line of code:
return toString.call(arr) == '[object Array]';
- There’s a package called is-positive-integer (GitHub) that is 4 lines long and as of yesterday required 3 dependencies to use. The author has since refactored it to require 0 dependencies, but I have to wonder why it wasn’t that way in the first place.
- A fresh install of the Babel package includes 41,000 files
- A blank jspm/npm-based app template now starts with 28,000+ files
All of this leads me to wonder…
Have We Forgotten How To Program?
On what possible plane of existence is this a better solution to past problems? How are hundreds of dependencies and 28,000 files for a blank project template anything but overly complicated and insane?
I get the impression that the NPM ecosystem participants have created a fetish for micro-packages. Rather than write any functions or code, it seems that they prefer to depend on something that someone else has written. It feels to me as if the entire job of an NPM-participating developer is writing the smallest amount of code possible to string existing library calls together in order to create something new that functions uniquely for their personal or business need.
Functions Are Not Packages
Functions are too small to make into a package and dependency. Pure functions don’t have cohesion; they are random snippets of code and nothing more. Who really wants a “cosine” dependency? We’d all really like a “trigonometry” dependency instead which encompasses many “tricky” functions that we don’t want to have to write ourselves. This is much more akin to how .NET and other frameworks create a “core” library of basic functionality. Such a library is vetted by the creators of the language and pretty much guaranteed to be correct and bug-free.
Third Party Problems
There’s absolutely no guarantee that what someone else has written is correct, or even works well. Even if correct, is it the most optimal solution possible? At least when you write the code yourself, you can easily modify it to fix bugs and improve its efficiency. Not that there should be many bugs in 1 line functions.
Second, even if the package’s logic is correct, I can’t help but be amazed by the fact that developers are taking on dependencies for single line functions that they should be able to write with their eyes closed. In my opinion, if you cannot write a left-pad, is-positive-integer, or isArray function in 5 minutes flat (including the time you spend Googling), then you don’t actually know how to code. Hell, any of these would make a great code screening interview question to determine whether or not a candidate can code.
Finally, stringing APIs together and calling it programming doesn’t make it programming. It’s some crazy form of dependency hacking that involves the cloud, over-engineering things, and complexity far beyond what’s actually needed.
What’s worse is that if any of your code (or the 3rd party library code) has a bug or breaks, you won’t know how to debug or fix it if you don’t know how to program.
Strive For Few Dependencies
Every package that you use adds yet another dependency to your project. Dependencies, by their very name, are things you need in order for your code to function. The more dependencies you take on, the more points of failure you have. Not to mention the more chance for error: have you vetted any of the programmers who have written these functions that you depend on daily?
Take on a dependency for any complex functionality that would take a lot of time, money, and/or debugging to write yourself. Things like a database access layer (ORM) or caching client should be dependencies because they’re complicated and the risk of the dependency is well worth the savings and efficiency.
But, for the love of all that is programming, write your own bloody basic programming functions. Taking on dependencies for these one-liners is just nuts. Don’t believe me? Just ask the React team how well their week has been going, and whether they wish they had written those 11 lines for left-padding a string themselves.
Follow David Haney on Twitter at @haneycodes
Ofcourse not, it is about hiding complexity. Read more about here https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328
I think you’re pointing to the right symptom and drawing the wrong conclusion.
These packages proliferate because JavaScript’s standard library is so woefully inadequate. It’s ridiculous that anyone has to write their own left-pad function in the first place. The problem isn’t that people “don’t know how to code”, it’s that they’re operating in an environment where basic functionality has to be implemented by end-users (i.e. developers), and where it’s — perhaps reasonably — standard operating procedure to import someone else’s solution rather than write your own.
And it’s not just JS, there are plenty of examples in other ecosystems. In Java there’s no reason to write your own StringUtils.isNullOrEmpty() when you can just import Apache Commons or Google Guava. That said, it’s embarrassing that such a commonly-needed thing isn’t in the JDK itself.
Spot on, David.
This is the one thing that really concerns me about front-end development as a whole. We have become so dependent on tooling and package managers, we don’t realise how weak the foundations of the ecosystem actually are. The entire front-end development ecosystem is one massive house of cards, pull out one of the cards and it all falls apart.
I am honestly surprised that something like this did not happen sooner to be honest. If developers can’t write their own left padding implementations, maybe the end of mankind is going to happen sooner rather than later. You point out there are packages for isArray and other numerous basic tasks that require very little code to implement. We’ve gone package mad. There are thousands of packages with just a few lines of code, who is using them nobody knows.
The flaw with Npm is that it encourages this micro package approach so much, you look at packages like Babel and it does your head in tracing all of the dependencies. One dependency sometimes has upwards of 30 or 40 dependencies, but guess what, most likely each of those dependencies has their own dependencies as well. It can really get quite perplexing, once you go down the rabbit hole you never truly reach the bottom when it comes to Npm packages. One package can possibly have hundreds of dependencies, no wonder Npm is slow.
There is a meme here.
“Yo dawg, I heard you like Npm packages, so we put an Npm package in your Npm package, so you can install while you install.”
You make a good point but it’s a bit exagerrated and I’ll informed. left-pad was not a direct dependency of React, or Babel. left-pad has a handful of dependents, but one in particular, line-numbers, which offers more than a one-liner of functionality, was a dependency of several high profile projects.
One advantage of micro packages is that if a certain piece of code is used in different packages, there is only one implementation of it sent to the client, instead every module re-implementing the same function thus increasing the client code footprint. That said, the price we pay for this is high. What if a hacker takes control of the NPM account responsible for one of these one liners? The possibilities are endless.
Hopefully this un-publishing incident will serve as a wake up call.