npm is the Beating Heart of the JavaScript Ecosystem

Did you npm install today? I know I did.

npm is now an inseparable part of most JavaScript developers' day to day development cycle.

I find myself using npm more and more every year. With the help of projects such as webpack, we’re able to use npm in any JavaScript project. npm is no longer just a “Node thing.”

It’s a “JavaScript thing.”

And up until recently npm was universally loved. But a recent event challenged that unconditional love.

A cloud in the sky

On March 22nd, 2016, one developer named Azer decided to unpublish all of his packages from the npm registry. More than 250 packages in total.

No one would have cared or noticed unless one of the unpublished packages happened to be a dependency of a dependency of a little project called Babel.

Yes, that Babel.

If you’ve missed the drama that was npmGate, The Register has you covered.

I would like to focus on a few things a lot of people including myself, realized that day regarding their beloved npm CLI tool.

Every one of the following points was common knowledge to some, since all of which are well documented and publicly available on the npm website.

However most of us didn’t care up until npmGate hit:

  1. npm, inc is a privately held company. It’s the company that hosts and maintains the npm CLI tool.
  2. npm has a package name dispute policy. npm package names are global, whoever publishes a package under a certain name first, captures it. npm, inc. reserves the right to manage npm’s global namespace according to its policies.
  3. npm allowed users to unpublish packages at will, regardless of how many other packages depended on them. This was the first thing npm fixed shortly after npmGate took place.
  4. Once Person A completely unpublishes a package, Person B can publish a completely different package with the same name. The only requirement is that the new author will advance the version by at least one patch version (0.0.1 -> 0.0.2). This was also shortly addressed after npmGate.

What followed all over the web was a long discussion about the various pros/cons and risks/benefits of each one of the points above. And as you might have guessed, the internet has a lot of opinions.

But before we get into the present and the future of npm, let’s talk a little bit about the past.

npm, the little package manager that could

Like a lot of great things in this world, npm started out as a side project.

No one asked for it. But someone decided to build it anyway. And we’re all glad they did.

That someone’s name is Isaac Z. Schlueter.

Isaac was just a developer who got into Node.js at a very early stage and went all in.

He became a Node.js core contributor early on, and decided node needed a decent package manger.

“…I wrote npm because I'd seen module packaging done terribly, and done brilliantly, and I wanted to make sure that Node didn't end up with something like Pear. Even CPAN, wonderful and impressive as it is, had some problems that I thought we could learn from.” - March 26, 2013, Isaac Z Schlueter.

When Ryan Dahl the creator of Node was hired by Joyent, he personally asked Isaac to join him. There, the two continued to work on Node, and Isaac continued to develop npm as well.

When Ryan eventually left Node.js and Joyent, Isaac was named the project’s new lead. He faithfully carried out this duty between 2012 and 2014.

Isaac is currently still the #3 all time top contributor to Node.js. Surpassed only by Ryan Dahl and Ben Noordhuis.

When Isaac left Joyent on Jan 2014, it was to co-found a startup together with his long time friend Laurie Voss.

I’m pretty sure you can guess which company they ended up founding.

How npm, inc. Came to Be

I don’t have the “inside scope”. Like you, I’m an outsider looking in.

But I was able to find something that clued me in. Something that inspired me.

I found an old blog post from mid 2011, 2.5 years before npm, inc. was founded. In his post, Isaac is detailing the future he sees for Node.js and himself, with uncanny accuracy.

The context for this post was Isaac’s response to recruiters who were trying to recruit him for various companies which he refers to here as “XYZCorp”:

”I am working on Node.js and npm. This is the technology that has already displaced Python and Ruby as the hot new high-level language platform of choice; within 5 or 10 years it’ll join the likes of .NET, Perl, PHP, Java as the go-to platform for enterprise development (ie, “serious business”) and new college grads. But there is so much more work to be done!

…When and if I leave Joyent, it will be to do some other work focused on npm and Node.js, probably as part of my own company. The only way that XYZCorp could ever recruit me is if they recruited me to do that.

…Altruistically, I want what’s best for the Node community, because I feel connected to it. Selfishly, I want to be one of the names that is remembered as part of a major revolution in software development. If you can show that these two goals are better served at XYZCorp than at Joyent, then let’s talk.”

August 31st, 2011, Isaac Z. Schlueter on responding to recruiters.

When Isaac wrote this over four and a half years ago, Node.js was somewhere between versions 0.4.0 to 0.5.0.

I remember when I was working for a .Net company at the time. I was just telling everyone about this “Cool new schtick” that allowed you to run JavaScript on the server.

No one there has even heard of Node at that point.

But Isaac was already seeing the day companies like PayPal would move to Node.js.

I’m going to be overly simplistic here for a minute. I’m sure Isaac had plenty of ups and downs from that blog post until now. But here is my take:

I can’t help but wonder if everything that happened from that blog post up until Issac founded npm, inc. was just an implementation detail.

When that level of determination and clear focus meets the right set of skills at the right time, amazing things happen.

npm and npm, inc. Present Day

Now that we have some context about npm and npm, inc. I’d like to talk about some of the concerns and questions I’ve heard floating around post npmGate.

Since I can’t point out exactly when and where I’ve heard these, let’s just say I’m dealing with a split personality disorder.

Just to be clear, I’m in no way affiliated with npm, inc. and the following views are mine and mine alone.

Will npm, inc. interests as a private company eventually hurt or neglect npm the open source project?

In this specific case, I think the answer is 100% NO.

Corporate ownership of an open source project is always a double edged sword. It’s just a matter of which end is sharper. And in npm’s case it’s clearly the right one.

Some corporate sponsorships do more damage than good, while others allocate significant resources which help support the project and take it to new heights.

In npm’s case, npm IS npm, inc. It’s the foundation of the company and everything it does. Their futures are completely interwound.

That’s enough motivation for any company to take their open source commitment seriously.

And as I tried to illustrate, the founders know a thing or two about managing and contributing to open source projects.

Allowing uncontrolled unpublish of modules was dumb

I think everyone agrees on that one. It was the first thing npm addressed post npmGate.

npm currently only allows to unpublish packages which no other packages depend on. Effectively killing the unpublish feature for any meaningful package.

For more details, here is npm’s explanation.

Will npm transfer my package name due to trademark issues?

I doubt it. As npm clearly stated, the decision which ignited npmGate to transfer ownership of the kik package from Azer to the Kik chat app company was purely based on npm’s naming dispute policy.

This wasn’t a legal issue according to npm, and I believe them. If nothing else only because Kik had such a laughably bad case.

And I’m pretty sure the whole trademark thing backfired for Kik anyway, now that there are lovely packages such as “kik-rtards” in the registry.

Will npm transfer my package name for any other reason?

This is where things get a little tricky. Here is literally the tl;dr from the npm name dispute resolution document on the date of this publishing:

  1. Get the author email with npm owner ls <pkgname>
  2. Email the author, CC
  3. After a few weeks, if there's no resolution, we'll sort it out. Don't squat on package names. Publish code or move out of the way.

Make of it what you will.

I personally hope that npm, inc. will never forcefully transfer a package like they did with Kik and Azer ever again.

But they reserve the power to do so if they choose.

You can check out the dispute resolution document for yourself in case it updates or you want more details.

npm is not safe!!!!

There are 2 reasons why people might think npm isn’t safe. Let’s tackle both:

1. A malicious package can replace an unpublished one

npm addressed this issue. When someone unpublishes a package, a replacement “security placeholder” package is put in its place.

This is a little cumbersome and feels more like a patch than a fix. But considering npm wishes to maintain its global namespace and allow unpublishing, I’m not sure there’s a better alternative.

2. Someone can publish an evil worm that will self-install in all npm packages!!@#@!#!#!

Theoretically possible, but highly unlikely.

I’m referring to this supposed vulnerability.

Running someone’s else’s code is an inherently dangerous thing. Using a package manager or otherwise. Open source mitigates this by having everyone see everyone else’s code.

Malicious packages could be potentially introduced to every package manager. But practically they don’t really exist.

Maybe there’s some psychological reason that explains why this doesn’t happen more often. It might be that developers don’t want to screw over other developers. Or perhaps developers are simply afraid of the retribution of their peers.

Bottom line, npm is just as safe as any other package manager out there at this point.

JavaScript is growing, and npm will grow with it.

To sum things up, npm is the heart of the JavaScript ecosystem. It makes sure our modules end up where they’re supposed to. Pumping them out as needed.

As JavaScript continues to grow, npm will grow with it. Supporting large enterprises, small startups, and tiny side projects. Being used for anything from web apps to robotics or home entertainment systems.

And although npm might occasionally skip a beat, our JavaScript ecosystem is a much better place because of it.

Learn Something New Everyday,
Connect With The Best Developers!

Sign Up Now!

& 500k+ others use Hashnode actively.

Alan Plum's photo

The problem with npm Inc is not that they're creating a bad product -- they're not. Sure, allowing unpublishing this easily was a stupid mistake and install scripts can do bad things and prepublish is broken and the fact they're essentially storing graph data in CouchDB is extremely cringy, but npm-cli is a great open source project and the npm registry is sufficiently reliable.

The problem is that npm Inc has made it clear that they wish to act as a private company but also not give up their special status in the Node ecosystem. This is akin to the situation that caused the creation of the Node Foundation and the transfer of the Node project from Joyent (a private company) to the Foundation.

Isaac and other major representatives of npm Inc have a well-established political agenda (which is fine for a private organisation) and they want to use both npm and the Node project to further their political goals. This won't work for Node because the Node Foundation is sufficiently large and apolitical to prevent ideologies from derailing the project's open source nature but as npm is fully controlled by npm Inc there are no such guarantees for the npm project or registry.

The problem isn't npm Inc's political agenda (whether you personally find it agreeable or not), it's that Isaac has been talking about utilizing the npm registry for it. Specifically, a couple of months ago he made a series of tweets sincerely proposing the idea that the npm registry should issue IP bans to individuals who misbehave outside of the scope of npm or npm Inc itself (e.g. for saying something horrible on Twitter), even when the IP is actually that of their employer -- who if they didn't want their company to be banned should simply fire the miscreant. Those tweets were of course deleted after the exchange sparked a lot of outrage but to this day neither Isaac nor npm Inc has made any statement indicating that they won't pursue this idea further.

The view npm Inc holds of open source is that "everything is political" and therefore open source itself must take a political stance or else it supports wrongdoing by default. This runs contrary to the idea of Open Source itself and is on the opposite end of the spectrum as Free Software (which sees software itself as unownable and would consider any restriction -- even to persons you disagree with -- inherently immoral).

Even if we ignore the danger of politicization of the Node project via the npm project (which, to remind you, is distributed as part of the Node project by the Node Foundation) there is still the problem that npm Inc is not accountable to the Node project, especially not with regards to how the public registry is governed.

The kik fiasco showed that npm Inc is willing to take away overloaded package names from active maintainers and give them to trademark holders even when

  • no trademark complaint has been made towards npm Inc
  • no legal threat towards npm Inc exists
  • the maintainer disputes the trademark's relevance and is willing to risk getting sued
  • the trademark owner has no obligation to sue for trademark infringement
  • the trademark owner has tried to bully the maintainer
  • the project is actively being developed and published
  • the maintainer has built a reputation within the community
  • the trademark owner already uses an alternative package name
  • npm name spaces are a thing now and could have been used to signify authenticity if this was really about misleading users to begin with

Despite all these reasons (and more), npm Inc decided at their own discretion (and without legal consultation because there was no threat) in this specific case that kik interactive was the "more apropriate" package owner after a very short e-mail exchange. By siding with a corporation (that is acting as a bad open source citizen) over the community they have made it clear where their loyalties lie.

In fact, throughout the entire fiasco they have lied about what happened: (npm Inc co-founder) Laurie claimed their lawyers had advised them to comply with a legal request; subsequently various complaints were shot down with jokes about the people complaining not being legal experts (when in fact kik made no legal claim whatsoever and no lawyers were involved at any point). Eventually Ashley admitted that npm hadn't actually been sued and the decision was made internally (by Isaac, as the e-mails published by kik would seem to indicate) before taking a hiatus from Twitter in the middle of the PR catastrophe.

Oh, and if npm Inc's behaviour wasn't problematic enough: keep in mind they're a funded startup and acting like one (i.e. they're most likely not currently sustainable).

The public registry is a major cost center. I can't find any official numbers but considering what Nodejitsu (before npm Inc came along) posted during their fundraiser (which set out to raise $200k) and that the public registry has only been growing since while the underlying infrastructure seems to be still largely the same scalability-wise, I sincerely doubt npm Inc is currently cashflow positive.

Then you also need to add the expenses from running the company itself, running and sponsoring various events etc, and that currently private packages and their enteprise solution are their only means of income (which they somewhat desperately try to advertise at any chance they get these days) their financial situation seems dubious at best unless they can keep getting multi-million dollar investments at this rate. As npm Inc is rather secretive about the company they are running (even Isaac's LinkedIn profile is literally a joke) I can't speculate anything positive.

Again: none of this would be concerning if npm Inc was just a private company and just a startup. Bleeding money is just something investment-seeking startups do. Startups don't need to be sustainable. Startups can have crazy opinionated politics. But we're talking about core infrastructure of the JavaScript ecosystem the (decidedly non-private) Node Foundation is not just officially blessing but providing a monpoly status. Not to mention that npm Inc employees have been ridiculing and derailing various attempts to discuss the creation of alternatives within the Node Foundation.

At best, it's a conflict of interest. At worst, it's a disaster waiting to happen. Either way, npm (both the registry and the client) needs to be annexed or replaced.

Jason Knight's photo

Beating heart, or house of cards? Though the article itself says it all:

Bottom line, npm is just as safe as any other package manager out there at this point

Which is to say, not safe at all -- far too often people are blindly sleazing together other people's code that they don't even understand, hoping and praying that it won't fall apart despite it doing so time and time and time again. WORSE is the false "bandwagon" assumption that "thousands of others can't be wrong"... millions of people are wrong every day, often about the same things.

I simply don't trust the idea, as any project that gets "big enough" to need this type of garbage, is probably overthought, overcomplicated, and doing something horribly and terrifyingly wrong... wrong like overusing scripttardery where JS has zero damned business, and giving a giant middle finger to speed, sustainability, and worst of all accessibility... that last one being an unforgivable sin in my book; sadly that describes most crapplets built with web technologies which on the whole tell users with accessibility needs to go jump off a cliff and * themselves -- the opposite of why HTML was even created in the first place.

But again, I'm the guy who usually uses 10k of code for what everyone else uses 200k or more... so YMMV.

Alan Plum's photo

Donkey Kong says Trans Rights.

The problem with npm compared to other package managers is that because npm is intentionally not a walled garden (nor should it be) there are no guarantees about the safety of a package (or even a new release of a package) at all. It's entirely based on trust but it's treated as if it were 100% reliable.

However the exploit wasn't just about unsafe packages. It was specifically about self-replicating packages by exploiting the lack of security in the npm publishing mechanism. The official response was "if you don't want to accidentally publish packages, just stay logged out" which is unacceptable. The registry should at least offer two factor auth for sensitive operations like publishing a new package or version.