“I just noticed ‘foreach‘ on NPM is controlled by a single maintainer,” wrote Vick in a Twitter post on Monday. “I also noticed they let their domain expire, so I bought it before someone else did. I now control ‘foreach’ on npm, and the 36,826 projects that depend on it.”
That’s not quite the full story – he probably could have taken control but didn’t. Vick acquired the lapsed domain that had been used by the maintainer to create an NPM account and is associated with the “foreach” package on NPM. But he said he didn’t follow through with resetting the password on the email account tied to the “foreach” package, which is fetched nearly six million times a week.
Anyone poking around is going to find accounts easy to take over in this way. I was not lucky or special
In an email to The Register, Vick explained, “As an NPM team member pointed out, the emails associated with NPM accounts and the emails used on the package themselves can sometimes be different, but even if this is the case controlling an owner account would make an easy social engineering case to customer support. I did not log into the account, as again, that crosses a line. I just sent a password reset email and bailed.
“Regardless of how much control I have over this particular package, which is unclear, NPM admits this particular expired domain problem is a known issue, citing this 2021 [research paper] which says, ‘We also found 2,818 maintainer email addresses associated with expired domains, allowing an attacker to hijack 8,494 packages by taking over the NPM accounts.’
“In other words, anyone poking around is going to find accounts easy to take over in this way. I was not lucky or special.”
His point, which he has been trying for several years to communicate to those overseeing NPM – a part of GitHub since March 2020 – is that taking over the NPM account of a popular project to conduct a software supply chain attack continues to be too easy.
“Git at least has code signing built in, and the NPM team was not even using that… which means anyone could even spoof code commits as any of their own internal developers,” Vick explained in an email to The Register.
“I was frustrated enough by 2020 that I made the potentially ill-advised choice to send my message about the state of affairs and calls to action to the NPM team in the form of a commit to their own repo. To drive the point home I demonstrated I could impersonate one of their security leads (Sorry, Adam).”
He said he also pointed out that it’s trivial to take over top NPM accounts because most didn’t have any phishing-resistant 2FA enabled.
Vick explained his rationale in a comment on his commit written several days later. “Major e-commerce platforms, major financial firms like PayPal, several major banks, as well as most major crypto-asset exchanges rely on NPM packages for critical infrastructure where billions of dollars are on the line,” he wrote.
“I work with many of these companies in a security capacity and the level of life-ruining theft I see at close range on a regular basis due to vulnerable/hijacked packages or lack of 2FA on critical accounts is gut wrenching.”
Naming and shaming
Vick went so far as to set up, with the help of John Naulty Jr, “a spreadsheet of NPM package maintainers with terrible security practices.” The spreadsheet was featured in a blog post about NPM security by Vick and Naulty that went up the same day as the rogue commit.
Naulty, a software security engineer, told The Register in a phone interview that Vick and he were motivated to do something as a result of the event-stream incident. He said those named on the spreadsheet were largely responsive to being called out and many have adopted better security practices.
And he credits Vick’s orphaned commit with getting someone’s attention in the Microsoft, GitHub, and NPM ecosystem. “Eventually, they released a feature that now says this commit is not attached to any branch in this organization,” he said.
We are all just trusting strangers on the internet to give us good candy from their truck
Naulty said the SolarWinds attack that emerged in late 2020 really brought attention to supply-chain security and has led to a number of startups focused on the space. And he credited projects like OpenSSF with pushing to improve supply chain security.
Naulty said other packaging ecosystems like PyPI have had similar problems and credited the open source community with at least making an effort. He said NPM security is improving but there are still many types of attacks that can be conducted.
“We are all just trusting strangers on the internet to give us good candy from their truck,” he said.
That’s still a risk. On Tuesday, JFrog reported an NPM supply chain attack targeting German industrial companies Bertelsmann, Bosch, Stihl, and DB Schenker via malware in NPM packages – though the attack may be just a penetration test that attracted the notice of a security firm.
And it’s been a risk for years. Vick’s post describes an effort dating back almost a decade to implement package verification in NPM that was abandoned for being too hard.
“We as a community have created a dumpster fire together and I think we need some major changes to correct it now,” wrote Vick.
2FA all the way
GitHub has been responding to the agitation, announcing a plan in December, 2021, to enroll all NPM maintainers in login verification and rolling out the initial phase of that program in February, mandatory 2FA for the top 100 package maintainers.
On Tuesday, GitHub launched a beta test of its improved 2FA implementation for all NPM accounts. According to Myles Borins, open-source product manager, NPM accounts now support: multiple second factors, including security keys, biometric devices, and authentication apps; a new 2FA configuration mention for managing keys and recovery codes; full CLI support; and the ability to review and regenerate recovery codes.
Borins also said that at the end of the month, on May 31, GitHub will enroll the next mandatory 2FA cohort, the maintainers of the top 500 npm packages. Then, later this year, a final group of maintainers – those with packages having more than one million weekly downloads or more than 500 dependents – will be required to adopt 2FA.
GitHub declined to comment on this issue beyond what’s said in the blog post.
Vick says he’s thrilled by the announcement, which came as a surprise.
“The timing is a bit fun though, because just this morning Github/NPM announced they are finally adding hardware MFA support to NPM, which is a huge win,” he said. “I am really happy to see this because that is the best way to protect accounts. We in the security community have been demanding this for years.
“That said, it still does not protect the code should a developer fail to set up 2FA properly, or have an email with weak 2FA, as most still do today. A malicious or compromised NPM employee could also alter any code they wish right now, and with some of that code being responsible for the movement of billions of dollars by major fintech companies, I don’t envy them walking around with targets that big on their backs.”
Vick argues that user code-signing can solve all of these problems. “I really hope NPM takes this step soon,” he said. “I am talking with a member of their team tomorrow and we will see where this goes.” ®