Last week, we wrote about how security outfit Rapid7 threw JetBrains, the company behind the popular CI/CD platform TeamCity, under the bus over allegations of silent patching. Now, JetBrains has gone on the offensive.
The software developer published its side of the story at the time, but felt the need to go a step further with another blog post this week, hammering home its argument that it did act responsibly and within the norms of vulnerability disclosure.
Further to that, it branded Rapid7’s approach, which was to release full details of the two TeamCity vulnerabilities as well as enough information for low-skilled attackers to develop exploit code just five hours after patches went live, “entirely unethical and harmful” to its customers.
“We fully support the timely disclosure of vulnerability details when a fix is released,” writes Daniel Gallo, TeamCity solutions engineer at JetBrains. “However, we provide only the necessary details for customers to understand the vulnerability’s scope and severity, enabling them to take the appropriate actions, but without disclosing so much information that it facilitates straightforward exploitation.
“We are also fully committed to disclosing complete vulnerability details (and the exploit steps) to benefit security researchers, but only after we see a significant number of customers have actually upgraded their systems to safe versions or installed patches.”
Following Rapid7’s detailed disclosure, within hours on-premises TeamCity users were reporting being hit by ransomware attacks. JetBrains notes that other customers had admin accounts installed on their machines and were most likely being recruited for use in DDoS campaigns.
“This was due to the immediate availability of publicly documented exploit examples published by Rapid7, which meant attackers of any skill level had all the resources they needed to quickly exploit the vulnerabilities in the wild,” says Gallo.
Seeing this kind of public war of words is a rarity in the infosec space which, generally speaking, is comprised of members that form a collaborative community and abide by agreed-upon norms.
We asked Rapid7 if it had a response but it didn’t immediately reply.
Unstandardized procedures
Speaking of those norms, JetBrains made a point of highlighting the disclosure norms of other major vendors in the industry, such as Google and Microsoft.
Google’s Project Zero team follows a policy that affords vendors 90 days to release fixes for vulnerabilities and a further 30 days after they’re released before detailed information is released about them.
Microsoft takes a more balanced approach, acknowledging that disclosing earlier forces vendors to develop fixes quicker, and sysadmins to apply them quicker too, but also notes that releasing information that can lead to exploit development raises the risk level significantly.
Similarly, OWASP acknowledges the merits of both sides and suggests finding a compromise on the disclosure policies if the two parties differ substantially. It does, however, note that it would be “sensible” for details about serious vulnerabilities to have a publication delay to limit the potential for harm.
National and international cybersecurity authorities generally agree a delay is necessary between reporting and publicly disclosing details of vulnerabilities, although these delays can vary between countries.
Spain, for example, will publish details of a security issue if the vendor fails to patch it within 60 days. Luxembourg’s default deadline is shorter at 30 days, albeit with flexibility built into it for more complex situations, while the US’s Cybersecurity and Infrastructure Security Agency (CISA) can disclose a vulnerability 45 days after the first attempt to contact the relevant vendor.
The EU’s cybersecurity agency, ENISA, sets out in its guidelines for developing a disclosure policy that a patch should always be developed within 90 days at the latest. It also advises that reporters should “respect a reasonable period” from the point of a patch being created before publicly disclosing the details of the vulnerability, allowing users to apply it as a priority.
Rapid7’s disclosure policy prioritizes timely disclosures. The policy reads: “Through transparent, open, and timely vulnerability disclosures, Rapid7 helps the entire internet protect and defend those assets and services critical to modern civilization.”
The policy stipulates that vendors have 60 days from the point of disclosure to release a fix before public disclosure, with the opportunity to extend that by 30 additional days if a fix can’t be developed in that time and, crucially, the vendor works in good faith.
It also says that the company will publish vulnerability details within 24 hours if they suspect a vendor to silently patch vulnerabilities.
Rapid7’s disclosure timeline indicates that both it and JetBrains differed on their definition of what is meant by “silent patching,” and after seeing JetBrains’ TeamCity patches go live, that’s what triggered the researcher’s full publication.
Plus, by JetBrains’ own admission, it decided four days after Rapid7’s disclosure that it would not be following a coordinated disclosure with the researchers. This was due to their reluctance to allow a delay in publication after patches were released.
JetBrains said this was a decision taken to protect its own customers from exploits, but Rapid7 likely saw it as a bad-faith working relationship.
Where now?
It may well now be the case where both vendors have had their say, and a resolution can’t be met, so it may be the last we’ll hear of it.
However, the to and fro should inform future discussions around public disclosures, and the ransomware attacks against TeamCity customers shouldn’t be taken lightly. The average cost of remediating an attack is roughly $1.5 million, meaning vendors must seriously consider the timing of their public disclosures.
Even if Rapid7 thought JetBrains was silently patching vulnerabilities, an assertion JetBrains denies, waiting a week before outing the developer may have helped customers apply patches to prevent these costly attacks.
There’s also no guarantee that waiting that time would have sprung admins into action to patch the flaws, so there are arguments to be made for both sides. Perhaps OWASP’s advice to reach a compromise, through good communication, may have been the best outcome here.
A spokeperson at Rapid7 sent us a statement: “Rapid7 abides by its disclosure policies.” ®