- Jun 12, 2007
23,000 HTTPS certificates axed after CEO emails private keys
A major dust-up on an Internet discussion forum is touching off troubling questions about the security of some browser-trusted HTTPS certificates when it revealed the CEO of a certificate reseller emailed a partner the sensitive private keys for 23,000 TLS certificates.
The email was sent on Tuesday by the CEO of Trustico, a UK-based reseller of TLS certificates issued by the browser-trusted certificate authorities Comodo and, until recently, Symantec. It was sent to Jeremy Rowley, an executive vice president at DigiCert, a certificate authority that acquired Symantec's certificate issuance business after Symantec was caught flouting binding industry rules, prompting Google to distrust Symantec certificates in its Chrome browser. In communications earlier this month, Trustico notified DigiCert that 50,000 Symantec-issued certificates Trustico had resold should be mass revoked because of security concerns.
When Rowley asked for proof the certificates were compromised, the Trustico CEO emailed the private keys of 23,000 certificates, according to an account posted to a Mozilla security policy forum. The report produced a collective gasp among many security practitioners who said it demonstrated a shockingly cavalier treatment of the digital certificates that form one of the most basic foundations of website security.
Generally speaking, private keys for TLS certificates should never be archived by resellers, and, even in the rare cases where such storage is permissible, they should be tightly safeguarded. A CEO being able to attach the keys for 23,000 certificates to an email raises troubling concerns that those types of best practices weren't followed. (There's no indication the email was encrypted, either, although neither Trustico nor DigiCert provided that detail when responding to questions.) Other critics contend Trustico emailed the keys in an attempt to force customers with Symantec-issued certificates to move to Comodo-issued certificates. Although DigiCert took over Symantec's certificate issuance business, it doesn't count Trustico as a reseller.
In a statement, Trustico officials said the keys were recovered from "cold storage," a term that typically refers to offline storage systems.
"Trustico allows customers to generate a Certificate Signing Request and Private Key during the ordering process," the statement read. "These Private Keys are stored in cold storage, for the purpose of revocation."
The discussion also raises new questions about Symantec's adherence to industry-binding rules during the time it was a browser-trusted certificate authority that allowed Trustico to resell its certificates. Under the Baseline Requirements for the Certificate Authority Browser Forum, resellers aren't permitted to archive certificate private keys. The email raises the specter Trustico had been doing just that when it offered to accept certificate-signing requests on its website. As the holder of the root certificate used to sign the TLS certificates Trustico was reselling, Symantec was ultimately responsible for ensuring this requirement was being followed, although in fairness, there was probably no way for Symantec to detect a violation. Trustico officials further called Symantec's security into question on Wednesday when they voiced serious concerns over Symantec's handling of an account Trustico used to resell the certificates.
"During our many discussions over the past week we put it to you that we believe Symantec to have operated our account in a manner whereby it had been compromised," the Trustico officials wrote. They continued: "We believe the orders placed via our Symantec account were at risk and were poorly managed. We have been questioning Symantec without response as to concerning items for about a year. Symantec simply ignored our concerns and appeared to bury them under the next issue that arose."
Symantec officials didn't respond to an email seeking comment for this post.
Wednesday's flap comes after Google and Mozilla have spent years trying to better secure the security of certificates their browsers trust. DigiCert's transparency and adherence to the Baseline Requirements demonstrates that many certificate authorities and resellers are acting in good faith. Unfortunately, the way the Internet's TLS certificate issuance process works, a single point of failure is all it takes to create compromises that endanger the entire system. Readers can expect Google and Mozilla to spend considerable time and resources in the coming weeks unraveling the breakdown that came to light Wednesday.
Update: Several hours after this post went live, Trustico's website went offline after a Web security expert posted a critical vulnerability on Twitter. The flaw, in a trustico.com website feature that allowed customers to confirm certificates were properly installed on their sites, appeared to allow attackers to run malicious code on Trustico servers with unfettered "root" privileges.