In the continuing saga of the PlayStation Network (PSN) outage that is now entering its 8th day, Sony has provided more details on exactly what happened, and the news only gets worse.  Beyond the information they provided yesterday, an email went out to PSN users today which provided much of the information that I have already posted.  What has come out today gives me an idea of exactly how this attack occurred.

First of all, the news that’s good. All credit card information was encrypted, and Sony does believe that no credit card data has been compromised.  While you should still monitor your cards closely for the next while, this is good news, and does make me breath a little easier.
Now, the bad news.  Based on wording in a Frequently Asked Question (FAQ) page for this issue, it appears that this was an actual physical attack on Sony’s data center.  This means that an individual or group of people obtained physical access to the PSN data center and directly connected to a server and was able to download the personal data.  According to Sony, the personal information was held in a database that was unencrypted, and that data was obtained by the intruders.

As a result of this attack, Sony is moving the PSN to a new data center in a “new, more secure location.” I believe that is why the restoration of service is taking so long. Moving a data center which accommodates over 70 million users is no small task, and will of course take some time.  There is a sheer scale here that both helps and hinders.  I’m sure that Sony is very efficient at adding capacity to the PSN by adding new servers to a cluster in a datacenter.  But building a cluster from scratch, and then adding more capacity, is not an easy task. I’m also willing to bet that all personal information will be encrypted from this point forward as well, and it is likely that software engineers not only have to write the software to make that happen, they also have to write software that will let the PSN access that encrypted data, something they likely hadn’t planned for when designing the PSN.  I actually feel bad for every employee of Sony who works on the software and hardware for the PSN. I have little doubt that this has probably been the worst week of their lives.  The work they are doing to restore the service is enormous, and they will likely never get credit for what will be an amazing feat.

The last bit of news is one that is good for everyone.  When the PSN comes back online, there will be be a software update (I assume for both the PS3 and the PSP), that will require users to change the PSN account passwords, since those have been compromised. I will theorize that the update will also include any new encryption pieces that are likely being build into the PSN right now.

I’ve thought about this for a bit, and I’m kind of torn on what to say.  When someone gets physical access to a server, it is significantly easier to hack into that server and gather information. This does make me feel better about the robustness of the PSN itself, since the attack came from within.  However, the fact that someone was able to get into what should be a secure area is unacceptable, but physical attacks can happen.  This is akin to a bank robbery or a theft from a museum.  We are appalled at how such a thing can be allowed to happen, and why there wasn’t more security, but the simple fact is that they do happen from time to time.

Because of the type of attack, I honestly don’t know where to start with how my personal information was obtained.  It is easy to say that all of that personal information should have been encrypted on the servers, and that it should have been impossible to access, but in reality, I don’t know of too many companies that actually encrypt all data on their servers.  To put it simply, encrypting everything is a significant amount of work, and does make recovering from issues more difficult.  Data should be safe in a physically secure environment, and sometimes we depend on that to keep that data safe.  I will grant that not many networks have data on the scale of Sony, but I also wonder if someone broke into a Google data center; how much personal information could be obtained there?  That’s a question I hope we never find an answer to, but it is a valid question.

At the end of the day, the result of the attack is the same. Personal information of tens of millions of people, including myself, has been compromised.  Everyone who is on the PSN is more vulnerable to identity theft, phishing attacks, and password attacks than they were two weeks ago. This is fact, and nothing I have said today changes that. I’m not trying to downplay the severity of this breach, because it is bad, among the worst I have ever heard of in the industry  That being said, I find it a bit easier to, and this may not be a good word for this, sympathize with Sony on the method of attack; one which is arguably the most difficult to predict and defend against. I’m willing to bet that 90% of workplaces would be in a similar, albeit smaller scale, situation if the same thing happened to them.  Would mine? I hope to never find out.

[Read] – Sony PlayStation Blog

[Read]  - Joystiq