The Falcon's View

If SOX is vague and qualitative, PCI is specific and quantitative. SOX talks to the need/desire for "trusted" systems (financials, specifically), while PCI spells out in painful detail what the expectations are for security. Scoping SOX properly was a challenge in the beginning due to the lack of specificity, while PCI has always been quite clear about what data is considered important. In fact, while it is easy to see shortcuts through SOX compliance, PCI has been more challenging for organizations to side-step (caveat: level 1 merchants - other merchants have been fudging their questionnaires since the very beginning).

So, without further ado, let's get into a compare-and-contrast of answers to the question "Have regulations made eCommerce safer?" (*Note: by "safe" I'm speaking to larger holistic security and risk notions)

YES
Starting with the affirmative is easiest, so we will. Yes, regulations have improved eCommerce safety. SOX raised awareness and required public companies to formalize their IT audit programs, at least with respect to financial systems.

Raising awareness has led to a better understanding for and appreciation of security concerns - threats and vulnerabilities - and thus has raised the expectations of business when it comes to proper security measures. While many large organizations already had IT audit capabilities, SOX has helped formalize them, requiring external audits on a regular basis, and giving the auditors some teeth to instigate change.

If SOX raised awareness a little, PCI has raised it a lot. Especially within the IT and developer ranks, very few people can legitimately claim no knowledge of PCI security requirements. It's now widely known that credit card data must be encryption, and the introduction of the QSA program has improved the overall quality of audits.

More importantly, execs are now quite aware of what's at stake if security is lax. With SOX, they feared signing-off on reports should then be found to be inaccurate as the consequences could be prison. With PCI, there is a real and verifiable change of being sanctioned (fined) for lack of compliance, if your not cut off altogether. Especially in the case of PCI, these concerns spread upstream and downstream from provider to customer. The chain of accountability has had the net effect of improving overall system security.

Last, but not least, these regs have caused orgs to take a hard look at their data and systems, with the effect that they now have a better idea of what they have and where it is. This self-awareness leads to better decisions and better support for security throughout the enterprise. As a result, overall security is improved because IT is less of a mysterious black box (hole?).

NO
Current reqs like SOX and PCI have not improved eCommerce safety. If anything, they may have increased the risks facings orgs. This increased risk is a result of not requiring a risk management approach, limited awareness & visibility, aggressive shortcutting, inadequate enforcement of regs, unqualified auditors, all of which lead to a false sense of security.

No Risk Management Approach
Perhaps the most egregious failure of current regulations is that none of them advocate strongly for a risk management approach. This point is important because without a risk management approach, organizations are effectively making arbitrary, operationally-focused decisions that may or may not benefit the enterprise as a whole.

In the modern security profession, risk management is viewed as the foundation of the strategic security program, with technology being relegated to its proper operational role. Orgs are run based on strategic decisions, and thus security needs to be framed accordingly in order to better inform business leaders.

Bottom line: failing to assess and manage risk means that resources may not be focused on the right areas, creating exposures that could lead to major compromises - in spite of compliance efforts and state.

Limited Awareness & Visibility
While many claim that SOX and PCI have increased awareness and visibility, this isn't necessarily true. Sure, financial systems and credit cards receive much greater scrutiny today, but it's often at the expense of other systems and data. To this day, resources are still in limited quantity and availability. If those resources are focused on securing credit card data, then that means they're not (as) focused on other systems and data.

Case in point, the Hannaford and Heartland breaches. Both orgs claimed PCI compliance and yet were compromised. Clearly, being compliant didn't solve the problem, and in all likelihood distracted resources from other areas of concern.

Aggressive Shortcutting
One of the most interesting observations made of orgs under regulatory purview is their desire to take as many shortcuts as possible to achieve compliance. And, unfortunately, these are rarely good shortcuts, such as halting the storage of credit card data altogether.

It has been my experience that, due to resource constraints, many IT departments would rather not deal with regulations if they can avoid them. Perhaps the best example of this avoidance is with the many level 2-4 merchants (under PCI) who self-assess. There is little incentive for these orgs to pay more than lip service to PCI. As for SOX, many of these same orgs are private, and thus not subject to the regulation. These shortcuts do more harm than good. In fact, one could go so far as to say that due diligence is not being performed, introducing additional legal exposure for these organizations if something ends up going wrong.

Inadequate Enforcement
This point and the next really go hand-in-hand. Enforcement of regs seems to be spotty and inconsistent. In some cases, different QSAs make contradictory statements about PCI regs (like that 3DES was ok while AES was not - what's up with that?). In other cases, orgs are asked to just self-assess for compliance and it's just expected that honesty will be the best policy, even though the consequences for honesty may be severe.

With SOX, the reg only applies to public companies, and those companies have figured out how to ratchet the scope down so as to minimize their audit exposure and level of effort. Moreover, the definition of "trusted" is open to interpretation and has become quite watered down. In the beginning, some audit firms tried to force orgs to adopt CObIT, but in the end most orgs opted not to do so (for good reason - CObIT served the audit industry, not the companies having it thrust upon them). As such, the effectiveness of SOX seems to have diminished over time.

In the case of PCI, the mountain of work is immense, often causing auditors to either shortcut the process, or to rely on 3rd party statements that may or may not be valid. Some shortcuts - such as sampling - are accepted practices, but auditors should be wary if they're guided to systems rather than being allowed to randomly sample in-scope systems.

In the end, there is a degree of futility involved in enforcing PCI. The credit card companies need their customers to accept credit cards as payment for goods and services. As such, there isn't much incentive for them to cut merchants off, and even levying fines could have a negative impact on their bottom line. As such, the game is stacked against proper enforcement because to strictly enforce the regs would be to cut into their own business.

Unqualified Auditors
Part of the reason regulations have inadequate enforcement is because the quality of auditors is fairly low. Now, I know, as a former auditor, that this isn't necessarily a fair generalization, but the simple fact is that it's a true statement. And here's why: well-trained, experienced, competent auditors are expensive. Moreover, audit is generally boring and thankless work. Nothing about that makes it appealing for the truly strong security minds in the industry.

There's another aspect here, too, that's more expensive, and that's the last bit above: good auditors are expensive. If your personnel cost more, then your bill rates will be higher, and thus either your realization will take a hit (audit firms) or your bills will become unsustainably high (customers). While it would be nice to have highly competent auditors in the field, the simple fact is that once people achieve a reasonable degree of competency, they will move onto other more interesting, more profitable work.

As such, we have a self-perpetuating problem where auditors are generally unqualified to perform what is ultimately extremely important work. I can think of a couple times where I've tangled with auditors (as the customer) and in both cases I ended up being right simply because I was more experienced than the auditor. In one case, there was a technical finding that was listed as a "high risk" that a) didn't make any sense, and b) didn't warrant a "high" finding. When we pushed back, we got to talk to a competent senior auditor who took one look at the report and said "yeah, I don't know why this is recorded as a 'high risk' finding because it isn't, and it also looks like it was never validated." In the other case, a QSA wrote up a finding that our use of AES-128 wasn't PCI compliant, and thus it was recommended that we move to "3DES 128". My response was that 1) there is no such thing as "3DES 128" - 3DES is actually 168 bits, and 2) 3DES is deprecated by the AES, as reflected in NIST documentation (which I provided). What's interesting is that I've heard this same story from others who've had the same experience with QSAs, so maybe this was a QSA training issue.

At any rate, the point is this: if your auditors aren't very good, then they will rely heavily on checklists, and that means that they will only find the obvious things. If you can't pass that kind of audit, then you might as well hang it up, unless of course you're just starting on your security work. :)

False Sense of Security
The cumulative effect of all these problems with regulations leads me to the conclusion that, while they have increased some awareness and security technologies, companies are in fact in a worse security state than before. Perhaps I'm overstating, but it concerns me greatly that companies feel good about their security because they're SOX and/or PCI compliant, and yet are still getting compromised.

In a discussion on this topic recently, a colleague argued that if perfect security is a 100 and no security is 0, then progressing from 0 to 20 thanks to regulations is an important advance. I agreed that this was, indeed, a good thing, but where I get concerned is when companies think their security is a 20 when it's in fact a 10. The last thing we need are orgs acting boldly because they think that their security is good, when in fact it's not very good at all, not to mention that it's nowhere near strong security.

I wonder if it wouldn't be better to have had no advances, followed by massive hacks, followed by much more strict regulations that make companies financially responsible for their security, rather than what we have today, such as with a self-regulated industry that has little real incentive in holding companies accountable.

On top of this, I also wonder about the "pass the buck" phenomenon that we see with PCI. Rather than address security weaknesses inherent in credit cards, such as by implementing pin-n-chip smartcards, the PCI Council members instead have pushed responsibility to the issuing banks and the eCommerce businesses that allow credit cards for payment. Like it or not, credit cards are now an essential tenet of business life, but yet credit card companies seem to believe that somebody else should protect them.

As another example, look at Microsoft. Yes, they're much maligned, and yes, they've made significant progress in the last 10 years, but to this day they are not directly punished for compromises that result from their insecure code. Until that dynamic is changed, the only people who pay the penalty are consumers, who are already paying through the nose for the "privilege" of owning technology.

Add this all up and we have a major problem: consumers hear about regulations and resulting compliance and think everything is ok. Businesses achieve compliance and then become complacent believing they're ok. The core providers, like the PCI Council members, in the meantime, defer responsibility to their customers, as do the technology giants like Microsoft. The result is a cognitive dissonance - a false belief that eCommerce is safe and secure when, in fact, it's not very safe at all.

What's worse, these sorts of problems have been occurring consistently throughout history. Look back at the Ford Explorer exploding tires problem as a somewhat recent example (see http://www.firestone-tire-recall.com/ for an example). Consumers end up paying the price, and yet the more things change, the more they stay the same. Anyway...

SOLUTION?!?
It's undeniable that there have been improvements in eCommerce security as a result of regulations, but it also seems clear that these improvements are often over-estimated and lead to laxness. There appears to be a fundamental mental disconnect what people believe is true and what is actually true when it comes to online security. The fact that the TSA is allowed to exist and operate in its current manner as they perform the charade of security theater is further proof that people simply don't get it.

So, how do we solve this problem? How do we get people to better understand security? I wish that I had a good answer. I think the solution will need to be two-pronged. One prong will have to be end-user education that results in meaningful change. The other prong will have to be very strong legislation that puts the legal responsibility squarely on the shoulders of businesses, including the root cause enterprises, to spur them into realistic change. If that means legislation requiring smartcard technology, then so be it. I hate to see legislation that micromanages to that level, but if credit card companies are unwilling to take those reasonable measures, then something must be done to compensate.

In the end, the consumer will end up paying the bill, so as a bonus 3rd prong, legislation will need to be written to protect the consumer. Consumers are already getting hammered in the current economy while banks hoard federal tax dollars. This sort of behavior needs to be nipped in the bud asap, and not just from a financial lending perspective.

Here's to hoping for a evolutionary jump to better practices!