Category Archives: incidents

IT Lacks Engineering Discipline and Rigor

Every week we read the news about new, spectacular security breaches. This has been going on for years, and sometimes I wonder if there are any organizations left that have not been breached.

Why are breaches occurring at such a clip? Through decades of experience in IT and data security, I believe I have at least a part of the answer. But first, I want to shift our focus to a different discipline, that of civil engineering.

Civil engineers design and build bridges, buildings, tunnels, and dams, as well as many other things. Civil engineers who design these and other structures have college degrees, and they have a license called a Professional Engineer. In their design work, they carefully examine every component and calculate the forces that will act upon it, and size it accordingly to withstand expected forces, with a generous margin for error, to cover unexpected circumstances. Their designs undergo reviews before their plans can be called complete.  Inspectors carefully examine and approve plans, and they examine every phase of site preparation and construction. The finished product is inspected before it may be used.  Any defects found along the way, from drawings to final inspection, results in a halt in the project and changes in design or implementation.  The result: remarkably reliable and long-lasting structures that, when maintained properly, provide decades of dependable use. This practice has been in use for a century or two and has held up under scrutiny. We rarely hear of failures of bridges, dams, and so on, because the system of qualifying and licensing designers and builders, as well as design and construction inspections works. It’s about quality and reliability, and it shows.

Information technology is not anything like civil engineering. Very few organizations employ formal design with design review, nor inspections of components as development of networks, systems, and applications. The result: systems that lack proper functionality, resilience, and security. I will explore this further.

When organizations embark to implement new IT systems – whether networks, operating systems, database management systems, or applications – they do so with little formality of design, and rarely with any level of design or implementation review.  The result is “brittle” IT systems that barely work. In over thirty years of IT, this is the norm that I have observed in over a dozen organizations in several industries, including banking and financial services.

In case you think I’m pontificating from my ivory tower, I’m among the guilty here. Most of my IT career has been in organizations with some ITIL processes like change management, but utterly lacking in the level of engineering rigor seen in civil engineering and other engineering disciplines.  Is it any wonder, then, when we hear news of IT project failures and breaches?

Some of you will argue that IT does not require the same level of discipline as civil or aeronautical engineering, mostly because lives are not directly on the line as they are with bridges and airplanes. Fine. But, be prepared to accept losses in productivity due to code defects and unscheduled downtime, and security breaches. If security and reliability are not a part of the design, then the resulting product will be secure and reliable by accident, but not purposely.

In air travel and data security, there are no guarantees of absolute safety

The recent tragic GermanWings crash has illustrated an important point: even the best designed safety systems can be defeated in scenarios where a trusted individual decides to go rogue.

In the case of the GermanWings crash, the co-pilot was able to lock the pilot out of the cockpit. The cockpit door locking mechanism is designed to enable a trusted individual inside the cockpit from preventing an unwanted person from being able to enter.

Such safeguards exist in security mechanisms in information systems. However, these safeguards only work when those at the controls are competent. If they go rogue, there is little, if anything, that can be done to slow or stop their actions. Any administrator with responsibilities and privileges for maintaining software, operating systems, databases, or networks has near-absolute control over those objects. If they decide to go rogue, at best the security mechanisms will record their malevolent actions, just as the cockpit voice recorder documented the pilot’s attempts to re-enter the cockpit, as well as the co-pilot’s breathing, indicating he was still alive.

Remember that technology – even protective controls – cannot know the intent of the operator. Technology, the amplifier of a person’s will, blindly obeys.

The security breaches continue

As of Tuesday, September 2, 2014, Home Depot was the latest merchant to announce a potential security breach.

Any more, this means intruders have stolen credit card numbers from its POS (point of sale) systems. The details have yet to be revealed.

If there is any silver lining for Home Depot, it’s the likelihood that another large merchant will probably soon announce its own breach.  But one thing that’s going to be interesting with Home Depot is how they handle the breach, and whether their CEO, CIO, and CISO/CSO (if they have a CISO/CSO) manage to keep their jobs. Recall that Target’s CEO and CIO lost their jobs over the late 2013 Target breach.

Merchants are in trouble. Aging technologies, some related to the continued use of magnetic stripe credit cards, are making it easier for intruders to steal credit card numbers from merchant POS systems.  Chip-and-PIN cards are coming (they’ve been in Europe for years), but they will not make breaches like this a thing of the past; rather, organized criminal organizations, which have made a lot of money from recent break-ins, are developing more advanced technologies like the memory scraping malware that was allegedly used in the Target breach. You can be sure that there will be further improvements on the part of criminal organizations and their advanced malware.

A promising development is the practice of encrypting card numbers in the hardware of the card reader, instead of in the POS system software.  But even this is not wholly secure: companies that manufacture this hardware will themselves be attacked, in the hopes that intruders will be able to steal the secrets of this encryption and exploit it. In case this sounds like science fiction, remember the RSA breach that was very similar.

The cat-and-mouse game continues.

Why wait for a security breach to improve security?

Neiman Marcus is the victim of a security breach. Neiman Marcus provided a statement to journalist Brian Krebs:

Neiman Marcus was informed by our credit card processor in mid-December of potentially unauthorised payment card activity that occurred following customer purchases at our Neiman Marcus Group stores.

We informed federal law enforcement agencies and are working actively with the U.S. Secret Service, the payment brands, our credit card processor, a leading investigations, intelligence and risk management firm, and a leading forensic firm to investigate the situation. On January 1st, the forensics firm discovered evidence that the company was the victim of a criminal cyber-security intrusion and that some customers’ cards were possibly compromised as a result.

We have begun to contain the intrusion and have taken significant steps to further enhance information security.

The security of our customers’ information is always a priority and we sincerely regret any inconvenience. We are taking steps, where possible, to notify customers whose cards we know were used fraudulently after making a purchase at our store.

I want to focus on one of Neiman Marcus’ statements:

We have … taken significant steps to further enhance information security.

Why do companies wait for a disaster to occur before making improvements that could have prevented the incident – saving the organization and its customers untold hours of lost productivity? Had Neiman Marcus taken these steps earlier,  the breach might not have occurred.  Or so we think.

Why do organizations wait until a security incident occurs before taking more aggressive steps to protect information?

  1. They don’t think it will happen to them. Often, an organization eyes a peer that suffered a breach and thinks, their security and operations are sloppy and they had it coming. But alas, those in an organization who think their security and operations are not sloppy are probably not familiar with their security and operations. In most organizations, security and systems are just barely good enough to get by. That’s human nature.
  2. Security costs too much. To them I say, “If you think prevention is expensive, have you priced incident response lately?”
  3. We’ll fix things later. Sure – only if someone is holding it over your head (like a payment processor pushing a merchant or service provider towards PCI compliance). That particular form of “later” never comes. Kicking the can down the road doesn’t solve the problem.

It is human nature to believe that another’s misfortunes can’t happen to us. Until it does.

Why there will always be security breaches

At the time of this writing, the Target breach is in the news, and the magnitude of the Target breach has jumped from 40 million to as high as 110 million.

More recently, we’re now hearing about a breach of Neiman Marcus.

Of course, another retailer will be the next victim.  It is not so important to know who that will be, but why.

Retailers are like herds of gazelles on the African plain, and cybercriminals are the lions who devour them.

As lions stalk their prey, sometimes they choose their victim early and target them. At other times, lions run into the herd and find a target of opportunity: one that is a little slower than the rest, or one that makes a mistake and becomes more vulnerable. The slow, sick ones are easy targets, but the healthy, fatter ones are more rewarding targets.

As long as their are lions and gazelles, there will always be victims.

As long as there are retailers that store, process, or transmit valuable data, there will always be cybercriminals that attempt to steal that data.

Rest in peace: officers Renninger, Griswold, Owens and Richards

Bookmark This (opens in new window)

Update 12/12/2010: Donate to Lakewood Police Independent Guild to benefit the families of the four slain officers

Today, four Lakewood WA police officers were assassinated in cold blood while conducting police business in a local coffee house. This happened very close to where I live, less than a month after a Seattle police officer was gunned down.

In my work I collaborate with and support law enforcement. I appreciate what they do for us.

These four officers leave nine children behind. This aspect makes this especially tragic.


Tacoma News Tribune

Seattle Times

the Zune failure and Microsoft

Bookmark This (opens in new window)

Many articles in the press have chronicled the failure of Microsoft’s first-generation 30GB Zune MP3 players.  They all simply froze on December 31, 2009. The remedy: they had to be powered on and allowed to completely discharge, and then wait until after 12:00 GMT on 1/1/09 before they could be used again. Total downtime – around 24 hours.

Microsoft is yearning to expand its market space into embedded systems in automobiles, military systems, and other areas. Am I being overly fearful of the consequences of a Microsoft whose products are even more deeply embedded into the machinery of our lives?  Today is one of those days when I am distrustful of technology as a path for an easier life.


Leap year Zune glitch persists for some (CNN)

Original Zune confounded by leap year, shuts down (Seattle Times)

Zune support page explanation

Zune insider blog

Going public with website vulnerabilities that expose credit card numbers

Bookmark This (opens in new window)

CybercrimeI am a customer of an international company whose logo is highly recognizable and whose brick-and-mortar services I use frequently. I pay for these services by credit/debit card on their website. I noticed two months ago that the website has a vulnerability that exposes credit card numbers through form field caching, which means that public-access computers could expose credit card numbers (and security codes) to others.

I have contacted the company three times in the past six weeks. Their website makes it impossible to know who their security people are or which continent they work on (this is a company that has presence in over 100 countries). I have written the press office three times. None of my communications have read by a human, as far as I can tell.

I will be giving them another week or two before I go public. I’ve told them so in every way that they make available. After telling them almost two months ago, I logged on today and the vulnerability is still there. It is SO easy to fix – it does not require any changes to their data model, workflow, or processes. All they have to do is add an ‘AUTOCOMPLETE = “off” ‘ to two fields in one form and they’re done.

As a security professional I am duty-bound to inform this organization. I’ve done so many times, and have not heard any response. If they continue to turn a deaf ear, I will go public in April.

Fraudulent Microsoft Update

Bookmark This (opens in new window)

There is lots of activity around an email and a fraudulent Microsoft Update web site (that the email directs you to), claiming that there is an urgent Microsoft update.

The web site looks like a legitimate Microsoft site and contains an “Urgent Install” button that, when clicked, attempts to download and install malicious software on your system. The file that attempts to download is not signed by Microsoft and is called “WindowsUpdateAgent30-x86-x64.exe”.

This web site is using fast flux DNS for its web hosting. That make it hard to track and close down, so we expect it to be around for awhile.

Please advise your users, if they receive this type of email, they should just delete it. Microsoft does not distribute updates by sending emails directly to individuals or distribution lists.

Credit to NW WARN for the contents of this advisory.

New severe home/small business router vulnerability requires attention

Bookmark This (opens in new window)

A severe UPnP flaw allows router hijacking. Experts believe that 99% of home routers are vulnerable. This is a potentially alarming development.

An attacker will most likely use the vulnerability to alter a home (or small business) router’s DNS settings, which will effectively direct every computer in the network to visit sites of the attacker’s choosing.

How the attack will work: attackers will place malicious code on web sites in SWF (Flash) or other active content that will contain UPnP commands that the router will intercept.

Things you can do:

1. Disable UPnP on your router. Most people don’t use it anyway. I use it but will probably deactivate it this week.

2. Implement OpenDNS or ScrubIT DNS on your internal systems. This will effectively bypass your router’s DNS, making a DNS attack on your router irrelevant.

3. Find someone who knows about home/SMB Internet router configuration who can tell you if your router has been compromised. Know your router’s configuration.

4. Change (or establish) the administrator password on your router. This is just a good idea anyway.

5. Contact your Internet service provider and ask for information about updates to counter this vulnerability.

6. Implement firewalls on individual systems in your network. If an attacker decides to deactivate the firewall function on your router, PC based firewalls will continue protecting them.

Links to information:

CERT Warning

Information Week story

Computerworld story

SANS Internet Storm Center article

Older story on home router vulnerability

Prediction about consumer online confidence comes true, a little late

Bookmark This (opens in new window)

I was a panelist at the annual WSA Predictions dinner event in December 2005. Each panelist was asked to make technology-related predictions for the coming year, 2006. My prediction: one or more significant security events would result in a downturn (or, at least, a slowdown in the rate of growth) in consumer online shopping.

Little did I know that the perpetrators of the TJX breach had already been busy at work skimming millions of credit card numbers out of TJX’s computers.

Well, well. I read an article yesterday that reads, “Holiday shoppers are in stores and online again this year — but they don’t feel too safe doing it, according to a report scheduled to be published Monday by security vendor Utimaco.” Consumers are shopping online, but nervous, as they recall the colossal TJX security breach earlier this year.

Link to full article:

TJX breach twice the size of earlier estimates

Submit: Add to your Digg This Slashdot GotNews StumbledUpon Reddit

It initially appeared that the size of the TJX breach was around 48 million credit cards, although the theoretical maximum was as high as 200 million cards. Recently, banks are finding that the TJX breach was more like 94 million cards. That nearly one credit card for every household in the U.S.

VISA and MasterCard have lost tremendous sums of money due to this breach alone. Losses are estimated at $1.04 to $1.28 per card, which translates into a total loss as high as $120 million. But the total cost of the incident will be much higher, close to $1 billion, when counting settlements and lost sales as well as the direct losses cited here.

It is common knowledge that the most likely attack vector was unsecure wireless networks using the extremely weak WEP protocol. WEP was known to be weak in 2000, and yet six years later TJX (and thousands of other businesses) were relying upon it to protect their networks. That’s about as effective as a sign reading “Please don’t come in” on an unlocked door.

The Canadian government’s privacy commissioner released a report criticizing TJX for its weak security. This report is succinct in its findings, and is good reading if you have yet read a detailed account of the TJX breach. TJX’s 10-K report is another good source of information.

TJX breach was twice as big as admitted, banks say (The Register)
Banks claim TJX breach twice as bad (ZDNet)

Skype restored, but executives still in hiding

Bookmark This (opens in new window)

Disclaimer: I’m a big fan of Skype – it’s my IM client of choice.

The Skype PR disaster continues unabated. There is no end in sight.

The Skype network service has been restored. Here is a short explanation of their problem:

On Thursday, 16th August 2007, the Skype peer-to-peer network became unstable and suffered a critical disruption. The disruption was triggered by a massive restart of our users’ computers across the globe within a very short timeframe as they re-booted after receiving a routine set of patches through Windows Update.

The high number of restarts affected Skype’s network resources. This caused a flood of log-in requests, which, combined with the lack of peer-to-peer network resources, prompted a chain reaction that had a critical impact.

Normally Skype’s peer-to-peer network has an inbuilt ability to self-heal, however, this event revealed a previously unseen software bug within the network resource allocation algorithm which prevented the self-healing function from working quickly. Regrettably, as a result of this disruption, Skype was unavailable to the majority of its users for approximately two days.

This and other postings were made by someone named Villu Arak, about which practically nothing can be found.

Skype had one of the most significant outages of any online service, and none of its executives have so much as said “Hello”.

Skype’s network disaster has been solved, but Skype’s PR disaster continues. Have they ever heard the terms “goodwill”, “media relations”, or “customer service”?

Earlier story.

Skype: not one disaster, but two

Bookmark This (opens in new window)

Disclaimer: I’m a big fan of Skype – it’s my IM client of choice.

Update to the story.

Aug. 19, 2007 – Skype suffered a colossal outage last week, and the network is just now coming back. They have promised to tell everyone on August 20, 2007, what happened in the previous week.

Two disasters hit Skype last week:

  1. The network outage, whatever it was about.
  2. The complete absence of Skype and eBay executives throughout the crisis.

While the first disaster might not have been preventable, the second disaster was a direct result of decisions made by Skype and eBay executives who apparently chose to hide. They appear to have left the Skype technical recovery team out to flap in the wind, alone, with no visible public support. This led to rumors and speculation about what really happened, and whether Skype and eBay executives care about the community of users. Those executives committed the cardinal sin of disaster management: communicating from a high level about what’s happened and what is being done about it. Instead of being told, we were left to wonder if they even noticed. Maybe they were too busy and could not be bothered about a world-wide outage. Their silence is deafening – we hear it loud and clear.

In a different part of the world, the Utah mine accident is playing out. Those executives got it right: they are right there, working and concerned, and are making frequent statements to the press. They are telling everyone what they know, what they don’t know, and what they’re doing.

Murray Energy Executives get two thumbs up for being there. Skype and eBay get two thumbs down for being silent and absent.

While Skype users may breathe a collective sigh of relief that the network is running again, I wonder if Skype and eBay have even noticed that the second disaster has taken place and has yet to be addressed.

FBI implanted spyware leads to arrest of bomb threat suspect

Submit: Add to your Digg This Slashdot GotNews StumbledUpon Reddit

The FBI used a novel type of remotely installed spyware last month to investigate who was e-mailing bomb threats to a high school near Olympia, Wash.

Federal agents obtained a court order on June 12 to send spyware called CIPAV to a MySpace account suspected of being used by the bomb threat hoaxster. Once implanted, the software was designed to report back to the FBI with the Internet Protocol address of the suspect’s computer, other information found on the PC and, notably, an ongoing log of the user’s outbound connections.

My earlier blog entry on whether anti-virus can detect law enforcement-installed malware.

Entire story here:

It was the employer’s policy to have data kept in employees homes for “safe keeping”

Submit: Add to your Digg This Slashdot GotNews StumbledUpon Reddit

In the security breach du jour, the State of Ohio announced that private information (names, social security numbers, and maybe more) on 64,000 state employees was compromised.

They were stolen from a state intern’s car, where they were written to a portable storage device (thumb drive? USB hard drive? the story does not say).

The practice of taking state information to employees’ homes is apparently the official policy. A news article reads, “Under protocol in place since 2002, a first backup storage device is kept at a temporary work site for a state office along with the computer system that holds all the employee information, and a second backup device is given to employees on a rotating basis to take home for safekeeping, officials said.”

They go on to say that the security procedure failed.

I take issue with that. The security procedure was carried out, but it was flawed from the beginning. The POLICY failed to take into account the risks associated with storage of official (and private) information away from work premises, in employee homes where the employer has no control or awareness of safekeeping practices.

Link to news article:

Breach of the week: Baghdad embassy plans posted on the Internet

Submit: Add to your Digg This Slashdot GotNews StumbledUpon Reddit

Excerpt from the AP story:

Detailed plans for the new U.S. Embassy under construction in Baghdad appeared online Thursday in a breach of the tight security surrounding the sensitive project.

Computer-generated projections of the soon-to-be completed, heavily fortified compound were posted on the Web site of the Kansas City, Mo.-based architectural firm that was contracted to design the massive facility in the Iraqi capital.

The images were removed by Berger Devine Yaeger Inc. shortly after the company was contacted by the State Department.

And now my thoughts:

Wow, how did THAT happen??!! There are really only two:

1. Mistake. Many possible scenarios. A goofed drag-and-drop that put sensitive info into another directory on the file server, which causes them to be automatically posted online. Or, someone restored data from backup, restored to much or into the wrong place. Or, someone picked the wrong directory from a list in web publishing software like Dreamweaver. The possibilities are endless. Bottom line: someone screwed up. Maybe.

2. Deliberate. Someone put the data there, knowing it was wrong to do so. Why? Revenge. Could be. If the company has poor access controls and logging, they may never know who did it.

Links to full story:

TSA loses hard drive containing data on 100,000 employees

Submit: Add to your Digg This Slashdot GotNews StumbledUpon Reddit

The Transportation Security Administration, the branch of the U.S. federal government responsible for airline passenger safety, has reported that a hard drive containing personal data on about 100,000 employees has been lost and remains unaccounted for.

TSA management sent a letter to TSA employees on Friday, informing them of the potential breach of security.

It has not been determined whether the hard drive was lost, or stolen.

The FBI and the U.S. Secret Service have been asked to investigate.

TSA’s website has a new entry informing employees about the matter and directs them to information about identity theft.

TSA Employee Security Incident

Rep. Sheila Jackson Lee, D-Texas, whose Homeland Security subcommittee oversees the TSA, has indicated her desire to hold hearings on the security breach. She stated that Homeland Security buildings are part of the critical infrastructure the agency is charged with protecting.

Stories on this incident:


This is another of a long series of mishaps in which computer-based data has been misplaced, lost, or stolen, resulting in the fear that such information may fall into the hands of identity thieves, resulting in costly and time consuming identity theft incidents that now plague millions of U.S. residents and millions more around the world.

No type of institution is immune to such incidents. Despite elaborate measures, sometimes we lose things. We started when we were children and we still struggle to keep track of our things. History is peppered with innumerable tales about lost and misplaced money, jewels, people, and objects of every size, shape, and type. I don’t think we’re going to solve this any time soon.

TJX intrusion was a WEP (WiFi) hack

Bookmark This (opens in new window)

A Wall St Journal article published today details the probable cause of the TJX credit card scandal I have commented on before. Today’s article confirms my suspicions: the perpetrators probably broke into TJX’s network by hacking into a retail store’s WiFi network in 2005 that was protected with WEP, a wireless protocol that was shown in 2001 to be too weak for commercial use. TJX was slow to adopt the newer WPA protocol (which I have urged people to switch to), must to its detriment as we now know.

As many as 200 MILLION cards may have been taken by the data thieves. Because the intruders left few tracks, and due to IT processes in place at the time, no one will ever know just how many cards were stolen. But so far it looks like 47.5 million is the minimum, and somewhere around 200 million is the theoretical maximum.

For an interesting account of the TJX breach, read their 10-K

Bookmark This (opens in new window)

TJX, parent corporation of TJ Maxx and notorious for the recent colossal credit card breach of 2006 (we weren’t told until 2007, but I digress), is a U.S. public company. As such, they are required to file a quarterly report to the SEC called a 10-K.

TJX’s 10-K provides a chilling account of the breach, from discovery through disclosure to law enforcement, and finally the public. It details what they know and don’t know.

Here is how the long narrative begins…


We suffered an unauthorized intrusion into portions of our computer systems that process and store information related to customer transactions that we believe resulted in the theft of customer data. We do not know who took this action and whether there were one or more intruders involved (we refer to the intruder or intruders collectively as the “Intruder”), or whether there was one continuing intrusion or multiple, separate intrusions (we refer to the intrusion or intrusions collectively as the “Computer Intrusion”). We are engaged in an ongoing investigation of the Computer Intrusion, and the information provided in this Form 10-K is based on the information we have learned in our investigation to the date of this Form 10-K. We do not know what, if any, additional information we will learn in our investigation, but that information could materially add to or change the information provided in this Form 10-K.

…the above contains some of the legalese (the terms “Intruder” and “Computer Intrusion” used throughout the report). The report continues with a description of the discovery…

Discovery of Computer Intrusion. On December 18, 2006, we learned of suspicious software on our computer systems. We immediately initiated an investigation, and the next day, General Dynamics Corporation and International Business Machines Corporation, leading computer security and incident response firms, were engaged to assist in the investigation. They determined on December 21, 2006 that there was strong reason to believe that our computer systems had been intruded upon and that an Intruder remained on our computer systems. With the assistance of our investigation team, we immediately began to design and implement a plan to monitor and contain the ongoing Computer Intrusion, protect customer data and strengthen the security of our computer systems against the ongoing Computer Intrusion and possible future attacks.

On December 22, 2006, we notified law enforcement officials of the suspected Computer Intrusion and later that day met with representatives of the U.S. Department of Justice, U.S. Secret Service and U.S. Attorney, Boston Office to brief them. At that meeting, the U.S. Secret Service advised us that disclosure of the suspected Computer Intrusion might impede their criminal investigation and requested that we maintain the confidentiality of the suspected Computer Intrusion until law enforcement determined that disclosure would no longer compromise the investigation.

With the assent of law enforcement, on December 26 and December 27, 2006, we notified our contracting banks and credit and debit card and check processing companies of the suspected Computer Intrusion (we refer to credit and debit cards as “payment cards”). On December 27, 2006, we first determined that customer information had apparently been stolen from our computer systems in the Computer Intrusion. On January 3, 2007, we, together with the U.S. Secret Service, met with our contracting banks and payment card and check processing companies to discuss the Computer Intrusion.

Prior to the public release of information with respect to the Computer Intrusion, we provided information on the Computer Intrusion to the U.S. Federal Trade Commission, U.S. Securities & Exchange Commission, Royal Canadian Mounted Police and Canadian Federal Privacy Commissioner. Upon the public release, we also provided information to the Massachusetts and other state Attorneys General, California Office of Privacy Protection, various Canadian Provincial Privacy Commissioners, the U.K. Information Commissioner, and the Metropolitan Police in London, England.

On January 13, 2007, we determined that additional customer information had apparently been stolen from our computer systems.

On January 17, 2007, we publicly announced the Computer Intrusion and thereafter we expanded our forensic investigation of the Computer Intrusion.

On February 18, 2007, in the course of our ongoing investigation, we found evidence that the Computer Intrusion may have been initiated earlier than previously reported and that additional customer information potentially had been stolen. On February 21, 2007, we publicly announced additional findings on the timing and scope of the Computer Intrusion.

…so that’s the timeline on the discovery. Their actions in terms of quickly involving law enforcement and banking at all levels is laudable and appropriate. Then again, they knew that this was a serious situation, and they’d be criticized for slow response later if they didn’t act quickly.

The report continues by describing the timeline of the intrusions:

Timing of Computer Intrusion. Based on our investigation to date, we believe that our computer systems were first accessed by an unauthorized Intruder in July 2005, on subsequent dates in 2005 and from mid-May 2006 to mid-January 2007, but that no customer data were stolen after December 18, 2006.

…short and to the point.

Next, TJX talks about the systems affected. This is where it gets interesting, because we get the impression that they aren’t really sure which systems were compromised, or from which systems data was stolen:

Systems Affected in the Computer Intrusion. We believe that information was stolen in the Computer Intrusion from a portion of our computer systems in Framingham, MA that processes and stores information related to payment card, check and unreceipted merchandise return transactions for customers of our T.J. Maxx, Marshalls, HomeGoods and A.J. Wright stores in the U.S. and Puerto Rico and our Winners and HomeSense stores in Canada (“Framingham system”) and from a portion of our computer systems in Watford, U.K. that processes and stores information related to payment card transactions at T.K. Maxx in the United Kingdom and Ireland (“Watford system”). We do not believe that the Computer Intrusion affected the portions of our computer systems handling transactions for customers of Bob’s Stores, or check and merchandise return transactions at T.K. Maxx. We do not believe that customer personal identification numbers (PINs) were compromised, because, before storage on the Framingham system, they are separately encrypted in U.S., Puerto Rican and Canadian stores at the PIN pad, and because we do not store PINs on the Watford system. We do not believe that information from transactions using debit cards issued by Canadian banks at Winners and HomeSense that were transacted through the Interac network was compromised. Although we believe that information from transactions at our U.S. stores (other than Bob’s Stores) using Canadian debit cards that were transacted through the NYCE network were processed and stored on the Framingham system, we do not believe the PINs required to use these Canadian debit cards were compromised in the Computer Intrusion. We do not process or store names or addresses on the Framingham system in connection with payment card or check transactions.

…we can speculate on the reasons why TJX doesn’t know which systems were affected. Could it be that the intruder(s) washed the audit logs, or accessed the data in a way that didn’t show up on audit logs?

There are also some hints appearing that suggest that TJX was not following PCI requirements in terms of what information may be stored and which may not. Note there they say “We do not believe that customer personal identification numbers (PINs) were compromised, because, before storage on the Framingham system…” !! It sounds like they were storing PINs on the “Framingham system” which is clearly a violation of PCI requirements. PIN must never be stored on a merchant system.

Next, the report describes the data that was stolen.

Customer Information Believed Stolen. We have sought to identify customer information stolen in the Computer Intrusion. To date, we have been able to identify only some of the information that we believe was stolen. Prior to discovery of the Computer Intrusion, we deleted in the ordinary course of business the contents of many files that we now believe were stolen. In addition, the technology used by the Intruder has, to date, made it impossible for us to determine the contents of most of the files we believe were stolen in 2006. Given the scale and geographic scope of our business and computer systems and the time frames involved in the Computer Intrusion, our investigation has required a substantial period of time to date and is not completed. We are continuing to try to identify information stolen in the Computer Intrusion through our investigation, but, other than the information provided below, we believe that we may never be able to identify much of the information believed stolen.

Based on our investigation, we have been able to determine some details about information processed and stored on the Framingham system and the Watford system. Customer names and addresses were not included with the payment card data believed stolen for any period, because we do not process or store that information on the Framingham system or Watford system in connection with payment card transactions. In addition, for transactions after September 2, 2003, we generally no longer stored on our Framingham system the security data included in the magnetic stripe on payment cards required for card present transactions (“track 2” data), because those data generally were masked (meaning permanently deleted and replaced with asterisks). Also, by April 3, 2006, our Framingham system generally also masked payment card PINs, some other portions of payment card transaction information, and some portions of check transaction information. For transactions after April 7, 2004 our Framingham system also generally began encrypting (meaning substituted characters for the actual characters using an encryption algorithm provided by our software vendor) all payment card and check transaction information. With respect to the Watford system, masking and encryption practices were generally implemented at various points in time for various portions of the payment card data.

Until discovery of the Computer Intrusion, we stored certain customer personal information on our Framingham system that we received in connection with returns of merchandise without receipts and in some check transactions in our U.S., Puerto Rican and Canadian stores (other than Bob’s Stores). In some cases, this personal information included drivers’ license, military and state identification numbers (referred to as “personal ID numbers”), together with related names and addresses, and in some of those cases, we believe those personal ID numbers were the same as the customers’ social security numbers. After April 7, 2004, we generally encrypted this personal information when stored on our Framingham system. We do not process or store information relating to check or merchandise return transactions or customer personal information on the Watford system.

…it is clear that much more than just credit card data was stolen. There were apparently many incidents of other information, including drivers’ license, military and state identification numbers, names and addresses, social security numbers, and perhaps more.

The report continues:

Information Believed Stolen in 2005. As we previously publicly reported, we believe customer data were stolen in September and November 2005 relating to a portion of the payment card transactions made at our stores in the U.S., Puerto Rico and Canada (excluding transactions at Bob’s Stores and transactions made at Winners and HomeSense through the Interac network with debit cards issued by Canadian banks) during the period from December 31, 2002 through June 28, 2004. We suspect the data believed stolen in 2005 related to somewhere between approximately half to substantially all of the transactions at U.S., Puerto Rican and Canadian stores during the period from December 31, 2002 through June 28, 2004 (excluding transactions at Bob’s Stores and transactions made at Winners and HomeSense through the Interac network with debit cards issued by Canadian banks). The data were included in files routinely created on our Framingham system to store customer data, but the contents of many of the files were deleted in the ordinary course of business prior to discovery of the Computer Intrusion.

…the report than shows a chart that indicates the number of cards compromised. I’ll summarize here:

Payment Card Status at Time of Believed Theft

Transactions from 12/31/02 – 11/23/03

Expired Cards: Track 2 data masked: 5,600,000 cards; All card data in the clear: 25,000,000 cards
Unexpired Cards: Track 2 data masked: 3,800,000 cards; All card data in the clear: 11,200,000 cards

Transactions from 11/24/03 – 6/28/04

Expired Cards: Track 2 data masked: unknown quantity; All card data in the clear: 0 cards
Unexpired Cards: Track 2 data masked: unknown quantity; All card data in the clear: 10 cards

The narrative continues:

Customer names and addresses and, for transactions after September 2, 2003, track 2 data were not included in the payment card information believed stolen in 2005. We do not believe that customer PINs were compromised.

In addition, we believe that personal information provided in connection with a portion of the unreceipted merchandise return transactions at T.J. Maxx, Marshalls, and HomeGoods stores in the U.S. and Puerto Rico, primarily during the last four months of 2003 and May and June 2004, was also stolen in 2005. The information we are able to specifically identify was from 2003 and included personal ID numbers, together with the related names and/or addresses, of approximately 451,000 individuals. We are in the process of notifying these individuals directly by letter.

TJX does not know how many records were stolen from 11/24/03 – 6/28/04 because they regularly purge data, and because they don’t know when the specific thefts took place, they do not know how many were taken.

They began encrypting card data on 4/7/04. Prior to that, according to the report, they either masked card data, or stored it all in the clear.

The report continues by describing data stolen in 2006:

Information Believed Stolen in 2006. As previously publicly reported, we identified a limited number of payment cards as to which transaction information was included in the customer data that we believe were stolen in 2006. This information was contained in two files apparently created in connection with computer systems problems in 2004 and 2006. Through our investigation to date, we have identified the following information with respect to the approximate number of payment cards for which unencrypted information was included in these files:

The report shows another table, a simpler one this time. In 2006, the numbers of cards that could have been stolen numbers in the tens of thousands, rather than in the millions. This suggests to me that TJM was more aggressively purging transaction data and keeping far less card data online than before.

Much more narrative follows:

Customer names and addresses were not included with the payment card information in these files. We do not believe that customer PINs were compromised. Some of the payment card data contained in these files were encrypted; we have not sought to decrypt these data.

In addition, the two files contained the personal ID numbers, together with the related names and/or addresses, of approximately 3,600 individuals, and we sent notice directly to these individuals.

We also have located a third file created in the ordinary course that we believe was stolen by the Intruder in 2006 and that we believe contained customer data. All of the data in this file are encrypted, and we have not sought to decrypt them.

As previously publicly reported, we believe that in 2006 the Intruder may also have stolen from our Framingham system additional payment card, check and unreceipted merchandise return information for transactions made in our stores in the U.S., Canada, and Puerto Rico (excluding transactions at Bob’s Stores and transactions made at Winners and HomeSense through the Interac network with debit cards issued by Canadian banks) during portions of mid-May through December 18, 2006. Through our investigation, we have identified approximately 100 files that we believe the Intruder, during this period, stole from our Framingham system (the vast majority of which we believe the Intruder created) and that we suspect included customer data. However, due to the technology utilized by the Intruder, we are unable to determine the nature or extent of information included in these files. Despite our masking and encryption practices on our Framingham system in 2006, the technology utilized in the Computer Intrusion during 2006 could have enabled the Intruder to steal payment card data from our Framingham system during the payment card issuer’s approval process, in which data (including the track 2 data) is transmitted to payment card issuer’s without encryption. Further, we believe that the Intruder had access to the decryption tool for the encryption software utilized by TJX. The approximately 100 files stolen in 2006 could have included the data that we believe were stolen in 2005, as well as other data relative to some customer transactions from December 31, 2002 through mid-May 2006, although, with respect to transactions after September 2, 2003 generally without track 2 data, and, with respect to transactions after April 7, 2004, generally with all data encrypted.

In addition, as previously publicly reported, we suspect that customer data for payment card transactions at T.K. Maxx stores in the U.K. and Ireland has been stolen. In that regard, we now believe that at least two files of the approximately 100 files identified above that the Intruder stole from the Framingham system in 2006 were created by the Intruder and moved from the Watford system to the Framingham system. We suspect that these files contained payment card transaction data, some or all of which could have been unencrypted and unmasked. However, due to the technology utilized by the Intruder in the Computer Intrusion, we are unable to determine the nature or extent of information included in these files. Further, the technology utilized by the Intruder in the Computer Intrusion during 2006 on the Watford system could also have enabled the Intruder to steal payment card data from the Watford system during the payment card issuer’s approval process, in which data (including the track 2 data) are transmitted to payment card issuer’s without encryption.

We have provided extensive payment card transaction information to the banks and payment card companies with which we contract as requested by them. While we have been advised by law enforcement authorities that they are investigating fraudulent use of payment card information believed stolen from TJX, we do not know the extent of any fraudulent use of such information. Some banks and payment card companies have advised us that they have found what they consider to be preliminary evidence of possible fraudulent use of payment card information that may have been stolen from us, but they have not shared with us the details of their preliminary findings. We also do not know the extent of any fraudulent use of any of the personal information believed stolen. Certain banks have sought, and other banks and payment card companies may seek, either directly against us or through claims against our acquiring banks as to which we may have an indemnity obligation, payment of or reimbursement for fraudulent card charges and operating expenses (such as costs of replacing and/or monitoring payment cards thought by them to have been placed at risk by the Computer Intrusion) that they believe they have incurred by reason of the Computer Intrusion. In addition, payment card companies and associations may seek to impose fines by reason of the Computer Intrusion.

The report, above, mentions several times “the technology utilized by the Intruder” without being more specific. In a 10-K, this terminology is appropriate. For the report to describe what SQL, ODBC, .NET, or command line interface was used to get the data would be far too much detail. Still, my professional curiosity is piqued. What technology *did* the intruder(s) use?

The portion of the 10-K report on the intrusion continues and concludes:

Financial Costs. In the fourth quarter of fiscal 2007, we recorded a pre-tax charge of approximately $5 million, or $.01 per share, for costs incurred through the fourth quarter in connection with the Computer Intrusion, which includes costs incurred to investigate and contain the Computer Intrusion, strengthen computer security and systems, and communicate with customers, as well as technical, legal, and other fees. Beyond this charge, we do not have enough information to reasonably estimate losses we may incur arising from the Computer Intrusion. Various litigation has been or may be filed, and various claims have been or may be otherwise asserted, against us and/or our acquiring banks, on behalf of customers, banks, and/or card companies seeking damages allegedly arising out of the Computer Intrusion and other related relief. We intend to defend such litigation and claims vigorously, although we cannot predict the outcome of such litigation and claims. Various governmental entities are investigating the Computer Intrusion, and although we are cooperating in such investigations, we may be subject to fines or other obligations. (See Item 3 with respect to litigation and investigations.) Losses that we may incur as a result of the Computer Intrusion include losses arising out of claims by payment card associations and banks, customers, shareholders, governmental entities and others; technical, legal, computer systems and other expenses; and other potential liabilities, costs and expenses. Such losses could be material to our results of operation and financial condition.

Above, the report mentions costs associated with strengthening computer security and systems. Are these costs associated with bringing systems up to PCI standards, or beyond them? The report is not clear on this point.

Future Actions. We are continuing our forensic investigation of the Computer Intrusion and our ongoing program to strengthen and protect our computer systems. We are continuing to communicate with our customers about the Computer Intrusion. We are continuing to cooperate with law enforcement in its investigation of these crimes and with the payment card companies and associations and our acquiring banks. We are also continuing to cooperate with governmental agencies in their investigations of the Computer Intrusion. We are vigorously defending the litigation and claims asserted against us with respect to the Computer Intrusion.

TJX may suffer more losses over the years and they may be material. Well, that’s a reasonable supposition. The TJX intrusion is a watershed event to be sure, and could result in lawsuits the likes of which we haven’t seen in the past, as well as new legislation related to protection of financial information and remedies for failures.

Update: intruders probably broke in through WiFi by breaking WEP. More here.