Poor Quality Information costs money – and you can take that to the Bank

Leave a reply

via Keith Underdown comes this story hot off the press release engine in the UK’s Financial Services Authority.

Barclay’s Bank have been fined stg£2.45 million for “failing to provide accurate transaction reports to the FSA and for serious weaknesses in systems and controls in relation to transaction reporting”. The fine would have been higher (stg£3.5 million but for the fact that Barclay’s co-operated with the investigation and agreed to settle the matter quickly). According the the FSA press release

“Complete and accurate transaction reports are an essential component of the FSA’s market monitoring work. Barclays’ reporting failures could have a damaging impact on our ability to detect and investigate suspected market abuse.

“The penalty imposed on Barclays is significantly higher than previous penalties imposed for transaction reporting errors. This reflects the serious nature of Barclays’ breaches and is a warning to other firms that the FSA will not tolerate inadequate systems and controls.”

This is an interesting warning that serves as an indicator of how Regulatory systems and enforcement will be changing in a post-Recession world.

Of course, the true cost to Barclays is much greater than the cost of the fine. For one, there is the reputational damage that comes from being fined to this extent by the FSA. And then there is the cost of correcting errors and fixing the defective processes, which Barclays has done “including commissioning a review of its transaction reporting process and committing extensive resources to improve its processes and resolve the errors.”

Reviews and resources don’t grow on trees you know.

This is a clear example of an IQ Trainwreck.

Did you check on the cheques we sent to County Jail?

Leave a reply

Courtesy of Keith Underdown comes yet another classic IQ Trainwreck  which he came across on the CBS News.

It seems that up to 3900 prisoners received cheques (or ‘checks’ to our North American readers) of US$250 each, despite the very low probability that they would be able to actually use them to stimulate the economy. Of the 3900, 2200 were, it seems, entitled to receive them as they had not been incarcerated in any one of the three months prior to the enactment of the Stimulus bill.

However, that still leaves 1700 prisoners who should not have received cheques who did. The root cause?

According to CBS News:

…government records didn’t accurately show they were in prison

A classic information quality problem… accuracy of master data being used in a process resulting in an unexpected or undesired outcome.

While most prisons have intercepted and returned the cheques, there will now need to be a process to identify,  for each prisoner, whether the Recovery payment was actually due. Again, a necessary manual check (no pun intended) at this stage but one which will add to the cost and time involved in processing the Recovery cheques.

Of course, we’ve already written here about the problem with Stimulus cheques being sent to deceased people.

These cases highlight the fact that an Information Quality problem doesn’t have to be massively impacting on your bottom line or impact significant numbers of people to have an impact on your reputation.

US Government Health (S)Care.

2 Replies

Courtesy of Jim Harris at the excellent OCDQBlog.com comes this classic example of a real life Information Quality Trainwreck concerning US Healthcare. Keith Underdown also sent us the link to the story on USAToday’s site

It seems that 1800 US military veterans have recently been sent letters informing them that they have the degenerative neurological disease ALS (a condition similar to that which physicist Stephen Hawking has).

At least some of the letters, it turns out, were sent in error.

[From the LA Times]

As a result of the panic the letters caused, the agency plans to create a more rigorous screening process for its notification letters and is offering to reimburse veterans for medical expenses incurred as a result of the letters.

“That’s the least they can do,” said former Air Force reservist Gale Reid in Montgomery, Ala. She racked up more than $3,000 in bills for medical tests last week to get a second opinion. Her civilian doctor concluded she did not have ALS, also known as Lou Gehrig’s disease.

So, poor quality information entered a process, resulting in incorrect decisions, distressing communications, and additional costs to individuals and governement agencies. Yes. This is ticking all the boxes to be an IQ Trainwreck.

The LA Times reports that the Department of Veterans Affairs estimates that 600 letters were sent to people who did not have ALS. That is a 33% error rate. The cause of the error? According to the USA Today story:

Jim Bunker, president of the National Gulf War Resource Center, said VA officials told him the letters dated Aug. 12 were the result of a computer coding error that mistakenly labeled the veterans with amyotrophic lateral sclerosis, or ALS.

Oh. A coding error on medical data. We have never seen that before on IQTrainwrecks.com in relation to private health insurer/HMO data. Gosh no.

Given the impact that a diagnosis of an illness which kills affected people within an average of 5 years can have on people, the simple coding error has been bumped up to a classic IQTrainwreck.

There are actually two Information quality issues at play here however which illustrate one of the common problems in convincing people that there is an information quality problem in the first place . While the VA now estimates (and I put that in bold for a reason) that the error rate was 600 out of 1800, the LA Times reporting tells us that:

… the VA has increased its estimate on the number of veterans who received the letters in error. Earlier this week, it refuted a Gulf War veterans group’s estimate of 1,200, saying the agency had been contacted by fewer than 10 veterans who had been wrongly notified.

So, the range estimates for error goes from 10 in1800 (1.8%) to 600 in 1800 (33%) to 1200 in 1800 (66%). The intersting thing for me as an information quality practitioner is that the VA’s initial estimate was based on the numberof people who had contacted the agency.

This is an important lesson.. the number of reported errors (anecdotes) may be less than the number of actual errors and the only real way to know is to examine the quality of the data and look for evidence of errors and inconsistency so you can Act on Fact.

The positive news… the VA is changing its procedures. The bad news about that… it looks like they are investing money in inspecting defects out of the process rather than making sure the correct fact is correctly coded in patient records.

A cautionary tale of GPS woes

2 Replies

From today’s Irish Times comes a story which shows the real significance and impact of a common Information/Data Quality problem, transposition of letters or numbers.

A Swedish couple holidaying in Italy were looking forward to their visit to the lovely sunny island of  Capri.

Unfortunately a “finger flub” on their GPS put them 650 kilometres north and inland of their intended destination in the lovely Italian industrial town of Carpi.

Oh dear.

No child left behind (except for those that are)

Leave a reply

Steve Sarsfield shares with us this classic tale of IQ Trainwreck-ry from Atlanta Georgia.

An analysis of student enrollment and transfer data carried out by the Atlanta Journal-Constitution reveals a shocking number of students who appear to be dropping out of school and off the radar in Georgia.  This suggests that the dropout rate may be higher and the graduation rate lower than previously reported.

Last year, school staff marked more than 25,000 students as transferring to other Georgia public schools, but no school reported them as transferring in, the AJC’s analysis of enrollment data shows.

Analysis carried out by the State agency responsible was able to track down some of the missing students. But poor quality information makes any further tracking problematic if not impossible.

That search located 7,100 of the missing transfers in Georgia schools, state education spokesman Dana Tofig wrote in an e-mailed statement. The state does not know where an additional 19,500 went, but believes other coding errors occurred, he wrote. Some are dropouts but others are not, he said.

In a comment which should warm the hearts of Information Quality professionals everywhere, Cathy Henson, a Georgia State education law professor and former state board of education chairwoman says:

“Garbage in, garbage out.  We’re never going to solve our problems unless we have good data to drive our decisions.”

She might be interested in reading more on just that topic in Tom Redman’s book “Data Driven”.

Drop out rates consitute a significant IQ Trainwreck because:

  • Children who should be helped to better education aren’t. (They get left behind)
  • Schools are measured against Federal Standards, including drop out rates, which can affect funding
  • Political and business leaders often rely on these statistics for decision making, publicity,  and campaigning.
  • Companies consider the drop out rate when planning to locate in Georgia or elsewhere as it is an indicator of future skills pools in the area.

The article quotes Bob Wise on the implications of trying to fudge the data that sums up the impact of masking drop outs by miscoding (by accident or design):

“Entering rosy data won’t get you a bed of roses,” Wise said. “In a state like Georgia that is increasingly technologically oriented, it will get you a group of people that won’t be able to function meaningfully in the workforce.”

The article goes on to highlight yet more knockon impacts from the crummy data and poor quality information that the study showed:

  • Federal standard formulae for calculation of dropouts won’t give an accurate figure if there is mis-coding of students as “transfers” from one school to another.
  • A much touted unique student identifier has been found to be less than unique, with students often being given a new identifier in their new school
  • Inconsistencies exist in other data, for example students who were reported “removed for non-attendance” but had  zero absent days recorded against them.

Given the impact on students, the implications for school rankings and funding, the costs of correcting errors, and the scale and extent of problems uncovered, this counts as a classic IQTrainwreck.

The terror of the Terrorist Watch list

Leave a reply

A source who wishes to remain anoynymous sent us this link to a story on Wired.com about the state of the US Government’s Terrorist watch list.

The many and varied problems with the watch list have been covered on this blog before.

However, the reason that this most recent story constitutes an IQTrainwreck is that it seems that, despite undertakings to improve quality, the exact opposite has actually happened given:

  • The growth in the number of entries on the list
  • The failures on the part of the FBI to properly maintain and update information in a timely manner.

According to the report 15% of active terrorism suspects under investigation were not added to the Watch list. 72% of people cleared in closed investigations were not removed.

The report from the US Inspector General said that they “believe that the FBI’s failure to consistently nominate subjects of international and domestic terrorism investigations to the terrorist watchlist could pose a risk to national security.”

That quote sums up why this is an IQTrainwreck.

Continue reading →

Irish State Exam leak being studied.

Leave a reply

A serious complication has emerged in Leaving Certificate exams run by the Irish State each year.. An exam Superindtendent accidentally distributed the wrong paper in one exam centre earlier this week. He put out the exam questions for Paper 2 of the English examination, which wasn’t the subject being examined. The paper was, it seems, only on students’ desks for a few minutes before the error was noticed. However, in this age of twitter, bebo, myspace, facebook and such things, details of the exam questions were soon being discussed in school yards the length and breadth of the country. 

To make matters worse, the Superintendent in question failed to notify the Department of Education until more than 6 hours after the paper was leaked.

As with all things governmental, an investigation is underway. Denials of responsibility have issued from various entities involved.  The Superintendent in question has been dismissed. The exam is being rescheduled, causing disruption to study timetables across the land.

But an examination (no pun intended) of the facts reveals a telling IQ Trainwreck.

One of the factors that determines the quality of information is the quality of information presentation. Indtroducing ambiguity into visual information invites error. Tom Redman, in his book Data Driven, describes the presentation of information as a key step in how information is used and a key part of its complexity. Redman tells us that a number of disciplines need to come together to make even the simplest information and data useful, including:

Presenting data in ways that make it easy for customers to understand and use them. Only in this last step do data and information contribute to internal operations and decisions…

Packaging two sets of highly sensitive information in highly similar packaging which is similar enough that a warning is required makes it hard for customers (Redman uses “customers” to mean the actual consumers of the information – in this case the Superintendent) invites misunderstanding and error.

Yes, the Superintendent could have and should have double checked the paper was the right paper before handing it out, but a key contributing cause was the use of overly similar packaging for both exams.

  • The Superintendent didn’t report a leak of sensitive information in a timely manner

All too often this happens in business. A laptop gets stolen, a memory stick gets mislaid, sensitive information gets left on a train. A key element of the response to this kind of problem is knowing that there is actually a problem, so early reporting to authorities of the leak is imperative. Had the State Examinations Commission had the information in a timely manner perhaps the cost of fixing the gaffe would be less.

  • The cost of remedying the issue is now put at approximately EUR 1 Million

The solution that the Department of Education and State Examinations Commission has come up with is to run a totally new exam paper on Saturday. That means:

  • Extra costs for transport for students to the exams (where State-funded school transport is used)
  • Extra salary costs for Superintendents and their assistants
  • Extra salary costs for school staff who are required to be on-site during exams.
  • The costs of printing a whole new batch of exam papers.

And of course, it being a Saturday:

The SEC is finalising arrangements for a deferred sitting of English papers for a small number of students from the Jewish community after getting legal advice that asking them to sit an exam on their Sabbath, when their religion prohibits it and it is against their conscience, could have been unconstitutional. All other students will be expected to attend, in line with other timetabled exams.

For more on that particular complication, see the Irish Times’ detailed story.

So, why is this an IQTrainwreck?

  1. The similarity in packaging on the exam papers was a key root cause. This is (or should be) a straightforward process of ensuring that all exam subjects and levels are distinctly colour coded and ensuring that packaging is not similar. Issuing a reminder is simply trying to inspect a defect out of the process. Yes, the Superintendent has to carry responsibility as well for not double checking but avoidable similarity should have been avoided (ergo preventing the confusion)
  2. The lack of rigour regarding the reporting of the accidental distribution of the wrong paper is inexcusable. 
  3. The cost impact of the error is extremely significant, particularly given the current state of Irish Government finances. EUR1 Million is a challenging amount to find in your budget at short notice.
  4. The disruptive impact on students during a stressful time can’t be underestimated. 
  5. The further complication presented by Ireland’s multi-culturalism adds further challenges (and potentially costs) for the SEC, the Department, and the students.

(On that note of multiculturalism, one is left wondering if the ISM school in Tripoli, Libya that offers the Irish Leaving Certificate to its students will have received their replacement exam papers yet of if they are even aware of the issue.)

Information Quality Problems in Danish EU Elections

2 Replies

In a story that is bound to bring tears to the eyes of at least one regular contributor to on-line discussions about information quality, news reaches us from Jan Erik Ingvaldsen of a series of information quality disasters in the Danish EU elections which take place on Sunday.

Denmark is also holding a referendum on the same day, which some are viewing as a root cause for these problems.

Here is the story in Danish (big thank you to Jan Erik for this link via twitter)

Here is a rough translation in Google English. As google’s machine translations are never 100% accurate, we’d welcome any links to this story in English.

The problems (as we can make them out from the translation)

  • In Ikast-Brande, a municipality of 1700 people, voters received polling cards directing them to the wrong polling stations (wrong people, wrong ballot boxes)
  • In Stvens, the polling station is correct (right people going to the right ballot box), but the address given for the polling station is wrong.
  • In Vejle the wrong zip code was included in the map of the electoral district (i.e. people entitled to vote in a given polling station)
  • 2700 voters in three constituencies have been instructed to vote in two different locations
  • Some voters who are entitled to vote in both the Referendum and the EU Parliament elections have only received polling cards for one of the two ballots
  • Some postal voters report that they have  not received any information on the candidates running in the area their vote is being counted in.

Danish speaker – please correct my listing of the problem (admitting your problems is the first step on the road to recovery).

Jan Erik Ingvaldsen has a great post which summarises the problems in English.

Despite all of this, some elections commentators say that the preparations for the election are good, and they will sort out the problems when the complaints come in. Hanging Chads anyone?

While the fact that the Danes are running an EU election and a referendum on the same day, with different voter eligibility rules for each, goes some way to explaining the challenges that might have contributed to these problems, the defence is weakened by the fact that Ireland is running two ballots as well this week for EU Parliament elections and Local Government Councils and has not had reports of similar problems (yet).

Of course, the problem in Ireland is that our electoral register is wonderfully inaccurate.

For the impact that poor quality information can have on democratic processes, the view that the errors and impact can be “inspected out”, and because investigating this story made me have to figure out Google’s attempts at translation, this is being classed as a definite IQ Trainwreck.

Also, as Jan Erik points out in his blog, if this was happening in a South American or African nation there would be widespread media outcry.

Thanks to Jan Erik  (@jeric40 on twitter) for flagging this one to our attention.

I am not a number – I’m a human being!

Leave a reply

Information Quality professionals (and indeed quality management professionals in general) often recite a mantra that good quality begins at the beginning of a process, that it must be designed in, and that defects need to be fixed as close to the start of the information chain as possible.

A post today on DataQualityPro.com from Dylan Jones highlights the significant truth that lies behind all these statements.

Dylan’s son was born in April. The first thing the State did for him was to slap an identifier on him. The second thing they did (to summarise Dylan’s excellent and forensic post) was to make a mess of linking the local hospital ID to a National patient record.

That error propogated and resulted in the parents of another child over 90 miles away getting an appointment for a medical checkup relating to Dylan’s son. It seems that the efforts made to correct the error Dylan spotted when his son was born haven’t propogated half as fast as the original error.

And that’s the problem. How many other processes and silo’d systems has this error propogated into? How many more times in Dylan’s son’s life will be be confused with another child 90 miles away? What other ‘life-events’ will this error impact? In future, how will he find himself trapped by his number?

Ultimately, Dylan’s son is not a number, he’s a human being.

We recently posted a long Trainwreck on the problems with Google Health due to poor quality information. It is possible that an error like the one affecting Dylan’s son could result in incorrect patient data about Dylan’s son being transferred to this type of electronic patient record. Who would be responsible for the impacts if that information was acted on in haste without Dylan (or Mrs Jones) being there to point out that the information was wrong?

Given that in an Irish hospital in 2003 the medical staff failed to act on an error in an expectant mother’s chart and delivered a baby 39 days prematurely, despite the parent’s insistence that there were errors in the chart, highlights that simple errors in medical records can have significant impacts. That the baby in question died further highlights that these impacts can be catastrophic, which means that the standard of care for quality in medical records needs to be high.

Dylan’s investigations also uncovered some other weaknesses/flaws in patient data quality which are unsettling. We’d suggest you take a look at Dylan’s post for more details on those.

For actual inconvenience and annoyance to Dylan’s family, and for the potential for catastrophic loss or injury, this counts as a definite IQTrainwreck.

Double Debits – directly. (Another banking IQTrainwreck)

6 Replies

Courtesy of our Irish colleagues over on Tuppenceworth.ie comes yet another tale of poor quality information in financial services. Although this time it is at the lower end of the scale, at least on a per customer basis. However, the impacts on a customer are still irksome and problematic. And the solution the bank has put in place is a classic example of why inspecting defects out of a process is never an exact or value adding science.

It seems that Bank of Ireland has recently introduced some new software. Unfortunately, a bug in the software has resulted in certain transactions (deductions) being posted multiple times to accounts, resulting in cash-strapped Irish people being more strapped for cash than they’d expected.

Simon McGarr, (one of the authors over at Tuppenceworth) sums up the story and the reason why this is an IQTrainwreck:

I spotted a double charge on my account, for a pretty significant sum of money (is there any other kind?).

When I rang up to query it, I was told Bank of Ireland have changed their computer systems recently (Two weeks or so).

As a result, some transactions are being applied to accounts twice if they were processed through Laser [a debit card system in Ireland — ed.], or if they were a Pass machine [what the Irish call ATMs –ed.] withdrawal.

They say that if you spot the double charge, and ring them up to complain, they’ll send an email to their programmers to reverse the second charge.

I suggested to the polite customer services person that the bank might want to warn their clients to be alert for these double charges, as they could suffer additional charges (from appearing to breach their overdraft limits, for example) unless they spotted the bank’s mistake.

(Emphasis is added by this author)

Simon goes on to add (in a comment) that he has been without the benefit of his hard earned cash for 10 days (and counting).

Continue reading →