Spectrum Online—Tomorrows Technology Today
Font Size: A A A

« December 2007 | Main | February 2008 »

January 2008 Archives

January 2, 2008

2007 Bad Year for Privacy: 2008 Worse?

The Washington Post had two stories on data security and privacy today. The first concerns a report by the Identity Theft Resource Center that more than 79 million records were reportedly compromised in the United States through December 18th, compared with nearly 20 million records reported in all of 2006.

The story also reported that Attrition.org estimates that more than 162 million records were compromised worldwide through December 21, compared with 49 million last year.

The number of data breaches has grown because there are more legal requirements on companies and governments to report them, but the number reported is also low since not everyone is required to or reports data breaches even when they should. As I have written about earlier, the UK government is just now owning up to a large number of data breaches that occurred months ago.

The other Post story concerns how easy it is to find a person's social security number on the web because local and state governments routinely post public records containing them. The Federal government has banned the publication of sensitive personal information like social security numbers since 2001. More recently states like Virginia and Maryland have also banned their publication as well. However, the law does not cover the hundreds of thousands of documents already published that contain social security numbers and that are accessible on-line. In Virginia, the law also doesn't seem to cover current arrest warrants or court summons.

So, as we begin this new year, anyone care to speculate on the date of the first major (let's say 1 million or more records) of the year in the US? Elsewhere in the world? And how long it takes from breach to disclosure of the breach?

Checking Grades On-line

Chalkboard.gif
The Chicago Tribune had a story on the increasing use of electronic grade books by Chicago area schools that both students and parents can access on-line. These accessible grade books started in the high schools, but are now migrating to middle and elementary schools. The idea is to create a tighter link between schools and the home, but some teens view it as an intrusion.

As a parent, I have mixed feelings. I know growing up I would not have been thrilled about my parents having access to every grade I received on every assignment each day, but as a parent I am interested in knowing where my children are having difficulties as well as excelling. Our school district doesn't have electronic grading yet, and so I see my children's homework once a week when they bring home a large folder with all of it stuffed in there.

The story doesn't talk about it, but I suspect that some teachers aren't thrilled to death about electronic grade books either, as I am sure many parents let the teacher know when they think their child's grade is too low. The helicopter parent problem can't be helped by it.

In one way, I am glad our school district doesn't have electronic grading, since I would also would want to know not only the grade but the assignment. This would inevitably lead to trouble, since whenever I found that an instructors made an error (like when one teacher was trying to teach my daughter that copper is naturally magnetic since a "copper fastener" was attracted to a magnet) I would have a hard time letting it slip.


UK Doctors Don't Trust NPfIT Security

Ear-examiner.gif
The London Times reported over the weekend that a poll it conducted sowed that more than three quarters of National Health Service (NHS) doctors, "are either 'not confident' that [patient] data will be safe or 'very worried' that data will leak once the £20 billion National Programme for IT (NPfIT) is running. Asked how well they thought that local NHS organisations would be able to maintain the privacy of data, only 4 per cent said very well. The majority, 57 per cent, said quite or very poorly."

Interestingly, the more experienced the doctor in IT, the less confident they are that the benefits of the NHS electronic health record system out weigh the risks to patient privacy.

In the London Telegraph, there are also two stories about the NHS changing how it plans to do business. The first is about a plan for millions of people suffering from "arthritis, asthma and even heart failure will be urged to treat themselves," as a means to save money. Some patients will be encouraged to report "medical information to doctors remotely by telephone or computer," which I assume will mean a big change to what will need to be captured in the NPfIT electronic health record.

The second story is also about Prime Minister Gordon Brown's desire to make people responsible for their own health, by denying medical treatment to patients that are deemed not to be taking care of themselves. The story says that, "Patients could be required to stop smoking, take exercise or lose weight before they can be treated."

Hmm, once the NPfIT is all in place, this should be easy to do. The government will be able to set up filters based on a person's medical history, and deny them access to treatments.

Brown says that, "I believe these are steps vital to securing the health of the NHS for the next 60 years."

"They will require a broadening and a deepening of reform to ensure that the NHS as a whole attaches the same priority to a personal and ­preventative service as many of you already reflect in your own day-to-day decisions."

In other words, the NHS will be there to treat you as long as you are already healthy.

I think UK doctors' might want to worry about patient privacy a bit more.

January 3, 2008

Software Problems Ring In the New Year on Schedule

fireworks.gif A bad computer file forced the New Year's Eve fireworks display in Seattle to be launched manually, resulting in a show that was out of sync with its choreographed music, according to a report in ComputerWorld.

Then, right after midnight, a software problem affected the Verizon wireless network in the Washington, DC area into early New Year's day, reports the Washington Post.

Next Intuit had to announce yesterday that the "permanent patch" for a bug in QuickBooks on the Mac that erased files from users' hard drives that was released on 31 December, does not in fact completely fix the flaw, reports ComputerWorld.

Finally, some Seminole County Florida residents opened their water bills for December 2007 and found bills for both December 2007 and 2006. As the Orlando Sentinel explains, "billing information from December 2006 was not purged from the [county's automated computer-billing] computer memory, so the system generated a bill based on that information."

Glad to see that 2008 is looking a lot like 2007 in the IS&T department, or as The Who would say, "Meet the new boss, same as the old boss."

Census Program in Serious Trouble?

Census.gif
In a story released this morning, Government Executive magazine reports that a MITRE Corporation talking paper implies that the Census Bureau's hand-held computer project (Field Data Collection Automation) on which the 2010 Census depends is in serious trouble. The talking paper states that the project is quickly running out of time, and that end-to-end system testing might be seriously affected.

In response the the issues raised by the MITRE paper, the Census seems to be taking a "failure is not an option" approach, although given what is in the talking paper, I definitely think it is a possibility.

January 4, 2008

Sad News

Tully.gif I just found out today that Middlesex University Emeritus Professor Colin Tully, an influential British computer scientist and a long-time friend, passed away on the 26th of December.

Colin and I first met over twenty years ago when I used to live in the UK, and we had many a lively discussion on information systems and technology risks and their management over the intervening years. Our latest conversations, the last of which was just a few weeks ago, have been about the risks surrounding the NHS National Program for IT (NPfIT), which Colin was deeply concerned about.

Colin will be missed in the software engineering community.

Does This Story Sound Familiar?

The Minneapolis Star Tribune reported earlier this week that a scheme by the Minneapolis' convention bureau to make money developing and selling software for booking and managing conventions has gone slightly awry.

According to the paper, the Minneapolis City Council lent the convention bureau $2.5 million in 2004 to create the convention booking software with the idea that it would help pay for the costs of marketing the city to tourists. In fact, the bureau told the City Council that it only expected to use $1.5 million of the $2.5 million loan to develop the software.

But "a series of unexpected setbacks involving the technical work of developing the software" in 2005 led the the bureau to asked for (and get) another $2.5 million in loans to complete it.

But then in 2006, the bureau said it needed to borrow up to another $5 million to complete the software and market it. Surprisingly - or maybe not - the convention bureau got its loans because it was able to convince the City Council that the opportunity was both "tremendous" and the software was nearly finished (95% complete?). Council Member Paul Ostrow declared at the time: "We're quite confident that this is relatively low-risk."

Well, fast forward to the end of 2007, and so far, the bureau has spent $9.1 million and sales of the software, while now apparently complete (it is called the Internet Destination Sales System (IDSS)), hasn't yet turned a profit. At least $1.7 million in sales per annum are needed to just break even.

In addition, the financial resources devoted to the effort has taken money away from marketing Minneapolis to tourists - a bit ironic to say the least.

The City Council finally wised up and stopped approving any more money to the project. In addition, the person who originally sold the idea to the City Council has also decided to "move on."

According to one current City Council member, "The customers who have the product are very pleased with it. As are we. From that standpoint, it's turned out to be a great product. Time will tell how the business model is going to work." The story didn't say how many customers are using the software, but I suspect the sales price is considerably higher and the potential market size is much smaller than forecast back in 2004.

The convention bureau thinks it can make a profit on selling the software this year, while others think 2009 at the earliest. I'll keep an eye on this story and let you know how it turns out.

January 5, 2008

UK Government Wastes Nearly £2 billion on Abandoned IT Projects Since 2000

Flame.gif
The London Guardian today published an analysis which found over £1,865bn has been spent on UK government IT projects that have been abandoned since 2000. As the newspaper notes, its "survey of abandoned projects is not exhaustive and the total of £1,865bn is likely to be a considerable underestimate of the actual cost to taxpayers because neither Whitehall nor the National Audit Office, parliament's financial watchdog, keep definitive lists of which schemes go wrong."

To say that it is an underestimate is itself an understatement, if Joe Harley, chief information officer at the Department for Work and Pensions (and former ICI Paints global chief information officer) is to believed. In May 2007, he said at the Turning the Tide Government UK IT Summit 2007 that, "Today only 30%, we estimate, of our projects and programmes are successful. It is not sustainable for us as a government to continue to spend at these levels. We need to up the quality of what we do at a reduced cost of doing so."

Given that the UK has spent nearly £100bn on IT since 2000, and 70% are not considered successful, I seriously doubt that only about 3% of those unsuccessful IT programs (no matter how you define success) were abandoned.

The UK government has gone on record as saying it wants a 90% plus IT success rate - meaning on time, on budget and to specification - by 2010/11.

To "Turn the IT Tide," assuming a uniform improvement rate of 15% and using the end of 2011 as the target date, then by the end of 2008, we can expect something like 45% of UK government IT programs to be successful. I'll be watching.

January 6, 2008

LAUSD Payroll Fiasco Sparks Call for Payback

Bus-1.gif
As I noted a few weeks back, the Los Angles Unified School District (LAUSD) now admits that its botched and blundered payroll system will likely cost upwards of $210 million when all is said and done. LAUSD school officials have said that they are considering litigation against the contractor, and have spent some $700,000 on two law firms to look into the matter, according to today's LA Daily News.

However, the Daily News says that California State Assemblyman Kevin de Leon is unhappy about what he sees as foot dragging by the LAUSD in pursuing damages, will introduce Assembly Bill 730 on 15 January, which would prevent any contractor found by a court liable for breach of an information-technology contract worth more than $1 million - and the judgment is greater than $250,000 - from bidding on any new business with the state or any local government for five years.

This might be an interesting approach that all government may want to consider, although $1 million may be a bit too low a threshold. I do suggest, however, the money spent on lawyers be considered as part of the overrun amount.

Porn Websites Get Hacked

The Boston Globe reported over the weekend that some porn sites have been hacked over the past few months. While it doesn't appear that credit card details have been compromised, personal information including email addresses of tens of thousands of customers appear to have been stolen.

This news has sent a wave of fear across the on-line adult entertainment business, since privacy is supposed to be guaranteed and is a sine qua non of the business. If hacking of adult web sites becomes "routine," the business might take a major financial hit.

Hmm, I wonder if this news will get some opposed to these sites to thinking.

AMT Patch Delays Tax Refunds as Expected

The IRS announced last week that because of the Congressional delay in the passage of the Alternative Minimum Tax (AMT) patch, it will take at least until the 13th of February before the AMT tax forms can be properly processed. According to Federal Computer Weekly, 13.5 million taxpayers who will use any of the five forms related to the AMT legislation will need to wait until then to begin submitting their returns, said Acting IRS Commissioner Linda Stiff.

Taxpayers affected are those using: Form 8863, Education Credits; Form 5695, Residential Energy Credits; Form 1040A's Schedule 2, Child and Dependent Care Expenses for Form 1040A Filers; Form 8396, Mortgage Interest Credit; and Form 8859, District of Columbia First-Time Homebuyer Credit. Once the tax forms are processed, refunds if any will take another 10 to 14 days to be sent out.

January 7, 2008

FAA to Boeing: Please Show that 787 Dreamliner Can't be Hacked

In a story that appeared last week in Flight International and then got legs via Wired, the Federal Aviation Administration (FAA) is going to require Boeing "to demonstrate that certain 787 flight critical domains - digital systems and networks that for the first time will be accessible externally via wireless and other links to airline operations and maintenance systems - cannot be tampered with."

The FAA Special Conditions Notice [Docket No. NM364 Special Conditions No. 25-356-SC] effective 1 February 2008 summary states:

"These special conditions are issued for the Boeing Model 787-8 airplane. This airplane will have novel or unusual design features when compared to the state of technology envisioned in the airworthiness standards for transport category airplanes. These novel or unusual design features are associated with connectivity of the passenger domain computer systems to the airplane critical systems and data networks. For these design features, the applicable airworthiness regulations do not contain adequate or appropriate safety standards for protection and security of airplane systems and data networks against unauthorized access. These special conditions contain the additional safety standards that the Administrator considers necessary to establish a level of safety equivalent to that established by the existing standards."

Continue reading "FAA to Boeing: Please Show that 787 Dreamliner Can't be Hacked" »

Declining Population? No Problem: Build Robots

Robot.gif
In this morning's Washington Post, there is an interesting story on Japan's declining population. According to the Post, "population shrinkage began [in Japan] three years ago and is gathering pace. Within 50 years, the population, now 127 million, will fall by a third, the government projects. Within a century, two-thirds of the population will be gone. That would leave Japan, now the world's second-largest economy, with about 42 million people."

Rather than open its doors to immigrants or encourage larger families, the Japanese government in partnership with industry has decided to turn to robots.

According to the Post's story, Japanese engineers say it's "service robots" that can "bail out Japan, which has the world's largest proportion of residents over 65 and smallest proportion of children under 15. One such gizmo, on display at the [Great Robot Exhibition in Tokyo's National Museum of Nature and Science], can spoon-feed the elderly. Others are being designed to hoist them onto a toilet and phone a nurse when they won't take their pills."

Some critics in Japan, however, call the "robot cure for an aging society as little more than high-tech quackery."

I wonder if the future Japanese population demographics include the wide-spread creation of sex robots?

January 8, 2008

Congress Poses a "Challenge" to the Census Bureau

Census.gif
A few days ago, I wrote about the Government Executive story about the problems that the Census Bureau appears to be having with its plans to use hand-held computers instead of paper to conduct the 2010 census.

Well, the story sparked more than a bit of Congressional interest. Representatives Tom Davis (R -Virginia) and Mike Turner (R-Ohio) sent a letter (posted at Federal Computer Week who have a story on the letter) yesterday to their Democratic counter-parts Representatives Henry Waxman (D-California) and William Clay (D-Missouri) to schedule a hearing of the Committee on Oversight and Government Reform Committee to look into issues raised by MITRE, especially the allegation that the hand-held project is in "serious trouble" and a contingency plan is required.

What really seems to have irritated Davis and Turner, and as it most likely will Waxman and Clay as well, is that when the Census Bureau Director Louis Kincannon testified to Congress on the 11th of December 2007, he never mentioned any meeting with MITRE, nor did he mention MITRE's concerns, which he supposedly heard on the 29th of November. According to the Davis and Turner letter, "Instead, Mr. Kincannon, in his written statement, offered a brief picture of the FDCA program, acknowledged some 'challenges,' but in general gave the impression that nothing was wrong."

You know, it always brings a hearty chuckle when I read or hear that a high risk IT project situation is euphemistically called a "challenge." You just know that when someone uses that word, they are scared-xxxxless.

Anyway, Davis and Turner want the hearing to clear things up, specifically asking for answers to four simple questions:

1. What is the true status of the FDCA program?

2. Given that the Information Policy, Census and the National Archives Subcommittee held a hearing on FDCA a full 12 days after Senior Bureau staff were reportedly briefed on MITRE's conclusions, why did the Bureau not divulge the FDCA information at that hearing?

3. Did the Bureau intentionally withhold information about MITRE's concerns?

4. Are there other technology programs, such as the Decennial Response Integration System (DRIS), about which the Bureau has received troubling reports?

It should be very simple for the Census Bureau to answer these four little questions and for Congress to get a good feel for the true state of the project: all it has to do is just count the number of times the word "challenge" is used in the testimony.

January 9, 2008

Irrelevant But Amusing Anyway

This story has nothing to do with information technology, but I can't resist anyway. The New York Times published a story today about 2 men who tried to cash a $355 Social Security check that wasn't theirs at a local check cashing company. It belonged, instead, to a friend/roommate of theirs who had died from natural causes the day before, and was "waiting" outside, Bernie-style.

As the Times writes, "Two men were arrested on Tuesday after pushing a corpse, seated in an office chair, along the sidewalk to a check-cashing store to cash the dead man’s Social Security check."

The story goes on to say that the 2 men placed their roommate's body in the chair and wheeled it around the corner, then parked the chair with the corpse in front of a Pay-O-Matic at 763 Ninth Avenue, a check-cashing business that the deceased had used.

The two went inside to present the check, but a clerk said the dead man would have to cash it himself, and asked where he was.

"He is outside," one of the men said, indicating the body in the chair. The two began to wheel the chair into the Pay-O-Matic (how he was going to sign the check is a bit of a mystery), but by then, a small crowd and a police detective had noticed the dead body in the chair.

The best-laid plans of mice and men often go awry.

UK Cancels $750 million Prison IT Project

prison.gif
In an unsurprising move, the UK government has put the kibosh on its Custody-National Offender Management Service Information System (C-NOMIS) that was intended to keep close track of the 330,000 prisoners and those serving their probation. The cost of the development, originally estimated at ₤240 million has jumped to an estimated ₤950 million, and sparked a government review of the project in August.

About ₤155 million has been spent so far, and cancellation the program is likely to cost ₤50 million in cancellation fees. The government said that it will try to salvage part of the system to track prisoners in prisons.

On the C-NOMIS website, it says, "The new Offender Management Model introduces the four Cs:

* continuity
* consistency
* commitment
* consolidation."

I guess a fifth C for cock-up, needs to be added to the list, which is better I guess than the other C, conspiracy. Its too bad too, since as the website says, "The concept of end-to-end offender management ensures that offenders are offered the best possible opportunity to change their offending behaviour." I guess it is going to take awhile before that opportunity comes again.

January 10, 2008

Holes in Illinois Automatic Highway Toll System

Illinois-toll.gif
This week the Chicago Daily Herald ran a three-part series called "Toll Gridlock" that reported on the problems with the Illinois State Toll Highway Authority's collection system. The series found that the toll authority is often "sending violation notices to the wrong addresses, leaving some drivers to miss out on chances to pay up before fines skyrocket or their driver's licenses are suspended."

In one case, a driver didn't know her I-PASS (electronic toll payment transponder) ran out of money and accumulated $179.50 in owed tolls. She was sent a letter saying she now owes $4,619, and had better pay up in two weeks, or owe the tollway $15,739 and eventually lose her driver's license.

The series also notes that "tollway officials say their license plate image readers have trouble discerning differences among the myriad of plate varieties, affecting about 25 percent of all plates on the road. This may result in fines being leveled against law-abiding motorists."

In addition, the series states that the toll authority doesn't know how many are cheating or how many motorists are being fined unfairly.

Finally, the series notes that the toll authority's management thinks all of the problems are minor, and that the way fines are assessed is "fair."

Glad I don't drive in Illinois.

Geeks.com Geeked

ComputerWeekly reported today that the Geeks.com, the computer equipment seller, has been hacked, with customer credit card information, phone numbers and e-mail addresses stolen. The site shows a banner from McAfee's ScanAlert service showing that it is "hacker-safe."

Of course, the very small print

small-print.gif

says, "This information is intended as a relative indication of the security efforts of this website and its operators. While this, or any other, vulnerability testing cannot and does not guarantee security, it does show that this site meets meets all payment card industry guidelines for remote web server vulnerability testing to help protect your personal information from hackers: HACKER SAFE does not mean hacker proof. ..."

I guess that is a reminder to us all.

January 11, 2008

REAL ID: Only to Cost States $3.5 Billion

DHS.gif
The Department of Homeland Security (DHS) announced new rules for REAL ID implementation today. As originally proposed the REAL ID Act of 2005 would have forced states to start issuing tamper-proof driver licenses and identify cards by May. The new regulations issued today by DHS says that the first deadline for compliance with REAL ID is Dec. 31, 2009.

However, DHS will allow states to request a waiver that would give them until May 11, 2011 at which point they would have to start issuing new driver licenses by December 1st, 2014 to everyone born after December 1,1964 (i.e., those under fifty years old). Then by December 1st, 2017 everyone born on or before December 1,1964 would have to have new licenses.

According to the new rules, "Effective May 11, 2008, Federal agencies cannot accept drivers' licenses or identification cards for official purposes, as defined herein, from States that have not been determined by DHS to be in compliance with the REAL ID Act unless a State has requested and obtained an extension of the compliance date from DHS."

Michael Chertoff, DHS Secretary said today that, "We have worked very closely with the states in terms of developing a plan that I think will be quite inexpensive, reasonable to implement and produce the results." DHS estimates that the revised rules will cost the states only $3.9 billion in implementation costs instead of the originally projected $14.6 billion. Supposedly, the new REAL ID requirements will add only $8 per new drivers license - a number I have little confidence in.

Of course, Congress has allocated about $81 million in dedicated funds so far to support implementation, and DHS promises another $280 million in general DHS funding that states can use if they so desire.

I guess a $3.5 billion unfunded mandate is better than a $14.2 billion unfunded mandate.

January 12, 2008

LAUSD Payroll System "Fixed"

Bus-1.gif
The LA Daily News reported yesterday that the Los Angles Unified School District (LAUSD) is claiming that the payroll crisis is now nearing an end. According to the paper, "Errors due to defects in the system were below 1 percent based on Thursday's payroll numbers, meaning 99.2 percent of the district's employees were paid accurately."

Dave Holmquist, LAUSD's interim chief operating officer said, "We're under 1 percent ... which was one of the goals we had ... and we're hoping to improve upon this. The goal was three consecutive, improving, reliable payrolls, and we believe we've reached a place of stability in our payroll.

"We're nearing an end to our crisis."

Not everyone is happy, however, as the LA Daily Breeze reported last week that, "Two Los Angeles Unified high school employees filed a lawsuit seeking class-action status Tuesday, claiming they have been underpaid, a violation of labor laws, as a result of the district's troubled payroll system."

The two teachers claim in their lawsuit that they have been "repeatedly underpaid, meaning they have been earning less than minimum wage." Their lawsuit seeks unspecified damages, including unpaid wages and general and punitive damages.

Finally, the LAUSD is trying to convince its payroll contractor to make good for the tens of millions in cost overruns, instead of having to resort to a lawsuit. It better hope that the contractor comes through soon because the LAUSD is facing at least a $40 million cut between now and July - and then a hefty $500 million cut next year - in state funding due to the severe budget crunch California is facing.

DIA - The Final End of the Automated Baggage System

DIA.gif
The Rocky Mountain News had an article today on Denver International Airport's (DIA) final agreement with two companies to demolish and cart away the remnants of its infamous automated baggage system. The cost for its removal is not known exactly, but it will be in the millions.

A fitting comment on was made by Denver City Council President Michael Hancock, "This thing never dies."

An interesting point in the story is that there may be a final, accurate accounting of the costs for the automated baggage system, something that has never been fully determined.

The article also said that a new baggage system will be in place soon.

DHS REAL ID - Real Fight Begins May 2008

DHS.gif
The new REAL ID requirements have not molified those opposed to a de facto national identity card. To them, the new requirements are effectively the same as the old ones in this regard.

In addition, the new rules seem to have set up a mano a mano situation between the 17 states (Arkansas, Colorado, Georgia, Hawaii, Idaho, Illinois, Maine, Missouri, Montana, Nebraska, Nevada, New Hampshire, North Dakota, Oklahoma, South Carolina, Tennessee and Washington) that have rejected or oppose complying with REAL ID and the Department of Homeland Security (DHS) for May.

If these states don't ask for a waiver by May, then residents of that state will have to use a passport or certain types of federal border-crossing cards if they want to avoid secondary screening by the Transportation Security Administration (TSA). DHS is obviously counting on passenger outcry to force the states to change their minds. It may be wise to avoid airports in those states in May - think of the mess at Atlanta Hartsfield-Jackson, Chicago O'Hare, and DIA.

In addition, I am still not convinced that the cost of REAL ID will be reduced because those over fifty won't be required to get a new drivers license until 2017 as DHS claims. I mean, a state that implements REAL ID isn't going to run the equivalent of two computer systems - one for those under 50 and one for those over fifty.

For instance, I'm over fifty, and I expect when my license expires next, my state will likely have implemented REAL ID. I cannot fathom why my state won't ask me to come in and get a new REAL ID license if for no other reason that it will have been over a decade since it last updated my photo and checked my vision.

The longer time now given to get a REAL ID will help, obviously, to avoid a surge of applications, but going getting a new license will be a painful, long process for those over or under 50, as even DHS admits.

And I still think the cost savings claimed will turn out to be a mirage.

January 14, 2008

Run! The Dam is Breaking! Oh, Never mind

The National Weather Service issued a flash flood warning last Tuesday morning stating that failure of the Norway Dam on the Tippecanoe River north of Monticello, Indiana "is becoming more likely."

But it really wasn't.

According to the Louisville Courier-Journal, "Michael Lewis, the warning coordination meteorologist with the National Weather Service's Northern Indiana bureau, said the erroneous warning was based on bad information that may have been entered into the agency's computer system up to two decades ago."

Two hours after it issued the alarm, the Weather Service rescinded it.

Lewis went on to say, "The problem is being fixed and the office's entire warning system will be reviewed."

January 15, 2008

Legacy Computer Models & Decision Making

Wood-Model.gif My post from yesterday about the false warning to prepare to evacuate based on an outdated computer model/data that a dam was in danger of bursting made me wonder about the flip side: how many outdated computer models are being used to make decisions that are too optimistic?

Recently, there was a story in the Washington Post on an Environmental Protection Agency (EPA) review that "found that a computer model of the Chesapeake, used by the EPA's Chesapeake Bay Program to gauge improvements in the estuary's health, tended to inflate the impact of some cleanup measures."

"Tom Simpson, a University of Maryland professor who led the review requested by the bay program, said there was no evidence that the EPA had been purposefully deceitful."

The Post story goes on:

"Simpson and other researchers were asked by the bay program to review some of the calculations plugged into its computer model. These equations described the impact of certain save-the-bay tactics: plant X amount of cover crops to hold fertilizer on farm fields, thus achieving a decline of Y in fertilizer-polluted runoff."

"But Simpson said his review found that many of the equations were based on small-scale experiments that might not predict what would happen on a large farm. Others were based on the educated guesses of experts."

"Fifteen assumptions were found to be accurate, and three were found to underestimate the benefit to the bay, according to the bay program"

This story reinforces the notions that essentially, all models are wrong, but some are useful, as George Box said (there is a nice paper here by John Sterman, Director, System Dynamics Group at MIT on the usefulness of models that is worth reading). However, I think we may need to update Box's saying to something along the lines of:

All models are wrong but old models that aren't reviewed are more wrong; make decision based on them at your own risk.