• On GameSpot: Wii Fit tells 10-year-old she's fat
August 27, 2008 5:55 PM PDT

After flight delays, FAA may add backup system

Posted by Stefanie Olsen
  • Print

In the wake of a computer failure that caused delays for 646 U.S. flights on Tuesday, the Federal Aviation Administration plans to upgrade its decades-old technology for flight-plan processing and potentially add a third backup system.

The FAA's central system in Atlanta for handling all U.S. flight plans went down Tuesday because of a corrupted file, according to a FAA representative. Then, when the administration's secondary and lone backup system in Salt Lake City got bogged down with repeated re-entries of flight plans from the airlines, the malfunction caused major delays at airports in Atlanta, Boston, Washington, D.C., and Chicago.

By Tuesday evening, the flight delays were under control, but the computer system in Atlanta wasn't functional until about 1:30 a.m. EST on Wednesday, officials said.

Now, FAA representatives said that by September, it plans to add more computer memory to its data communications network known as National Data Interchange Network (NADIN). And by early next year, the FAA plans to completely upgrade the decades-old data communication network with new hardware and software.

"The big difference is that (the new system) has a lot more memory, so what happened yesterday could never happen again," said FAA spokeswoman Diane Spitaliere.

Laura Brown, another FAA spokeswoman, also said that the administration is considering adding a third backup system for NADIN at its New Jersey research and training facility.

At the heart of the problem--and a potential source of other trouble--was old technology at the FAA, officials said. One representative referred to NADIN as "'70s-era technology" that was bought from a third party and then maintained in-house at the FAA. (Another representative said the data communications network was put into operation about 15 years ago.) Despite its age, the failure--sourced to an IPX9000 packet switch--was caused by a human error that "resulted in the wrong configuration data being loaded onto the switch," officials said.

The cause for the flight delays came from the backup system in Salt Lake City that should have been able to take over operations for the Atlanta system smoothly. But it slowed to a crawl under the weight of airlines re-entering their flight plans repeatedly once they got an error message from the first system. The FAA had to call the airlines to tell them to stop hitting re-enter; the system didn't have enough memory.

The FAA is also upgrading host computers in 20 "En Route Centers" throughout the country. Those centers process flight plans sent from NADIN for planes that are flying in high-altitude air space over their area.

"Those computers were very old and running a language that only a few people left alive knew about," Brown said.

Recent posts from Business Tech
IPOs a thing of the past?
IBM to start-up: Industry vet responds to recession
Intel, Hitachi to develop solid-state drives
Report: AMD Phenom II chips echo Intel's i7
Markets tumble on recession news
Add a Comment (Log in or register) 12 comments
by Commander_Spock August 27, 2008 6:24 PM PDT
The way to go! Just as was expressed in my yesterday's post... Re: "Laura Brown, another FAA spokeswoman, also said that the administration is considering adding a third backup system for NADIN at its New Jersey research and training facility..." How about "unlimited redundancy"? Including "Space-Based"! Live Long And Prosper. ;-)
Reply to this comment
by n3td3v August 27, 2008 6:26 PM PDT
They only have one back up system for the entire U.S air space?
Reply to this comment
by FlyHeading August 27, 2008 6:56 PM PDT
They don't have a third backup for NADIN yet they have plenty of spokesmodels and they both make over $150,000.00 a year. More waste at the FAA paid for by the aviation industry and the tax payers.

http://php.app.com/fed_employees/search.php
Reply to this comment
by briancgraham August 27, 2008 11:31 PM PDT
hmmm... there's more to this story than we're being told...

check out this article from March 2005...
http://www.networkworld.com/newsletters/servers/2005/0321server1.html

....
"For Andy Isaksen, computer scientist for the Federal Aviation Administration in Atlanta, having servers that are up 24/7 is imperative."

"Isaksen uses two Philips DS714 mainframe computers as his message-switching network. The DS714 was originally manufactured in 1968 and upgraded with new processors in 1981. Since then, they have been getting increasingly harder to maintain, support and write code for. "

...
"Isaksen will install two Stratus FTservers in his production network - one in Atlanta and the other in Salt Lake City. Two other identical servers are part of a test bed for running applications Isaksen and his staff write."

...

?The two centers run in a load-shared mode and at any instance can take over for each other," says Isaksen. "If one server dies, the other one almost instantly takes over so there is no loss of service to the aviation community."

...
"Isaksen will be deploying the Stratus servers to the field in the middle of next month but doesn?t expect they will go into production until the first of next year. "



So did they push off the upgrade for over 2years and still on the were they still on the Philips machines?? Or, did the Stratus servers fail??

I can understand human error.... to a point. I hope it's a reminder to all of us to be realistic about our plans for the redundancy of our critical services.
Reply to this comment
by faahope August 28, 2008 4:09 AM PDT
The FAA comments and ideas are old too.

Talk about having a backup system. According to 1900.47B directive the FAA has contingency plans for the entire U.S. But they have never trained any one as it states in paragraph 15 of 1900.47B. I dare anyone to visit an ARTCC high altitude center, and at random walk up to a controller and ask them, If the center that boarders your airspace goes down or what the FAA calls ATC Zero what section of their air space do you assume?

All they know is that there is a contingency plan somewhere and they also know that there is an agreement with the next center about airspace. But they have no idea what it is because it has written on it For Official Use Only.

Maybe if the controllers had some training, when a center like Memphis goes to ATC Zero as it did again on Aug. 9th , with 5 centers boarding its airspace, just maybe they could have assumed Memphis airspace.

If went to a public meeting, at this meeting I asked the question, If Miami Center goes ATC Zero who will assume their air space? Rick Ducharme Deputy Vice President of Terminal Services said Jacksonville. But as I asked the next question Has any ARTCC center ever assumed another centers airspace? RICK DUCHARME said NO but yet this is their contingency plan.

Sure make you wonder how safe we are.
Reply to this comment
by Bill_I August 28, 2008 8:05 AM PDT
I agree with Brian, using computers old enough to vote is INSANE. Its time to fire the drones at FAA and get some outfit like EDS to replace the swap-meet junk with something that works. Maybe call Verizon, their phones stay on 24 / 7. This all reminds me of the movie ' WarGames'.
Reply to this comment
by lyntone August 28, 2008 10:00 AM PDT
The Republicans want to privatize all of the government agencies, that is why the undermine every agency with inept people to run them.
Please understand this, just look at FEMA.
these agencies are all protecting the busineses that they are to control.
Reply to this comment
by Get_Bent August 28, 2008 11:38 AM PDT
Here's a novel idea: Instead of having a primary and a backup system, how about using a distributed system with at least two network connections per node? If one node went down, the other nodes could pick up the slack. They might run a little slower due to the extra workload, but at least the whole system wouldn't come to a screeching halt. And make sure that each node's network connections are on separate LAN equipment and WAN links (with different physical carriers) for better redundancy.
Reply to this comment
by jwmclean August 28, 2008 12:07 PM PDT
The recent spate of computer malfunctions at the FAA which disrupted travel across the U.S. and caused significant delays for thousands of travelers, graphically demonstrates the decrepit state of our obsolete air traffic control system. Congress must act now to begin funding modernization before crucially needed funding expires on September 30th.

Most of the hard work is already done. The House of Representatives and the Senate Finance and Commerce Committees have passed legislation on the funding portion of FAA reauthorization that would not only provide crucial funding to modernize our air traffic control system, but create jobs and generate real economic growth for our communities by funding many long-term airport projects.

If this bill is not passed this year, we will have to start all over again with a new Congress and a new Administration. The aviation industry simply cannot afford to wait that long. We call on you and your fellow Senators to finally pass the pending FAA re-authorization.
Reply to this comment
by JAuskelis August 28, 2008 2:58 PM PDT
Hey Stephanie - what is a "IPX9000 packet switch?" A google search only returns an echo of your news article.
Reply to this comment
by GlennAllen September 9, 2008 8:11 AM PDT
Anyone who calls the Philips DS714 a "mainframe" doesn't know what a mainframe is.
Reply to this comment
by PepperJKat October 1, 2008 6:59 AM PDT
Of course, DS 714's aren't real mainframes, however, the manufacture (Philips in Holland) at their inception called them "mainframes." Probably because they were 32-bit computers and Philips wanted to distinguish the DS 714 from mini-computers at the time.
advertisement

In the news now

A tech veteran responds to the recession

LogLogic's Patricia Sueltz heard a clear message about the economy from investors, but she already knows a thing or two about navigating through tough times.


Obama's AG pick on privacy

Eric Holder has criticized the warrantless wiretapping program, but his views on other online policies may not be that far from those of the Bush administration.


Resource center from CNET News sponsors
Business. Ready.
Sony VAIO® Professional PCs.

Click Here!
A new grade in mobility demands a new kind of notebook. And Sony delivers.Tough, portable and featuring up to 7.5 hours of battery life! VAIO® Professional notebooks are built for business. Learn more.

Click Here!
Built tough for business.

Learn more about the rigorous quality testing Sony puts its notebooks through.

Protect your investment.

Find out why VAIO® tech support recently won a Laptop Editors' Choice Award, July 2008.

Long battery life.

Up to 7.5 hours of battery life! See how VAIO® PCs will keep you productive longer when on the road.

Travel light

Check out our ultraportable line-up, starting at 2.87 lbs.

PCs for every need.

Find out which VAIO® notebook is right for you.

About Business Tech

Your destination for the latest news on enterprise-level information technology, from chip research and server design to software issues including programming, open source and patents.

Add this feed to your online news reader

Business Tech topics

advertisement
advertisement

Inside CNET News

Scroll Left Scroll Right