This Bug's For You

BY SAM WILLIAMS

[This is a pre-edited draft of a business story that appeared under the headline "Worminators" in the New York Post, August 31, 2003.]

For laborers like Sanford Whiteman, the callouses are all on the inside.

As chief technology officer of Cypress Integrated Systems, a company that specializes in keeping other company's computer networks afloat, Whiteman knows the meaning of hard work. In a month during which much of the city's white collar workforce went on vacation, Whiteman has traversed the city, performing an endless string of emergency house calls.

"I've had at least three all-nighters in the last three weeks," he says.

Come Monday, Whiteman will tip a much deserved beer to celebrate the passing of a month that brought with it not one but two major Internet outbreaks, torrential rains, and the largest blackout in U.S. history. Like most members of his profession, however, he'll have to keep his Labor Day fesitivities in check should the pager on his belt bring news of another client emergency.

"I have never not been on call in this business," says Whiteman, tapping his pager. "I've been through them all: the early alphanumerics, BlackBerries. You name it."

The recent "Sobig.F" and "Blaster" outbreaks have drawn attention to the lingering vulnerabilities of corporate computer networks to malicious attack. Likewise, the August 15 blackout offered a sobering reminder that Murphy's Law still holds true in even most modernized systems. Still, when it comes to the people who keep those systems running, the spotlight remains deliberately dim.

Maybe that's because many companies are loathe to publicize just how much they depend on a few technically-trained workers. Those that do give credit, like New York City Health and Hospitals Corporation, tend to balance it with reminders of overall system security.

"If the email slows down, it's an inconvenience, but it doesn't affect the core business," says chief information officer Frances Pandolfi, acknowledging the recent impact of Sobig, a self-propagating computer "worm," on her company's network. Sobig reduced the responsiveness of the hospital's 20,000 PC network but never threatened private patient records, Pandolfi says.

Still, for those who spent the night of August 14 making sure all 20,000 of those PCs had backup power, only to battle Sobig a few days later, the memory is dramatic nonetheless.

"It was definitely a back-to-back slam," says Corey Cush, HHC's senior director for networking services.

Most of the drama, Cush says, comes from having to solve problems that always seem a little bit different every time. What might seem like glorified plumbing to some is really more like a high-speed chess match. You have to think well and execute on the fly.

"Sometimes you want to pull your hair out, but if you have the right mindframe, it can be fun," says Claude Vilfort, systems engineer for Miningham & Oellerich, Inc., a company that sells software and support services to Manhattan banks.

For Vilfort, both Blaster and Sobig appeared first in the form of user complaints. With Sobig, bogus emails from servers outside the company were hitting in boxes at a rate of 15 per minute. Vilfort says he put in a 12 hour day pumping out the latest version of Norton Antivirus, which had been updated to identify Sobig's encrypted source code, to every machine on the company network. Although a few users opened the attachments, infecting their machines, Vilfort says he was able to quarantine the spread.

"People ask me why I stay so calm. I say, 'This is what I get paid to do: Solve problems' "

For Cypress Integrated Systems' Whiteman, whose clients don't tend to have a wary network monitor on staff, Sobig has proven a tricky foe. One of his top clients is a company that distributes laptops to its mobile sales force. For a week, he says, stricken laptops were bombarding the network each afternoon at 5 pm when the sales people logged in from the road. It wasn't until a few days ago, that he got the last of those laptops users to install the security "patch" that blocks Sobig infection.

"What surprises me is that people have still not learned the fundamental rule of not opening attachments from people they can't identify," he says.

Fortunately, Whiteman is well-compensated for his efforts. At a time when most traditional software programming jobs are heading overseas, Whiteman remains in high demand, charging between $100 to $175 an hour depending on the job, client and location. On Thursday, he and a work colleague, Tom Grady, paid a house call on a Greenpoint warehouse owned by Manhattan Store Interiors. The company's computer-operated CNC router, a machine which can turn wood panels into custom shapes and letters, relies on software files piped in from a remote network server. Grady and Whiteman cleared the server for any signs of Sobig before moving upstairs to the company's desktop computer system.

Like Whiteman, Grady was looking forward to the three day weekend. He says his own billings for August were more than 50 percent higher than his July billings. At the same time, he remains ready for the next emergency call.

"There's only so many hours in the day," he says. "That's the problem. If we could just extend the day, we'd be doing really good."