Burning Thoughts

Blake Abbott was my temporary manager back in 2005, when MegaTel went bonkers. I flew down to Tampa to learn how Blake’s team created interfaces from old custom-built systems to the incoming SAP enterprise system. The nuts and bolts of the job were unimportant. They’re background against which operated the maw of a large corporation.

Blake wore wire-rimmed glasses. He was somewhere in his 40s with a shy handshake. A medium-sized guy, fair-haired with a liking for starched shirts and well-pressed khakis. In the first meeting he introduced me to the group, except Javier who was late. Javier was the team’s tech expert. Blake wouldn’t go into to tech stuff until Javier arrived.

Blake talked of his children, his wife, and his church. My teammates nodded. I had the feeling they’d heard all this before. My attention wandered to the SAP system diagram on the whiteboard behind his head.

Before I finished examining it, an energetic guy, with dark wavy black hair, early 30s, shirt not completely buttoned up, rushed in, saying he fixed a compile problem at 1 AM.

Breathlessly Javier gave a quick overview of the Java Beans approach and the BEA tool set we were using. It was chaotic overview, not arranged in logical order, but clearly right because his tests were successful. Blake interjected a thought now and then, mainly to explain the business role of the extracts and who were the important players in the other organizations for the interfaces.

I stayed in Tampa a week, learned a good bit, and was tasked with developing a number of online interfaces.

Status Calls

When I returned to my home office in Silver Spring, the director that Blake reported to, was given an office around the corner from my cube. Sahen was okay. He was Indian. Once the Southeast Asian CIO was put in charge, you had to be Indian to move up. Maybe he would have been a good guy if he was calling the shots, but he had a directive from the new CIO–get SAP up and running in 6 months.

If you’re not in IT, that might sound fine, but there were hundreds of choices, actions, and programs, each affecting the others, all affecting the entire MegaTel supply chain, across many different teams in many departments. In toto, tens of thousands of employees and billions of dollars of inventory.

Sahen instituted twice daily conference calls, one at 7 AM, the other at 6 PM. Six days a week. Everyone had to work 12 hours a day. There must have been 50 or 60 people on that call. The agenda was never written down, nor were meeting notes kept. It was Sahen asking each manager (5 in all) to give a status of their team’s work, any insuperable problems that he could help them with, and what they would accomplish by the next call. Most of the programmers never spoke, except to confirm their name at the start of the call. What a waste of our time!

I quickly saw an important flaw in the plan. Sahen demanded that work start on the interfaces immediately, before the values and relationships of key fields in the new system were decided by the subject matter experts, the SMEs. The clients, as we in IT referred to the supply chain users, weren’t really sold on SAP. Why were going to it then? The new CIO who’d been wowed by SAP’s schmoozing him that led to the purchase. The CIO ordered Sahen to get the system running so he could demonstrate to the users how great it was. And, as shit flowed downhill, Sahen, then Blake and then me, had to get interfaces ready that would give meaningful insight into the flow of dollars and material through the MegaTel supply chain.

After two weeks of having to answer “no interfaces complete” on Sahen’s conference calls, Blake started another conference call before Sahen’s 6 PM call, so he’d know what to report. During his team call, Blake in his Sunday school voice, said try your best to make the interfaces work, but don’t kill yourself doing it. There are more important things in the world. Yet, despite that caveat, everyone was still forced to work 6 days a week, 12 hours a day, plus callouts that came your way. Oh, MegaTel IT programmers are all management–no pay for overtime.

Happy Path

Blake developed a new status–‘The Happy Path’. If your program worked for good data, it was assigned to the Happy Path. It wasn’t necessarily robust. Bad data could make it fail. If all the other systems worked properly, it would work. Blake reported ‘Happy Path’ as ready for system test.

Over the next two months, progress was made. During conference calls, we programmers started Instant Messaging each other, making fun of the maniacal decisions that our betters were making. Comments that we weren’t allowed to make aloud. Through it all, Blake’s low-key voice reported the percentage of interfaces completed, using his Happy Path metric. Blake’s voice softened when he reported to Sahen of communication links for interfaces that were balky. Sahen huffed and puffed, saying, although it was a great difficulty, he’d get those other directors to support his team properly.

A week later Blake told us that we could skip the 6 PM calls. He’d handle the evening status updates. We were to send him status reports everyday by 5 PM. I learned that the Texas branch of his team regularly was late with them. Blake didn’t berate them, as I would have. His voice just got softer as he reminded them he needed them by 5 p.m. East Coast time.

End-to-End Test

After four months, we had our first big end-to-end test. Run the feeder system, generate the interface data, insert that data into SAP, send output to the downstream systems. It was a huge muckup! The ordering systems data either made the delivery systems throw up or the data was accepted but had different key structures than expected. Probably no one was surprised, except Sahen and the new CIO, who asked, “What went wrong?” During that conference call, everyone was forced to listen to an hour harangue.

It was a delight. Not! It was like your father telling you as a little kid, don’t chase the ball into the street–cars will hurt you. Yet your father put you in the front yard to play so he could watch football.

I didn’t have the nerve to say anything, even though forcing tech people to code interface before user experts defined the linkages was crazy. I pinged Blake that message. He replied. “True. Do I want to be fired? Or do you want to be fired?”

When no one offering an explanation of the system test fiasco, the CIO tried again. “Why did the inbound interfaces not use the same values as the outbound interfaces?” I bet the CIO ran his finger down a list of names, “Sahen?”

Sahen hemmed and hawed, “Umm … ahhh. We had very short time to work on so many interfaces. That limited the communication between the various teams that did interfaces. Blake Abbott was in charge of interfaces inbound. Blake, can you explain further?”

Blake’s voice was softer than ever. However, with no one else was breathing, his words were clear. “I concur with Sahen.” He paused. I imagined him in his office in Tampa looking at the picture of his family, struggling to get out of this trap. Finally, he said, “I’d only add, isn’t the purpose of the end-to-end test to discover errors like this? It did. It worked.”

The best possible answer, Blake!

There was a pause while the CIO digested Blake’s verbal jujitsu. “Hmm. I see. So the test was successful.” He paused again, then said firmly. “Well, now that you have a list of the errors, fix them. Let’s have another system test next Friday.”

That led to dead silence.

These errors were not simply put a “2” in the field instead of a “1”. The supply systems had specific values in keys, used to match, sort, summarize and report. The SME users developed these codes over years. The provisioning systems had different values for tracking inventory locations. The end customer ordering systems had a third set of categories. They might all be called location, but what was the mapping between them? Some values were undefined in supply, had massive detail in provisioning, and were tags in customer ordering. Which was to be used? Such questions required intensive discussion with internal MegaTel SMEs–which we were forbidden to have. The CIO sold the system to the users as solving their problems. It would decrease their work load, not add to it.

“Well, good then,” the CIO summed up, “since there are no objections, let’s rap up this call. I’ll expect a better review after the next end-to-end test. Thank you all for your efforts.”

We were shell-shocked after that meeting. I walked around my work building a couple of times, just to clear the cobwebs of CIO crap off my real thoughts. I knew we could make the proper changes when the requirements were settled, though it might take more than a week. It’s not like each interface was a single program. They were sequences of modules which shared data with modules written by members of many teams.

Too Much

Monday morning, I arrived at work early. I’d already missed a call from Javier. Strange, he was not a morning person.

I returned the call right away. “What’s up?”

It still takes my breath away. “Blake died. Yesterday. After the Sunday call. Heart attack.”

I know I must have said something to Javier. Although what I don’t know.

I walked over to Sahen’s cube. He already knew. He was gracious in his kind remarks about Blake.

But my mind screamed, You forced him to another path. His family has lost its Happy Path!

---

Image: Landscape with the Fall of Icarus. By Pieter Brueghel the Elder – GalleriX, Public Domain, https://commons.wikimedia.org/w/index.php?curid=5279287