Offshoring

Wikipedia describes offshoring as "the relocation of a business process from one country to another." There once was a time in Silicon Valley that offshoring was a dirty word. I was here (in the Valley) at the time and I remember it well. Let me take you back a few years in time and describe what I've seen of offshoring over the years.

I think there is still a general fear and bitterness associated with offshoring in the United States. It was much the same when I first heard about it in the Valley around 2002. It had all started several years before, but it was in the midst of the Valley's worst recession that it started to take full effect. The big companies of the Valley were under tremendous pressure to cut costs. They had offshored things like call centers, and so it only made sense to move up the value chain. There was plenty of programming talent to be found in India and China. Soon it wasn't just big companies, but even startups. It wasn't uncommon to hear about a startup where the founder was non-technical and shipped the programming tasks to a firm in

For me personally, I heard a lot about this kind of offshoring, but it did not affect me until 2004. That was when I was working for Vitria Technology. Vitria had been a shining star in the dot com era, and made a lot of money selling the enterprise integration software known as BusinessWare. However by 2004, Vitria's glory days were long past. They had been searching for their second hit for years, but were still being buoyed by recurring revenue from BusinessWare. I joined to work on one of their new ideas, what was eventually known as Resolution Accelerator or RA for short. I worked on a small team of engineers at Vitria's office in Sunnyvale. We developed RA, but our QA was in India. Vitria had moved its QA functions to India the year before we built RA.

RA turned out to be a success. We had lots of paying customs in healthcare and telecom. RA was built on top of BusinessWare, as that was part of our strategy. However a lot of the technical leadership behind BusinessWare felt that the RA team had built a lot of reusable features that belonged in BusinessWare. So even as we were selling RA to happy customers, we started a new project to migrate many of RA's key components to BusinessWare and then rebuild RA on top of this new version of BusinessWare as it was being developed at the same time. Great idea, right?

I was working with the BusinessWare team who were essentially re-inventing my wheels but within the mammoth piece of software that was BusinessWare. However this team turned out to be a lot different than the team that I worked on that built RA. There were no developers. There were multiple architects who were assigned to BusinessWare, as it was the key piece of software in the company. Then there was a tech lead from the BusinessWare team. However this tech lead did not code. Instead he wrote specs that were then implemented by a development team in China and then tested by our QA team in India. This was the model they had adopted fro BusinessWare for a couple of years already. I was essentially a consultant to the tech lead and was not supposed to interact with the developers in China -- even though they were largely writing code that was functionally equivalent to code I had already written. Meanwhile I was the tech lead of RA and we were now supposed to "graduate" to the same development process as BusinessWare. So now I had a development team in China who I was supposed to direct. My job had "evolved" into writing a spec for developers in China and coordinating with a QA team in India. Sunday - Thursday you could find me on the phone from 9 - 11 PM every night. Good times.

Meanwhile all of the developers on the RA team took off. One went to Google and worked on GMail. One went to Symantec. Finally I could take no more and split for a startup… That startup burned through cash like crazy, and went out of business a year later. All I've got is this patent to show for it, along with the valuable experience of writing my own programming language. In 2007 I went to work for eBay and ran into offshoring at a much larger scale.

Development organizations typically consisted of engineers in San Jose plus teams from our Shanghai office. Plus they were heavily augmented with contractors from two firms based in India. The contractors would have a tech lead in San Jose who interacted with a tech lead from eBay. In addition there would be a development manager who "owned" the application and who was usually the manager of eBay's tech lead on the project. The tech lead's job was similar to what it had been at Vitria. He was in charge of a spec and coordinated with the contractors plus engineers in San Jose and Shanghai. As you might guess, most of the actual coding was being done either in Shanghai or by contractors in India. The tech lead usually didn't interact with contractors in India though, instead they worked with the contractors tech lead/liaison.  Finally in addition to the development manager for the application, there might also be an architect -- if the application was important enough. The tech lead worked with the architect on the design that would become the spec that would be sent off to India and China. The tech lead would also interact with a data architect to design database schemas and operations architect to design the physical deployment and requirements of the application. The point is that the tech lead had almost no chance of actually coding, and this was just accepted and understood.

Just before I left eBay, things started to change a bit. In particular eBay brought in some new blood from Microsoft of all places, who took over the search organization. This was the organization that trail blazed offshoring at eBay. It had been a search VP who had "opened" our Shanghai office, and most of the engineers and QA there were part of the search organization. New management decided that this was not working and sought to move all search engineering to San Jose. I'm not sure how the Indian contractors would play in this new vision, but it should sounded like they would be squeezed out too. The Shanghai engineers were unilaterally "reorged" to another organization in the company (an organization that had already been gutted and whose VP was being pushed out, but that's another long story.)

Ok, so what's the point of all of this? I'm not sure to be honest. From my perspective, I was very dissatisfied with how we used offshoring at Vitria. When I joined eBay we were using it in a similar way. If anything, it seemed the eBay way was even more inefficient. There was an unwritten rule at eBay that if a San Jose tech lead estimated that a technical task would take N days, then you would need to set aside 2*N days if the development was being done in China and 2.5*N days if the development was being done by contractors in India. This might seem harsh or much worse, but certainly a big part of it was the operational overhead with offshoring. Further this system was judged a failure by the largest organization at eBay.

At the same time it would be foolish to say that offshoring has been a failure. There is a lot of awesome software development being done via offshoring. I'm not so sure about the practice of "design it in America, develop it offshore" when it comes to software. At the very least I have not seen this work well in person. Then again perhaps the problem is simply one of tech leads who don't code and that just happened to coincide with my offshoring experiences.Whatever the case, one thing is for certain. There was palpable fear about offshoring a decade ago, and it turned out to be a false fear.