Thursday, May 29, 2008

Industry vs. Research

As a graduate student looking for jobs, a common question I heard was "Industry or research?" Industry jobs include developers, technical managers, and even applied researchers to a large degree. Fundamental research jobs are professors and research scientists at industrial labs, such as Microsoft Research and Yahoo! Research. The definitions are somewhat fuzzy (applied research is industry? industrial labs are research?), but a generally distinguishing characteristic is whether publishing papers is a primary aspect of the job.

It was this characteristic that made me realize the fundamental difference between industry and research. It's one I wish I had known when I started graduate school. In short,

Industry is primarily about selling products, while research is primarily about selling stories

This is why publishing papers is telling: papers are a medium for selling stories. Of course, a good story helps sell a product, and a working product helps sell a story. So there is definite overlap. However, it's telling how well the pros and cons of industry and research derive from this basic difference.

To illustrate, consider some classic pros and cons. In industry, since you sell products, your work has direct impact on people that use the product. Since people will typically pay for this impact, the product itself is the source of funding. And if your product is sufficiently impactful, it is the source of a lot of funding, and you get rich. However, this means it's critical to quickly and consistently create marketable products. The result is a dampening effect on the problems targeted by industry: they are dictated by the market, and typically have shorter-term visions with fewer (or at least more calculated) risks.

In contrast, research has significantly more freedom in the problems it tackles. They are often longer-term, riskier visions. Research can do this because it only has to sell stories describing core ideas, not fully working products. Thus, it can focus on interesting technical problems. However, "selling" a story does not usually mean for money, but rather convincing people that it describes a good idea (e.g., getting a paper accepted to a conference). Since neither the story nor the idea generates money directly, researchers must seek out external funding such as grants, or, in industrial labs, income from products (which, to be fair, often contain the final fruits of research).

Given such pros and cons, the distinction of product vs. story seems obvious in hindsight. However, what made me first realize it was a more subtle situation. My advisor asked me to devise a data model for the system we're building. I came back with two options: a very common model, and a novel model that was simpler and more expressive. I favored the novel model, but my advisor said we should use the common one. His reason was that the data model was not our primary contribution, and papers with too many innovations can confuse readers. And he was right. Even though the novel model would make for a better system, the common model makes for a better story — and I'm currently in the business of selling stories. At some later date, after we sell our current story, we may sell another story that focuses on a new data model.

To conclude, I want to say that this isn't meant to promote either industry or research. In my particular case, I've found that I lean more towards selling products than stories. However, I've spoken with both developers and researchers, and both agree with the product vs. story differentiation, and each prefers their side. Of course, I'd love to hear from anyone else on the topic. I just think that understanding this difference is vital to making an informed decision about graduate school, and life afterwards.

Saturday, May 24, 2008

Job Decision

It's finally official. After nearly two months of applications, interviews, travel, negotiation, introspection, and extremely hard thinking, I've made a job decision. Come January next year, I'll start as a Program Manager in Microsoft's SQL Server team.

This was a very difficult decision, as I had to choose between five compelling offers. In the end, there were two primary considerations: location, and how I want to contribute to my field.

My offers spanned two locations: Microsoft in Seattle, and the others in Silicon Valley. I characterized my options as better quality of life in Seattle vs. proximity to networking and friends in Silicon Valley. Seattle's quality of life is better due to lower cost of living, much cheaper housing (I can actually afford a nice house my first year), and significantly nearer mountains. It also feels more laid-back. On the other hand, Silicon Valley hosts constant interaction between innumerable tech companies, providing excellent networking opportunities and mobility. Also, several of my friends live there.

For me, Seattle and Silicon Valley were effectively tied. However, this was a two-person decision, so Sarah joined me in visiting both places. She met and loved my Silicon Valley friends, and received a great tour of Seattle courtesy of Microsoft. Sarah sees locations differently than I do. I pick a job, and that decides the location; Sarah picks a location, then finds a job. Location is part of how she defines where she wants her life headed. As it happens, before we were engaged, she was already looking to move to the Pacific Northwest. Thus, though she liked California, and especially my friends, Washington is closer to where she wants to be. This was one consideration.

The other strong consideration clarified after many conversations with mentors. The key question is how I want to contribute to my field. One path is as a technical luminary, with primarily technical contributions. This path includes god-like developers, researchers, and other deeply technical people. My offers at IBM, Oracle, and Yahoo! followed this path. Another path is as a technical manager, with primarily leadership and strategic contributions. This path includes general managers, CEOs, and other big-picture people. My offers at Google and Microsoft followed this path. I've spent most of my life as a deep techie. However, due to some eye-opening experiences and a lot of introspection, I've decided that, at least currently, my calling is management and leadership.

Neither of these considerations alone decided me. But due to both together, plus several others secondary, I've accepted the Microsoft offer. A couple things in particular really impressed me about the position. First, I got to meet several team members, including my future boss, and they're all amazing. Second, Microsoft is very serious about investing in people and building careers, so the opportunities for mentorship and advancement are fantastic. I'm extremely excited, and really looking forward to starting. All that's left is to finish my doctorate!

Finally, to wrap up, I want to very sincerely thank everyone who helped me throughout this process. All of my mentors for their advice; all of my friends for their time, love, connections, and support; and all of my family for putting up with weeks of waffling (individuals may fall into more than one category). I know not everyone will be happy with my decision, but I hope you will all be happy for me. Of course, feel free to send along any particularly strong variations on "You fool!". I promise no hard feelings.