On our one-year anniversary of our power toys release, I gave a Microsoft Engineering Excellence Talk on the lessons we learned the hard way “going open,” to motivate other teams to explore all the way they can embrace their communities, including sharing source code.  (yes, a broken leg didn’t slow me down at all.)

A Power Toy Story

Our charter is to develop after-market solutions for Visual Studio 2005 that either provide additional functionality or address adoption pain-points.  We based our selection criteria off of the top most-voted customer suggestions.  But why stop there? 

We have conversations around bug reports on Connect, how do I question on the Forums, and even have conversations around the specs we share.  But what could we learn about conversations at the source code level? 

Going open for the first time

We discovered with our 3 projects on the CodePlex Beta that releasing power toys was the easy part.  What we didn’t know was how to run a successful open source project.  For example, “how do you cost the code reviews from potential community contributions?”  and “where do community contributions come from?” 

Where we rocked

With 3 Program Managers, 2 Developers and 1 Tester (Software Design Engineer in Test), we delivered 10 power toy solutions, 2 of which were in the top 40 most-voted customer suggestions for Visual Studio 2005.  We used Scrum, allowing us to do multiple releases each month at various development phases. Additionally, we created the Pack Installer, allowing users to pick and choose which power toys they want to download.  Lastly, each team member has a blog, where we’ve shared what we’ve learned over the past year.

Personally, I’m a huge believer that you learn more from analyzing your failures, so I’ll share with you some very painful lessons that I learned the hard way, so you don’t have to.

Where I failed

The number one thing I would have done differently if I were to do it all over again is to create a single Power Toys for Visual Studio CodePlex project and fit as many power toys in there as possible.  I was too focused on building a successful open source project around the individual power toys, instead of thinking “how can I build the community around a power toy collection?” 

I focused solely on providing solutions, rather than collaborating with the community on incubation-style projects.   Success could have looked like a customer saying, “hey, that power toy was my idea!”

Lastly, I didn’t have a solid story for the Microsoft Downloads Center versus CodePlex.  In retrospect, I would have tried having the Download Center point people to the CodePlex for discoverability when searching, unless user feedback suggested having the Microsoft releases available additionally on the Download Center.

What is OSS?

At this point in the power toy story, I attended James Howison’s talk at OSCON 06. I put together a presentation to the power toy team on the lessons I learned from OSCON, and had Port 25 tape the conversation. Yes, you can actually watch a Microsoft team discuss how to do open source.

Tips going open

The decision to go open must be action-oriented.  You cannot just throw code over the wall and hope for the best.  “If you build it, they will come,”  is just a movie quote.  =)

Thus, decision is all based on motivations.  One common example is to provide additional extensibility points.  Think of reversing the traditional “platform play” (instead of the core being open and the add-ons being closed, have the core be closed and the add-ons be open) to explore all the ways you can collaborate on top of this extensibility layer.

Our power toys with the most community-code contributions were those that we partnered with community members that had a similar need.  For example, MSDN Forums moderators always want to keep track of what was going on.  So Joe spent roughly half a day on a Forums Moderator Toolkit prototype that we released on CodePlex.  And this tool had our most contributions.

Some CodePlex getting-started tips

When you go open, there are a few absolute priority-one must-haves:

• When users download your sources, they must always just build.  Always build.  Always. 
• Make sure to provide crystal clear build instructions.  You only get one chance to make a first impression.
• Always provide a getting started guide on how to use the project.  Screencasts are ideal for this.

And lastly, the project driver / project owner / benevolent dictator should feel free to reject check-ins that are not in scope or go against the coding guidelines, as explored in detail in the lessons learned from OSCON.

Happy CodePlex Community Tips

• Establish contribution guidelines early
• Once open, always open is a fundamental rule of OSS.  Avoid internal conversations. 
• On the same note, conversations must be conducted as close to the code as possible.  When we started, we tried to have a power toy discussion board on the MSDN Forums.  The result was two login systems and a hodge-podge of Q&A across various power toys.  When I got back from OSCON, I manually ported all the Q&A to their individual CodePlex discussion boards.  And this goes back to the, if we just had a single Power Toy codeplex project…
• Release early, release often is yet another rule of OSS.  Whenever we get a contribution, we create a new minor release.  And now that we release at beta, instead at 1.0, helps us hold true to this release-often rule.
• Provide ways for the community to contribute to upcoming releases by listing out the needed bug fixes or feature requests.
• Lastly, something else I learned from James Howison’s talk is that “Ship” is a very Microsoft term, which has its own connotations, implying “supported proprietary software.”  He strongly recommended that we “release” our OSS project, and not “ship” them.

Testing an OSS project

One of the most significant contributions we’ve made is sharing our best practices for software testing.  Below is a “cliff note” version:

• When we go open, we open everything, from specs, to design docs, to test plans, and test suites.  
• We use OSS tools, like Nunit and WiX, which go nicely alongside the free Visual Studio Express Versions.
• When we go open at beta, everyone on the team does a bug bash for baseline functionality. 
• We also ensure we hit 80% code coverage with our test suites. 
• We log all known bugs as work items on CodePlex when we go open at beta.

And lastly, our single tester on the team John, who writes all of our automated tests and the automation test infrastructure itself, is the mastermind behind this testing content. He blogs about Testing OSS @ MSFT, in his two articles:

• What’s different about testing open source projects?  (Part 1)
• Recommendations for corporate teams developing Community Based Open Source projects (Part 2)

In fact, several of his blog posts were picked up by Mary Jo Foley, as “Just how closely is Microsoft studying the open-source playbook?”  Sweet.

And now… FY08 here we come!!