Brooklyn Game Lab

Primarily an after school program where kids (ages 7-13) play modern, multiplayer board games and card games, discuss tactics, analyze mechanics, improve rules, create expansions and become game designers.

Software Gaming It was his victory, not theirs, and a hollow one at that, a cheat; why did they act as if he had won with honor? Ender’s Game in general: addiction | anti-cheat protection: cognisafe | casual gaming:, popcap, | consoles: ouya| developers: Reloaded,, trionworlds,, wooga | development platforms: unity3d | documentary: The Story of Mojang | drive traffic to games: playerize | free: kabam strategy, nexon | game for good: hoopla | genetics: phylo | gesture recognition: omekinteractive | hand motion sensing: eyesight-tech, | industry news:, idlethumbs | in-game advertising: supersonicads | kids: Brooklyn Game Lab after school,  search spongebob | literacy: instituteofplay | massively multiplayer online role-playing games (MMORPG): curse, redrobotlabs, zam | new game a day: bigfishgames | on-demand: chess, gaikai,, kongregate, onlive, | professional gaming: | puzzles: cartalk, | search: gaming | social gaming: playdom, 6waves | 3D: musegames | trading cards: fightmymonster | ugo | video networrk: machinima | virtual trading: | zero-sum gamification big ideas: kids learn to code, and, code to learn | about | blogs: seriousgamesmarket | brain training: lumosity | charity: theplaymob | corporate learning: axonify | education: classroom,, gogolabs, tutpup | forums: gamification summit | health: healthprize, reflexionhealth rehab | kids build games: gamestarmechanic, ScratchJr  | quizzing: quipper | science: global warming | social change & impact: gamesforchange | social rewards & loyalty: badgeville, bigdoor | tools: twochop mobile: | zynga real-time player behavior | King: Candy Crush books, articles, reports: Creating Casual Games for Profit & Fun Partridge | Ender’s Game Card | Fun Inc. Chatfield | Of Dice and Men Ewalt | Reality Is Broken McGonigal | Solving tough problems with games Schrope | Violent Video Games: More Playing Time Equals More Aggression | Why Gamification Is Serious Business Ryan Parenting | Golf | Sci-Fi | Virtual | Investing | Robotics | Google | Biotech

The Heilmeier Catechism Visits the University

by scott pickard

About two years ago, one of our major corporate sponsors came to town to inform us that the rules of the sponsored research game would be changing: If the research output we were delivering was not being used and integrated into their operational and manufacturing operations; and, it was not aligned with their current strategic priorities, then that particular research would no longer be financially supported.

That was a sobering message at the time.  Previous to this meeting we had enjoyed a happy relationship for many years where we would propose interesting research themes, the sponsor would approve and give us notice to proceed, and then we would pursue this research in the traditional way involving graduate students over a 1-2 year timeframe.  Every fall they would travel to our campus where we would present those results at an end-of-year, two-day review meeting that mashed up corporate and academic researchers and graduate students in an intellectually stimulating and enjoyable format.  When it was over, everybody was smiling and shaking hands as we returned to our respective offices and labs to do it all over again.  But now, the harsher (we first thought) new message to us in so many words was, that research approach would no longer cut it.

So from that day forward, we proceeded to go through a constructive transition and reengineering process with the sponsor where we listened a lot and did our best to learn and apply some new principles and tools that they recommended to guide the development of our next research proposals, plan, and budget.  It’s fair to say that our transition to and acceptance of this new level of accountability was stressful and possibly not easily embraced by all faculty.

You see, University faculty researchers pride themselves as big thinkers working on the big ideas out on the leading edge of current knowledge, and this kind of research cannot be “put on the clock” because researchers are attempting to stretch the limits of current knowledge and it is therefore a very risky endeavor with no guarantee of success.  Many great world-changing innovations have been built on the “shoulders of giants,” but only after many iterations of research trials and failures.

Also, the academic research culture has a very strong immune response against any corporate-like tactics such as project management, milestones, and performance metrics.  If this sounds like a criticism, it’s not, it’s just the way it is and has been for a long time, and history has shown that when given enough time and money, academic research has yielded some of the most fantastic and world-changing discoveries underpinning our society.

So why the sudden change of tune from the sponsor?  What’s the problem?

The problem is, our global community is under extreme economic pressure that has been building over years which is threatening the financial sustainability of countries, states, cities, corporations, and yes, even universities.  The impact can be seen, heard, and felt in major corporations that are forced to deliver more for their internal and external (sponsored) research dollar, which tends to push their research priorities a bit closer to the applied side of the scale.  And it’s not just corporations that are bringing this message, but also our major federal research sponsors.  What they really want, if they can get it, is both: breakthrough technology over the medium to long-term , while also demonstrating to politicians and taxpayers some real applied results in the near-term for obvious political and accountability reasons.

There should be no surprise in this behavior because this is what we do as individuals, families, groups, organizations, and governments when times get tight and tough, and we have to adapt to these conditions as necessary.  In this particular case, these were the new requirements from the sponsor:

The research topics we proposed needed to be in alignment and support the strategic priorities of the sponsor.  In our defense, we had never seen such a list from our sponsor.  It was revealing and something we needed to see.  How else could you be in alignment?

Each specific research project we proposed needed to be assessed and validated by the (well-known) Heilmeier Catechism.  I put (well-known) in parenthesis because at that time, there were maybe a few faculty researchers who were vaguely familiar with the Heilmeier Catechism (questions), but none of them had ever been truly accountable to them.  Heilmeier used this standard set of questions at DARPA to review and screen new R&D projects or funding proposals:

  • What are you trying to do? Articulate your objectives using absolutely no jargon.
  • How is it done today, and what are the limits of current practice?
  • What’s new in your approach and why do you think it will be successful?
  • Who cares?
  • If you’re successful, what difference will it make?
  • What are the risks and the payoffs?
  • How much will it cost?
  • How long will it take?
  • What are the midterm and final “exams” to check for success?

Each proposed research project had to be summarized in a “quad chart” which put visual organization to the answers to the Heilmeier Catechism and put a no-nonsense look-and-feel to each research proposal value proposition.

A sample quad chart from a DARPA project.

It should be made clear that this particular case is about one sponsor on one funded center and it did not instigate a sweeping change in research practices across all researchers and projects at the University.  The academic research culture, traditions, protocols and processes at this University have been forged since the 1800s, so no one sponsor or event is going to change that overnight in any revolutionary way, and, why should it?  It  is a research model that has worked spectacularly as measured by the generation of fundamental research and intellectual property integrated across all U.S. research universities over 100 years and is the envy of the modern world.

Even so, we did respond to our sponsor’s new requirements in the following ways:

  • We developed an increased sense of urgency about what was at stake.  We had enjoyed a long, trusting, and we think successful relationship with this sponsor which we highly valued, so we were strongly motivated to maintain and strengthen that relationship if we could.
  • We listened harder to what our sponsor was saying about what was strategically important to them, and built a research plan to support those priorities.
  • We strived for a mutually-acceptable balance between the need to show practical results, and the need to stay true to our mission as a premier research university.  Our sponsor understood and respected our commitment to leading-edge research which provides high-quality experiences for our graduate students that prepares them for productive and successful careers in academia and industry.  That’s why they came to us in the first place.
  • We improved our two-way communication channels by more calls, on-site visits (both directions), and by deploying  a secure collaboration platform (messaging, file sharing, to-do lists, milestones) for day-to-day connectivity.
  • As requested by the sponsor, we addressed the Heilmeier questions at the front-end of each project and later assessed and defended our results against the Catechism at the back-end of each project.

Today, we still enjoy a strong and ongoing research collaboration with this sponsor and there is no question that we have fundamentally changed for the foreseeable future the way we work together.  But it would not be true that the “Heilmeier way” has gone viral across all of the University’s researchers and projects – far from it.  But we must acknowledge that this experience made an impact on our organization and from a technology transfer perspective – not a blue-sky research perspective – it was a useful and positive impact.

Any research operation whether corporate, university, or national lab would be well-served to seriously consider the merits of the Heilmeier principles.  Two articles about Heilmeier give a good overview of the game-changing impact this man and his Catechism have been making on high-technology development since his ground-breaking work with liquid-crystal displays (LCD) at RCA Laboratories in the ‘60s:

Based on his work with LCDs, Heilmeier received the 2012 Draper Award “for the engineering development of the LCD that is utilized in billions of consumer and professional devices.”  But the legacy of his achievements most visibly lives on through these nine simple questions which continue to be adopted and adapted by researchers, engineers, managers and entrepreneurs to inspire and guide the development of new products and solutions.

So if it’s true that every innovation starts with a question, then clearly we need to take care to ask the right questions.