Inspect and Adapt

In an agile approach like Scrum, the latest product increment is demoed to the stakeholders in the
sprint review meeting, as the picture below illustrates. (Please see my post “The Scrum Cycle”
for a detailed explanation of the picture.)

While the product demo allows understanding which stories have been completed, I find that
using it as qualitative market research technique unleashes its real potential. Its primary goal is
then to collect feedback from users and other stakeholders in order to validate ideas and improve
the product. The demo is best done in person with everyone being present in the same room, but
you can also conduct it as a videoconference. The following tips should help you leverage your
product demo as an effective market research method.

Be Clear on your Research Goal

Understand what questions you would like to get answers to, and what ideas you would like to
validate before conducting the demo. Your sprint goal should help you with this: If, for instance,
your sprint goal is to test your user interface design ideas, then you should plan the demo
accordingly: You may want to present different versions as mock-ups to the users to understand
which one they prefer and why that’s the case. Having one sprint and research goal helps you
focus the presentation. It increases the likelihood to collect relevant feedback, and it makes it
easier to analyse the feedback.

Invite the Right People

Use your sprint goal to decide who can help you validate your ideas and improve the product and
who should therefore attend the demo. If the goal of the sprint is to establish the right software
architecture decisions, then end users are probably not the right attendees. In the worst case, the
demo could be a frustrating experience and prevent them from attending another review meeting.
But if the goal is to better understand how users are likely to interact with the product, then end
users should be present. Otherwise, you are in danger of collecting lots of interesting but
irrelevant or misleading data.

Explain what the Product does for the User

Avoid listing features and functionality in your demo, and describe what the product does for the
user in order to receive meaningful feedback. A great way to do this is to use a scenario. If you
develop a mobile banking application, for instance, you may want to say: “Imagine you are on
the train on your way to work, and you remember you still need to pay your water bill. You open
the banking app, log on, and then you would see the screen I am showing you now.” If you
employ my Product Canvas, then you should be able to use its scenarios and storyboards for your
demos.

Engage in a Dialogue

An agile product demo should not be a one-way communication or a sales event. Instead, its
objective is to generate valuable feedback that allows you to gain new insights. Unfortunately,
users and other stakeholders don’t always provide helpful feedback straight away. You
sometimes have to ask the right questions and create a dialogue. For instance, if the feedback you
receive is “Great demo, I really like the product”, then that’s nice. But what does it actually
mean? How does it help you, and what can you learn from it? Dig deeper, ask why the individual
likes the product, which aspects are particularly valuable, and which could be improved.

Take Notes

To be able to analyse the feedback afterwards, I recommend you record who provides the
feedback and what you hear and see. Ask the team members to take notes too. This reduces the
risk of overlooking feedback. I also suggest you record relevant background information about
the attendees including demographics and job role. The information will be handy when you
analyse the feedback.