Todos los objetivos deben estar reflejados o traducidos a resultados financieros.

La siguiente frmula nos

permite determinar el nivel de valor de una actividad, tarea, proceso, producto o servicio en trminos
financieros desde la perspectiva del cliente:

siendo para el caso:

V = Valor. Es la efectividad y/o productividad en trminos de rentabilidad o utilidad de un objetivo.

Q = Calidad. Grado en el cual el producto o servicio cumple con las expectativas.

s = Servicio. Nivel de satisfaccin del cliente por la calidad, precio y oportunidad del producto o
servicio recibido.

c = Costo. Insumos requeridos para generar el producto o servicio.

t = Tiempo. El grado de oportunidad en que se recibe el producto o servicio.
Antes de comenzar el proceso de construccin de un Balanced Scorecard, se necesita relevar
informacin estratgica que servir como entradas para los primeros pasos de la construccin del tablero.
Se sugiere preparar una checklist de fuentes de informacin estratgica, incluyendo cosas como planes
estratgicos, planes financieros, planes de mercadotecnia, planes operativos, reportes anuales,
programas de mejora de la calidad, anlisis de los consumidores, entrevistas con la gerencia ejecutiva,
otros documentos de planeamiento, anlisis competitivos, anlisis de tendencias de la industria, anlisis
de tendencias tecnolgicas, anlisis de tendencias en mercadotecnia y otros anlisis de la industria.
Y hasta aqu van las cuestiones introductorias que quera mencionar para entrar en el planteo inicial
del Balanced Scorecard para el rea de Tecnologa Informtica. Yo diferenciara las cosas en funcin del
tipo de organizacin. Si se trata de una empresa que no es especficamente de base tecnolgica pero
posee un rea en tal sentido, pensara en un Balanced Scorecard transversal a toda la organizacin,
tratando las cuestiones de tecnologa como objetivos dentro del mapa completo. Ahora, si
la empresa fuera de base tecnolgica, me orientara hacia alguno de los modelos especficos, como
por ejemplo:

El Balanced IT Scorecard (BITS), propuesto por el European Software Institute (ESI)que provee
una nueva versin de las cuatro perspectivas originales agregando una quinta en relacin con la gente
People.

El Balanced Scorecard de Advanced Information Services Inc. (AIS), que considera a los
empleados Employee como una perspectiva distintiva, expandiendo as el anlisis a cinco
elementos.

Metrics Identified and Discussed

Productivity Metrics
1. In Sprint Cycle Time (4) - measuring the turnover of stories throughout
the sprint as measured by when a story was started vs. when it was
completed. A shorter cycle time is a good indicator. A longer cycle time is a
poor indicator.
2. In Sprint Work in Process (3) - measuring the continuous work in process
flow during a sprint relates to the flow of the team. More work in process is a
poor indicator. Less work in process is a good indicator. This assumes the
lean principle that less work in process leads to higher throughput.
3. Sprint Completion Bar - measuring sprint points using a stacked bar with 4
elements of Done (green), Not Tested (yellow), Not Coded (orange), and
Waste (red). Each element presented in a separate color. The stacked bar
shows work sprint over sprint. Longer Done bars is a good indicator. Longer
Not Tested or Not Coded are a poor indicator. Any waste (as defined by work
that was subsequently removed due to not needed) is a very poor indicator.

Productivity Myths
1. Completed vs. Commited % (a.k.a. Earned Value)- measuring the
complete vs. the original sprint plan commitment expressed as a
percentage. Used to say if > 80% then the team succeeds the sprint, less
than that the team fails. This is often misused because if pushed, teams will
tend to either be safe in their commitment (e.g. lower productivity) or hide
unfinished work.This metric can be used effectively if the term "failure" is
removed and the team is not pushed to meet a certain percentage.
2. Velocity - measured by number of points completed per sprint. If used as a
productivity measure, it may initially drive the team to increase productivity,
but if pushed, will eventually lead to story point inflation. Avoid using
velocity to compare different teams. Rather, velocity should be considered as
a predictability measurement.
3. Individual Performance Reviews - Most companies use performance
reviews to measure individuals for compensation. These tend to drive
individual (anti-team) behaviors that work against team productivity.
4. Resource Utilization - Measuring individual percentage busy-ness factor.
Teams that are pushed to utilize each resource fully tend to locally optimize
their teams - which means their whole system is sub-optimized. This is a
lean principle trying to increase team throughput requires sub-optimizing
individuals.
5. Business Value measuring performance - Attempting to measure
productivity through delivery of business value counters a typical prioritized
backlog approach. If properly prioritized, higher valued stories will be at the
top of the product backlog and thus early sprints will deliver more value. If
value is used to measure productivity - it will lead to dysfunction when lower
valued, but higher complexity stories enter the backlog. Rather, this metric
should be used to validate the prioritization of the backlog - not performance
of the team.

6. Source Lines of Code (SLOC) - measured by the number of lines of code

to determine productivity. A higher number does not necessarily translate to
higher productivity - only more code. Using this as a measurement will go
against the principles of "simplest thing possible" and "re-factoring" to keep
code simple. A productive story may actually reduce code, not increase it.

Quality Metrics
1. Technical Debt Points (12) - measuring the volume and throughput of
technical debt to determine the quality evolution of a product. A higher
number of points is a poor indicator. A lower number of points is a good
indicator. This can be used with a technical debt limit - if the debt exceeds a
certain value, the team will be put into a forced debt reduction format. This
can also be used in a stacked bar sprint over sprint showing technical debt
vs. new stories.
2. Running Automated Tests (4) - measuring unit and functional automated
tests that are passing each sprint. A higher (and growing) number is a good
indicator. A lower (or flattening) number is a poor indicator. This metric was
detailed by Ron Jeffries. Paired with code coverage metrics, it can drive both
productivity and quality behaviors.
3. Post Sprint Defect Arrival (3) - measuring the number of defects that are
found after the sprint they were initially developed. A higher number is a
poor indicator, a lower number is a good indicator. This is a leading indicator
of quality in that it attempt to predict released quality based on incremental
sprint quality. Reminder - this is a team metric - not a QA only metric.
4. Post Release Defect Arrival (1) - measuring the number of defects found
after release to customers. A higher number is a poor indicator, a lower
number is a good indicator. This is a lagging indicator of quality meaning
that quality is not measured until after release. Reminder - this is a team
metric - not a QA only metric.
5. Root Causes Fixed - measuring the number of defects that defined and
fixed a root cause. A higher number is a good indicator, a lower number is a
poor indicator. This works well in a sustaining team environment to drive
fixing core issues, not just the symptoms.

Quality Myths
1. Number of Test Cycles (1) - measured by how many times a story or
sprint iterates between development and test. As presented, it was indicated
as a lower number is a good indicator. This metric, however, could lead to
fewer feedback loops.
2. QA Story Points - measure stories in separate engineering and quality
points. This allows QA teams to better predict their productivity and
throughput. The concern is that this separates the engineering and quality
teams going against the Scrum principle of cross-functional teams. It also
limits the engineering/test conversation which is critical to effective Scrum
teams.
3. Quality Number of Defects - measuring Quality and Quality productivity
by the number of defects they find. A higher number is thought to increase

quality and measure productivity, however it goes against the Scrum team
principle of whole team. It is better to encourage both QA and developers to
work towards story completion - not
4. Quality Checked After Development - not really a metric, but not a good
thing to do.

Predictability Metrics
1. Enhanced Burndown Chart (12) - measuring both velocity and scope
change throughout a release as a measure of meeting a release goal predictably. This metric can be combined with cost metrics (below) to
measure predictability in project costs.
2. Velocity (4) - essentially a subset of the metric above.
3. Cost per Story Point (2) - measuring the $ per story point by knowing the
number of people on the team, the sprint length and the story point velocity
to determine the average cost per story point. This can also be used to
measure capitalized vs. non-capitalized expenses in IT firms by separating
the story points which are capitalized and non-capitalized on projects. It is
recommended to use a average number for both costs and velocity by using
a number (3) of sprints.
4. Hours per Story Point - measuring the average number of hours estimated
per story point. Used to determine if the estimated labor to implement story
points is changing over time. For example, a team that starts out at 10 hours
per point on average migrates to 20 hours per point shows that the point
value is decreasing over time. This metric can easily be misused either by
the team in estimating points using the reverse of this metric or by leaders
pushing teams to decrease their estimated hours per point (assuming they
are making them more efficient). This should only be used to evaluate
predictability. Other methods are to use a "reference" story, or to create a
estimated labor hour range per story point rather than a specific number.

Value Metrics
1. Business Value Delivered (7)- measuring the value delivered each sprint
based on assigning a business value to each item in the backlog. A higher
number is a good indicator. This can also be an employee motivator because
teams want to deliver value and feel good about being able to. As mentioned
above, this can be dysfunctional if teams attempt to sustain business value
over the course of multiple sprints because if the backlog is prioritized
correctly, business value should be lowering over the course of the release.
2. Customer Satisfaction Survey (6) - measuring feedback quantitatively
and/or qualitatively from customers and other stakeholders through a regular
(quarterly) survey. Various feedback can be gathered on quality,
predictability, productivity of delivery, support, appropriateness of new
features, etc.
3. Employee Satisfaction Survey (3) - measuring the qualitative and
quantitative feedback internally from employees with a regular survey
(quarterly) and tracking the results over time. This can measure
effectiveness at roles, quality of work/life balance, teamwork, product

definition, process adherence, feedback, happiness value, etc. A happy

workforce tends to increase productivity due to people wanting to be at work
and contribute to team results.

Metrics In Agile

Home
Metrics
Metrics in Agile

Christian Miles Thursday, August 06, 2015

I've been meaning to write a blog post all about metrics for a very long... It's a really
important subject and one that doesn't get spoken about anywhere near enough - I
also plan to come back and add to this as time allows.
Metrics in Kanban as opposed to Scrum is perhaps more common place, Kanban is
much more a case of bench marking where we are, Conducting 'experiments' and
measuring the result before rolling back or continuing.
But, Good metrics not only allow you to understand the effects of experiments
made but also allow you to start planning with at least something approaching a
scientific methodology! Although admittedly it does take more effort than the old
finger in the air, double it + a safety margin trick that!

Below are some of the main metrics I use regularly.

Flow efficiency
I love this little metric - It gives a quick indication as to how efficient your process is.
The exact way you record it might differ based on what you're trying to measure...
but roughly, Record the date a story is requested, record the date it's released and
record the number of days it was actually worked on.
So if a story takes 20days from being moved from the backlog to the todo column
and is worked on (developed/tested/etc) for 5 days you have a flow efficiency of
5/20= 25% Which although looking low is probably a really good figure!

Cumulative flow diagrams

I love a good CFD... It's a really good visual way to track a project. Most software
tools will allow you to generate a Cumulative Flow Diagram - Otherwise you can
export the data to Excel and generate one.
Generally speaking the closer together the lines the lower the WIP (Work In
Progress) and shorter the lead times should be and the more work should get done
(green)
You can see from the graph a very flat section early on where no work was getting
done - That was before I joined as Scrum Master ;-) And another slight decrease in

work 'done' and a longer lead time about 3/4 along... That could be a cause for
concern.. It was actually the Christmas/New Year period but it shows graphically
how well a team is working and how much work is left - These are great to take into
retrospectives.

Lead Times
Lead times.... Are simple, How long from requesting a story (Moving from the
backlog to todo) until release (or 'Done')
However you probably want to start recording lead times by story/ticket type.....
If your using story points starting plotting lead-times against story points using a
histogram.
You start to get some really useful information back,
For example - You might be able to say that 85% of stories estimated at 13SP are
delivered in under 5 days but up to 5% take longer than 10days.. However 50% are
delivered in 3 days or less.
Having powerful metrics like this allows you to really plan... You can pick the right
time to start doing work and know with a level of confidence when it will be
complete.
You can prioritise stories more precisely - Knowing you can leave that really
important legal change story until 10days before it's required as it's estimated at
13SP story which than means you can work on the 20SP story first as it'll save Xthousands every day!

Cycle Times
Cycle times are very common in Kanban and Scrumban...
It's a measure of the number of tickets in progress and the number of tickets
completed daily - You can sample at different rates but I tend to run Scrumban in
weekly cycles but different teams will work differently.
So... To keep the numbers simple, If you have 10 tickets in progress and complete 2
a day (average) - You cycle time is 5. 10/2
You should find if WIP increases so will the cycle time and the opposite too!
Start plotting cycle times and you'll begin to see trends... And trends are how you
know it you're improving... or not! It's also useful to start plotting different types of

tickets together - So below we can see that the cycle tine in May/June increased
from the norm... But so were the cycle-times for bugs... You can't draw a conclusion
from this but you can ask questions and understand.

Happiness/Satisfaction
This probably gets overlooked more than any other measurement. But measuring
the happiness of your team and customers is probably one of the most important...
Reality is important but perception probably more so!
This can be especially important if running an agile transformation... Bench mark
how happy the team is! Understand your customers issues/concerns and overall
level of happiness with the service.

You could create a mood wall, questionnaires, etc. The The mood wall in particular
are really good for discussing in retrospectives.

Velocity!

Well I couldn't write a blog covering metrics without mentioning it... But I don't
really want to discuss it too much on this blog, Velocity is fine for an internal team
metric and helping to predict burn-down against a backlog or for allowing
meaningful negotiation about story sequencing, etc.. But don't get carried away
with it.. Try not to chase it and don't let people abuse it!
And if you do want to know more..... Velocity

Burn down
The burn-down chart - These are very common in Scrum, both for tracking progress
in a sprint and for measuring against the backlog.
I must admit... I'm not really a great fan of the Sprint Burn-down - It seems to have
become another chart that management overuse (Like velocity).
You should be able to track progress of stories completed via the Kanban/Task
board.
If you are doing a sprint burn-down - Perhaps rather than looking at hours against
tasks and burning down perhaps looks at the number of stories in the sprint and
burn down against that measurement - It's actually recording what's important
(Completed stories)
I do however like a burn-down against a backlog (Although if you're using the CFD this is less important) If you do have a well estimated well 'refined' backlog and you
have a stable velocity figure these can be useful to either predict delivery or to alert
early on that you're going to miss a release point.

In Conclusion
Metrics are important... They allow you to safely change the workings of a team
and understand the effect. Management typically prefer's this way of working, it's
safer and you have evidence to back up decision making.
At the planning level metrics are invaluable - And allow you to plan with some
confidence.... Although planning when you have actual metrics suddenly becomes a
much more complex and tiring process than just guessing it!
But you have to them use metrics responsibly and remember Goodhart's law
- "When a measure becomes a target, it ceases to be a good measure."
Try and keep away from absolute numbers and monitor trends instead - The CFD's
are brilliant for this - Look at the general trend not specific numbers. Don't chase a
number but try and understand why those numbers are changing.

Agile Velocity

Home
Velocity
Agile Velocity

Christian Miles Friday, September 19, 2014

I was in a meeting recently explaining some ideas on how we could track velocity
within a team I was leading. When the ops manager I was talking with finally came
clean and admitted he had no idea what velocity was! And why was everybody
talking about it and could I just explain in simple terms what it was all about!
Well that's one of the problems with IT and especially agile people... we do love our
buzz words.

Planning Poker being played

So what is velocity?
Well - There's probably lots of subtly different definitions.. but put simply - It's a
measure of how quickly the team deliver's 'done' items of functionality over a
sprint.
I've just read that back and to be honest there is nothing simple about that
explanation! So no wonder he still looked confused afterward!
So perhaps if I explain how velocity is calculated it will help!

So how do you calculate the team's velocity?

With every sprint a number of user-stories should be selected and committed too..
Each of these stories should be estimated with a number of story points, typically
these are based more or less on a Fibonacci number sequence scale (1/2,
1,2,3,5,8,13,20,40,100) - Fibonacci number sequences are fascinating being found
in nature and used in computer science (Fibonacci search technique and Fibonacci
heap to name just two uses) - The numbers we use in planning poker don't exactly
stick to this sequence.... 1/2 has no place whatsoever in a Fibonacci sequence and
40 should be 33!
There are various methods of estimating work - planning poker is just one and I've
blogged about several techniques in the past - but for now suffice to say that every
story (and perhaps task) of work should be estimated with a relative amount of
effort to complete.
At the end of the sprint you sum up all the story points which are 'done' (remember
it's important to enforce that definition of done) and that's the team's velocity!
So - If you have 10 stories and via planning poker or some other estimation
technique you decide that each task is worth 8 units of effort and you complete all
10 stories the velocity for the team in that sprint is 80.
You can now start to use this figure in planning - Both for helping to establish how
much work the team should commit to within a sprint - The best logical order to do
work and for producing burn-down or burn-up charts (my favourite is a burn down
chart)
I tend to use an average velocity figure for planning - You should expect that the
velocity figure will probably change over sprints (hopefully upwards)
In the past I used to calculate the average from the 2nd sprint onwards (I always
assume the 1st Sprint might be a little different as the team beds in) and than use
standard deviation to exclude the unusual sprints! more recently I've been
calculating the average on just the last 3 or 4 sprints - I don't have definitive proof!
but this does seem to give a more realistic velocity figure and tends to allows for
improvements to filter through quicker to the average and be reflected in the burndown charts
Hopefully that's explained Velocity!