Software Process Management and Metrics

Topic #2
Measurement

Measurement

Motivation
No metric, no movement

You can neither predict nor control what you cannot measure
S. Bhatti, Software Process Management and Metrics 2

Measurement

What is measurement?

S. Bhatti, Software Process Management and Metrics

Measurement

Measurement is the process by which number or symbols are assigned to attributes of entities (objects) in the real world in such a way as to ascribe them according to defined rules.
Objects Object Attributes Attribute 1 Value 11 Attribute 2 Value 21
Attribute n Value n1

Values Value 12 Value 22

Value n2

Value 13 Value 23
Value n3
4

S. Bhatti, Software Process Management and Metrics

Measurement

It is wrong to say we measure an object or an attribute

For example, it is wrong to say we measure a room or that we measure height, from a measurement point of view, although we use such language in our daily lives all the time

Then what is the proper way to say these things?

S. Bhatti, Software Process Management and Metrics

Measurement
The height of a person is a commonly understood attribute of a person, which can be measured. Can intelligence of a person be measured? Can we measure the height of a person in meters, inches or feet? Which is correct? Can we measure the height of a person in miles instead of feet or inches? If so why, if not why not? It is meaningful to take the average height of a football team. Why? Is it meaningful to take the average of the jersey numbers of the team? Why?
S. Bhatti, Software Process Management and Metrics 6

Measurement

Metrics standards (i.e., commonly accepted scales) that define measurable attributes of entities, their units and their scopes.
For example, prices of clothing (number of Pak Rupees) and sizes of clothing (XL, L, M, S)

Measure is a relationship between an attribute and a measurement scale.

For example, the height of a person and inches

S. Bhatti, Software Process Management and Metrics

Measurement

Metrics Example
Entity Requirement Specification Design Attribute Size, Reuse, Redundancy Size, Reuse, Redundancy Size, Reuse, Modularity, Cohesion, Coupling Metrics for Size # of requirements # of features # of classes # of lines of code (LOC) # of test cases

Code

Size, Reuse, Modularity, Cohesion, Coupling, Complexity

Size, Coverage
S. Bhatti, Software Process Management and Metrics

Test Cases

Measurement

Measurement Example: Developed Software

S. Bhatti, Software Process Management and Metrics

Measurement

Measurement Example
1. 2. 3. 4. Select entities: Developed Software Select attributes: Size Select metric for attributes: Number of LOC Specify rules (relationships) ascribed to the attributes and entities:

Developed Software = Developed Software t=0 + (Software Development Rate) dt Where Software Development Rate = LOC/dt

S. Bhatti, Software Process Management and Metrics

10

Measurement

Another Measurement Example

1. 2. 3. 4. Select entities: Availability Select attributes: Uptime, downtime Select metric for attributes: time in seconds Specify rules (relationships) ascribed to the attributes and entities: Availability = uptime / (uptime + downtime)

S. Bhatti, Software Process Management and Metrics

11

Measurement
Challenges of Measurement in Software Engineering
Entity Physical Non-physical Human Human attribute Height IQ value 178 89 units cm IQ index

Software Engineering metrics are mostly non-physical Availability Reliability, Maturity, Portability, flexibility And relations are hard to quantify, but not hopeless!
S. Bhatti, Software Process Management and Metrics

12

Measurement
Challenges of Measurement in Software Engineering
Metrics can be misleading
Fact 1: Knowledge is power Fact 2: Time is money Rule/Relationship: power = work /time The harder you work the more power you have As power approaches infinity, time approaches 0 Substituting power and time Knowledge = work/money The less you know, the more you make! As knowledge approaches zero, money approaches infinity
S. Bhatti, Software Process Management and Metrics 13

Measurement

What does measurement in software engineering help us do?

S. Bhatti, Software Process Management and Metrics

14

Measurement
Measurement in software engineering help us understand, control and improve:
The product The people (control in a good way, i.e., productivity) The technologies (for example, software configuration management) The process

But Measuring in software engineering is relatively new and controversial It is not yet clear which are the appropriate metrics for a software project or whether people, processes, or products can be compared using metrics
S. Bhatti, Software Process Management and Metrics 15

Measurement
Measurement in software engineering entails selecting, measuring and combining many different attributes of:
Software products Software development resources Software development technologies Software development processes This includes metrics which are directly measureable, such as lines of code (LOC), as well as items which are calculated from measurement, such as software quality.
S. Bhatti, Software Process Management and Metrics 16

Measurement
Who benefits from measurement?

S. Bhatti, Software Process Management and Metrics

17

Measurement
Managers

What does each step of the process cost? How productive is the staff? How good is the code that is being developed? Will the user be satisfied with the project? How can we improve?
Are the requirements testable? Have we found all the failures Have we met our process or product goals What can we predict about our software product in the future

Engineers

Users?
S. Bhatti, Software Process Management and Metrics 18

Measurement

What is the scope of software metrics?

S. Bhatti, Software Process Management and Metrics

19

Measurement

Scope of software metrics include:

Product

Process

People

Technology

S. Bhatti, Software Process Management and Metrics

20

Measurement
Scope of software metrics
Cost effort estimation Productivity measures and models Data collection Quality model and measures Reliability models and measures Performance evaluation and models Structural and complexity metrics Capability maturity assessment Management metrics Evaluation Methods and tools
S. Bhatti, Software Process Management and Metrics 21

Measurement Scope of software metrics

Cost effort estimation
Software cost estimation is the process of predicting the amount of effort required to build a software system Estimates for project cost and time requirements are derived during the planning stage of the a project Models used to estimate cost can be categorized as either cost models (e.g., Constructive Cost Model COCOMO) or constraint models (e.g., Software Life cycle Management SLIM) based on linear programming
S. Bhatti, Software Process Management and Metrics 22

Measurement
Scope of software metrics
Productivity measures and models
Productivity = size/effort, LOC/person month Productivity model based on decomposition to measureable attributes:
Productivity

Value

Cost

Quality

Quantity

Personnel

Resources

Complexity

Reliability

Defect

Size

Functionality

Time

Money

Hardware

Software

Constraints

Difficulty

S. Bhatti, Software Process Management and Metrics

23

Measurement Scope of software metrics

Data collection
Very critical and very hard step 1. What Data should be collected? 2. How it should be collected? 3. Is collected data reproducible? Example: software failure data collection 1. Time of failure 2. Time interval between failure 3. Cumulative failure up to a given time 4. Failures experienced in a time interval
S. Bhatti, Software Process Management and Metrics 24

Measurement
Scope of software metrics
Quality model and measures
McCalls quality factors

S. Bhatti, Software Process Management and Metrics

25

Measurement

Scope of software metrics

Reliability models and measures
Plot the change of failure intensity () against time Models: basic exponential model and logarithmic Poisson model The basic exponential model assumes finite failures in infinite time; the logarithmic Poisson model assumes infinite failures
S. Bhatti, Software Process Management and Metrics 26

Measurement

Scope of software metrics

Reliability models

S. Bhatti, Software Process Management and Metrics

27

Measurement

Scope of software metrics

Performance evaluation and models
Using externally observable performance characteristics such as response time and completion rate Efficiency of algorithms (Big O Notation)

S. Bhatti, Software Process Management and Metrics

28

Measurement Scope of software metrics

Structural and complexity metrics Structural
Control flow structures Data flow structures Data structures Information flow attributes

Complexity
Cyclomatic complexity, that is, defining number of independent paths in the execution of a program
S. Bhatti, Software Process Management and Metrics 29

Measurement Scope of software metrics

Capability maturity assessment
Software Engineering Institute (SEI) model CMMI: Level 1 trough Level 5

ISO 9001
Model for quality assurance in design/development, production installation and servicing

ISO 9000-3
Guidelines for application of ISO 9001 to the development, supply and maintenance of software
S. Bhatti, Software Process Management and Metrics 30

Measurement Scope of software metrics

Management metrics
Metrics for project control 1. Metrics for requirements and specification 2. Metrics for the design model 3. Metrics for code 4. Metrics for testing 5. Schedule metrics 6. Resource metrics 7. Change metrics
S. Bhatti, Software Process Management and Metrics 31

Measurement

Scope of software metrics

Evaluation Methods and tools
Benchmarking Certification Configuration management system Requirements management system Defect management system Integrated Development Environment Development Tool Chain
S. Bhatti, Software Process Management and Metrics 32

Research

For Your Research.

You will need to know
Process Modeling System Dynamics The calculus underlying process dynamics Rates and Levels Software Process Modeling Modeling and Simulation Tools

S. Bhatti, Software Process Management and Metrics

33

Research
Topic #5 will formally introduce you to Process Modeling Topics #6 will formally introduce you to software process modeling AND give you a complete example (based on Brooks claim) of what is expected from you for your research project Topics #2 (today), #3 and 4 will introduce you to some preliminary material on:
Process Modeling System Dynamics The calculus underlying process dynamics Rates and Levels Software Process Modeling Modeling and Simulation Tools

S. Bhatti, Software Process Management and Metrics

34

Research
Software Process

Requirements Resources

Software Process Jan 1, 2011 Dec 31, 2012

Product

Software Project

S. Bhatti, Software Process Management and Metrics

35

Research
Another way to look at the Software Process
Levels Rate

S. Bhatti, Software Process Management and Metrics

36

Research
Yet another way to look at the Software Process
Level: Requirement 10 Requirement 8 FP/Day 6 4 2 0 0 2 4 Days 6 8 10 1.2 1 Rate: 1 FP/Day 10 8 FP 6 4 2 0 0 5 Days 10 0 2 4 Days 6 8 10 Level: Developed SW

0.8
0.6 0.4 0.2 0

Rate = Level/ time Rate Req = - Req/Day Initial + (Rate Req * Day) = Req
S. Bhatti, Software Process Management and Metrics

Rate SW = FP/Day Rate SW * Day = FP

37