You are on page 1of 4

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 9, December 2010

Adaptation of GQM Method for Evaluating the


Performance of Software Project Manager
Nagy Ramadan Darwish Mahmoud M. Abd Ellatif
Department of Computer and Information Sciences, Department of Information Systems,
Institute of Statistical Studies and Research, Faculty of Computers and Information,
Cairo University Mansoura University
Cairo, Egypt Mansoura, Egypt
drnagyd@yahoo.com drmmlatif@yahoo.com

Abstract—This paper is concerned with evaluating the For example, scope management includes the following
performance of software project manager using Goal Question activities:
Metrics (GQM) method. It clarifies the Software Project  Identifying the project background.
Management (SPM) domains and the performance metrics of each  Assessing the initial feasibility of the project.
domain. This paper presents the basic concepts of GQM method.
Based on a combination of statistical techniques, this paper  Defining project scope and deliverables.
presents how to apply GQM method to evaluate the performance  Listing project assumptions and constraints.
of a software project manager. A software company can use the  Verifying the project scope.
proposed approach to track, evaluate, control, correct, and  Establishing procedures for tracking project progress.
enhance the performance of software project managers to increase  Assessing the project feasibility.
the success rate of software projects.
 Managing project scope changes.
Keywords Software ; Project Manager; Performance; Evaluation  Tracking project progress.
- GQM – Metrics – Performance Report
System improvement requires measurement and analysis
I. INTRODUCTION AND PROBLEM DEFINITION [21]. Performance measurements are used in project
SPM is the on-going activities for planning, organizing, management and quality processes to determine and
directing, and controlling progress to develop an acceptable communicate status and accomplishments measured against
system, i.e. conform to the quality standards within the specific objectives, schedules, and milestones. These
allocated time and budget [11]. The mismanaged projects may measurements extend to include delivery of desired products
lead to: unfulfilled or unidentified requirements, uncontrolled and services to customers, whether external or internal [3].
change of project scope, uncontrolled change of technology, Performance measurement can be useful to improve future
uncontrolled risk of the project, uncontrolled subcontracting work estimates [15]. Performance measurement is the ongoing
and integration, cost overruns, and/or late delivery [12]. The monitoring and reporting of project accomplishments,
failure rate of large software projects is larger than the success particularly progress towards pre-established goals.
rate [16]. Therefore, there is a need to track, evaluate, control, Performance measures may address: the type or level of project
correct, and enhance the performance of software project activities conducted, the direct products and services delivered
managers to increase the success rate of software projects. by a program, and/or the results of those products and services
[2].
GQM method can be used to evaluate the performance of
software project managers. This method was developed for A metric is a quantitative measure of the degree to which a
multi-purpose evaluation of software. GQM method consists of system, component, or process possesses a given attribute [7].
three steps: determination of a Goal, construction of sets of Performance metrics should be objective, timely, simple,
Questions which have possible answers. The last step is accurate, useful, and cost-effective. The performance metrics
analytic of the set of metrics, which consists of weight can be divided into three basic categories [2]: measures of
coefficient for each set of answers [22]. efforts, measures of accomplishments, and measures that relate
efforts to accomplishments.
II. SPM DOMAINS AND PERFORMANCE METRICS
The researchers elaborated a set of performance metrics
SPM activities can be organized in nine domains [9]: for scope management. These elaborated performance metrics
integration management, scope management, schedule can be calculated at specific time check points such as project
management, costs management, quality management, human milestones. In addition, they can be calculated for a given time
resources management, communications management, risk interval such as a week, month, and so on. Therefore, they can
management, and procurement management. Each domain be calculated weekly as an integral part of project progress
includes a set of activities related to a specific field in SPM report. The elaborated performance metrics for scope
practices. management include:

304 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 9, December 2010
 Percentage of users involved in defining scope and adaptation of GQM method to fit the performance evaluation of
deliverables vs. planned. software project manager. Each software company includes a
 Percentage of project deliverables achieved vs. planned. quality group to evaluate and track the performance of the
 Percentage of project deliverables reviewed and approved software project managers.
vs. achieved. Calculating metrics is a simple process because it depends
 Percentage of major milestones met vs. planned. on simple or known statistical or mathematical formulas such
 Percentage of project team meetings achieved vs. planned. as Return On Investment (ROI), payback, percentage, ratio,
 Percentage of scope change requests subjected to cost deviation, and time deviation (in hours, days, weeks, or
feasibility studies vs. all requests. months).
 Percentage of scope change requests subjected to
integration tests vs. all requests. Goal Metrics
Question
 Percentage of scope change requests subjected to
configuration management tests vs. all requests.
 Percentage of preliminary feasibility studies achieved vs.
required at the initiation phase.
 Return On Investment (ROI) calculated for the potential Goal = SPM
Metric 1

preliminary project’s costs at the initiation phase. Domain


Question 1 Metric 2
 ROI calculated for the potential detailed project’s costs at
Goal 1:
the planning phase. Integration Question 2 Metric n
 Payback period calculated for the potential preliminary Management
project’s costs at the initiation phase. Question n

 Payback period calculated for the potential detailed


project’s costs at the planning phase. Question 1
 Percentage of detailed feasibility studies achieved vs.
required at the planning phase. Goal 2: Question 2
Scope
III. GOAL QUESTION METRICS METHOD Management
Question n

Victor Basili and et al at Maryland University developed a


goal oriented approach for measurement [5]. This approach Question 1
depends on three steps:
Goal 3:
 Set goals specific to needs in terms of purpose, Schedule
Question2

perspective, and environment. Management


 Refine the goals into quantifiable questions that are Question n

tractable.
 Deduce the metrics and data to be collected (and the means
for collecting them) to answer the questions.

In GQM method, each goal generates a set of quantifiable


questions that attempt to define and quantify this goal. The
question can only be answered relative to, and as completely Question 1
as, the available metrics allow. In GQM, the same question can
be used to define multiple goals. Also, metrics can be used to Goal 9:
Question 2 Metric 1
procurement
answer more than one question. Unfortunately, this approach
Management
suits to mature and well-understood problem areas. Question Metric 2
n

IV. THE PROPOSED APPROACH FOR EVALUATING Metric n


PERFORMANCE OF SOFTWARE PROJECT MANAGER
Figure (1): GQM Method and SPM domains.
GQM method can be used to evaluate the performance of
software project managers. Each SPM domain has a goal, for Figure (2) illustrates a general flowchart that presents the
example schedule management aims to administrate and proposed approach. The proposed approach includes the
control of the finite resource of time. The three conventional following main procedures:
measures of project success are budget, schedule, and  Define the goals, questions, and performance metrics.
functionality. The project manager must manage the schedule
 Calculate the value of each performance metric and
carefully for preventing or correcting any slippages. Each goal
compare it with the accepted range of the metric value.
is decomposed into several questions, and each question can be
 Interpret and analyze the performance report.
answered by a metric or more. Figure (1) illustrates the

305 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 9, December 2010

Table (1) illustrates an elaborated list of questions and


A. Define the Goals, Questions, and Performance Metrics
performance metrics for scope management.
Each SPM domain have a goal, therefore there are nine
goals: integration management, scope management, schedule
Question Metric
management, costs management, quality management, human Q1: Is the users 1. Percentage of users involved in defining scope
resources management, communications management, risk involved in defining and deliverables vs. planned.
management, and procurement management. Each goal is scope and
decomposed into several questions, and each question can be deliverables?
answered by a metric or more. The definition of each metric 1. Percentage of project deliverables achieved vs.
should include the mathematical or statistical techniques for Q2: Is the project planned.
deliverables planned, 2. Percentage of project deliverables reviewed and
calculating this metric. achieved, reviewed, approved vs. achieved.
and approved.
Start
Q3: Is the major 1. Percentage of major milestones met vs. planned.

Define the goals, questions, and


milestones of the
Performance metrics project met?

Q4: Are the project 2. Percentage of project team meetings achieved


team meetings vs. planned.
Calculate the value of each performance metric and compare it
with the accepted range of the metric value achieved?
1. Percentage of scope change requests subjected
to feasibility studies vs. all requests.
Interpret and analyze the performance report
Q5: Are the scope 2. Percentage of scope change requests subjected
change requests dealt to integration tests vs. all requests.
well? 3. Percentage of scope change requests subjected
End to configuration management tests vs. all
requests.
Figure (2): The proposed approach for Evaluating Performance 1. Percentage of preliminary feasibility studies
of Software Project Manager. achieved vs. required at the initiation phase.
2. Return On Investment (ROI) calculated for the
potential preliminary project’s costs at the
In addition, the definition of each metric should include the initiation phase.
accepted range of the metric value. The time check points for 3. ROI calculated for the potential detailed
calculating performance metrics should be determined. These Q6: Are the feasibility project’s costs at the planning phase.
studies do? 4. Payback period calculated for the potential
metrics can be calculated weekly as a part of project progress preliminary project’s costs at the initiation
report. The project manager should be involved in this process. phase.
The quality group should present the SPM performance metrics 5. Payback period calculated for the potential
to the project manager and deal with his objections by detailed project’s costs at the planning phase.
6. Percentage of detailed feasibility studies
clarifying, negotiating, or modifying these metrics. Previous achieved vs. required at the planning phase.
experience from similar projects can be useful in this process.
In addition, this process can be achieved with the assistance of Table (1): The elaborated list of questions and performance
external consultants to define and validate the SPM metrics for scope management.
performance metrics. Figure (3) illustrates the steps of this
procedure. B. Calculate the Performance Metrics and compare it with the
accepted range
1. Select a goal from the list of goals of SPM domains (nine goals).
2. Define a list of questions related to the selected goal. The second procedure of the proposed approach is
3. Select a question from the list of questions.
calculating the performance metrics for the questions of a
4. Define a list of performance metrics related to the selected question.
specific goal. Calculating performance metrics is a simple
5. Select a performance metric to be defined in detail.
process because it depends on simple or known statistical or
mathematical formulas. So, the researchers don't focus on
6. Define the mathematical or statistical technique for calculating the selected metric.
calculating performance metrics. After calculating the value of
7. Define time check points for calculating the selected metric.
the performance metric, the quality groups compare this value
8. Define the accepted range of the metric value of the selected metric.
with the accepted range of the metric value. If the metric value
9. Negotiate the project manager and deal with his objections by clarifying or
is out of the accepted range, the quality groups add a deviation
modifying the performance metric. note to the performance report. The quality groups prepare a
10. Check the list of performance metrics. If it is not empty, then go to step 5. performance report that must include the metric value, accepted
11. Check the list of questions. If it is not empty, go to step 3. range, and any deviation notes. Some performance metrics may
12. Check the list of goals. If it is not empty, go to step 1. be Not Applicable (NA) in some specific cases [19]. So, during
computing the value of the performance metrics, the NA
Figure (3): The definition of the Goals, Questions, and Performance Metrics.

306 http://sites.google.com/site/ijcsis/
ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,
Vol. 8, No. 9, December 2010
metrics are eliminated. Figure (4) illustrates the steps of this www.crim.ca/Publications/2010/documents/plein_texte/ASD_AliE_al_
procedure. QAOOSE_2010.pdf
[5] Fenton, Norman, Robin Whitty and Yoshinori lizuka, ―Software
C. Interpret and Analyze the Performance Report Quality Assurance and Measurement‖, International Thomson
Computer Press, 1995.
The third procedure of the proposed approach is [6] Girish H. Subramanian, James J. Jiang, and Gary Klein, ―Software
interpreting and analyzing the performance report. The quality quality and IS project performance improvements from software
group should report their interpretation to their top development process maturity and IS implementation strategies‖, the
Journal of Systems and Software 80, 2007, pages: 616–627
management. If the performance deviations is not accepted, top [7] IEEE Standard 610,‖Glossary of Software Engineering‖, 1990.
management may take corrective actions or inform the project [8] Ince, ―Software Quality Assurance - A Student Introduction‖,
manager to take corrective actions. The value of performance McGraw-hill international (UK) limited, 1995.
deviations should be analyzed to discover the weaknesses and [9] Information Systems Audit and Control Foundation (ISACF),‖Project
Management: Skill and Knowledge Requirements in an Information
strengths of project management practices. This analysis can be Technology Environment‖, 2002.
used to reduce or avoid many risks or obstacles that may be [10] ISO 10006,‖Quality Management Systems – Guidelines for Quality
encountered in later points of time or in next software projects. management in Projects‖, Second Edition, 2003.
[11] Jeffrey A. Hoffer, Joey F. George,and Joseph S. Valacich, ―Modern
V. CONCLUSION System Analysis and Design‖, Addison Wesley Longman, Inc, 1999.
[12] Jeffrey L. Whitten, Lonnie D. Bentley and Kevin C.Dittman, ―System
The main objective of this paper was proposing an Analysis and Design Methods‖, Fifth edition, Mc Graw Hill Companies,
approach for evaluating the performance of software project Inc, 2001.
[13] Joel Henry,‖Software Project Management – A Real-World Guide to
manager using GQM method. In GQM method, each goal
Success‖, Pearson Education, Inc, 2004.
generates a set of quantifiable questions that attempt to define [14] Jolyon E. Hallows, ―Information Systems Project Management: How to
and quantify this goal. The question can only be answered Deliver Function and Value in Information Technology Projects‖,
relative to, and as completely as, the available metrics allow. AMACOM, a division of American Management Association, 1998.
[15] Karl E. Wiegers, ‖A Software Metrics Primer‖, Process Impact, 1999,
Evaluating the performance of software project manager is http://www.processimpact.com/articles/metrics_primer.html
helpful for increasing capability level and productivity, [16] Paul Dorsey, ‖Top 10 Reasons Why Systems Projects Fail‖, 2000,
improving quality, tracking project progress, and assessing http://www.duclcian.com/papers/top%2010%20reasons%20why%20sy
project status. stems%20projects%20fail.htm
[17] Paul Goodman, ―Software Metrics: Best Practices for Successful IT
We conclude that the roles of quality group are very Management‖, Rothstein Associates, 2004.
important in software projects. They can use the list of [18] Robert T. Futrell, Donald F. Shafer, and Linda I. Safer, ―Quality
performance metrics and the proposed approach to evaluate and Software Project Management‖, Prentice Hall PTR, 2002.
track the performance of the software project manager. In [19] William E Perry, ―Quality Assurance for Information Systems:
Methods, Tools, and Techniques‖, QED technical publishing Group,
addition, we conclude that special emphasis must be given to
1991.
performance metrics in software projects to discover and avoid [20] Yael Dubinsky and Orit Hazzan, ―Using a role scheme to derive
the weaknesses of practices. On the other hand, the strengths of software project metrics‖, Journal of Systems Architecture 52 pages:
project management practices must be utilized and encouraged. 693–699, 2006.
[21] Zahedi, Fatemeh, ―Quality Information Systems‖, Boyd & Fraser
VI. FUTURE WORK Publishing Company - a Division of International Thomson Publishing
Inc., 1995.
For future work, the following points are expected to be [22] Zdena Dobešová1 and Dagmar Kusendová2, ―Goal-Question-Metric
focused: method for evaluation of cartographic functionality in GIS software‖,
proceedings GIS Ostrava, 2009,
 Improvements in integration and scope management of www.geoinformatics.upol.cz/app/visegrad/images/GISOstrava.pdf
software projects.
 Improvements in schedule and cost management of
software projects.
 Achieving higher levels in Capability Maturity Model
Integration (CMMI) for IT companies.
 Enhancements in the quality of e-government projects.
REFERENCES
[1] Chang E. Koh, Victor R. Prybutok, and Xiaoni Zhang, ―Measuring e-
government readiness‖, Journal of Information & Management 45
pages: 540–546, 2008.
[2] Department of Energy (DOE), ‖Basic Performance Measures for
Information Technology Projects‖, 2002,
http://cio.doe.gov/ITReform/sqse/download/PE-WI-V3-011502.doc
[3] Department of Energy (DOE), ‖IT Project Management Review
Process Guide‖, 2003,
http://cio.doe.gov/ITReform/sqse/download/QR-CP-F3-011403.doc
[4] El Hachemi Alikacem and Houari A. Sahraoui, ―Rule-Based System
for Flaw Specification and Detection in Object-Oriented Programs‖,
2010,

307 http://sites.google.com/site/ijcsis/
ISSN 1947-5500