Quality & Security Management

Information Sciences

Tim Bradley December 2010

Quality and Security Management

Contents
1 Introduction.........................................................................................3 2 The necessity of quality assurance...........................................................3 3 Quality within the Lifecycle.....................................................................4 3.1 The Triangle of Objectives.................................................................5 4 Quality Management Strategies...............................................................6 4.1 Plan (Define and Prioritise)................................................................6 4.2 Verify and Validate..........................................................................7 4.2.1 Process audits...........................................................................7 4.2.2 Peer reviews.............................................................................8 4.2.3 Work Product Analysis................................................................8 4.2.4 Testing.....................................................................................8 4.3 Improve (Analyse and Tune)...........................................................10 4.4 Manage........................................................................................10 5 Conclusion.........................................................................................11 6 Cause and Effect (Ishikawa) Diagram.....................................................12 7 Priority List........................................................................................13 8 Self Control........................................................................................14 8.1 Expectations:................................................................................14 8.2 Knowledge of Performance..............................................................14 8.3 Regulation of Performance..............................................................14 9 Quality Control...................................................................................16 10 Consumer Audit................................................................................18 10.1 Touch........................................................................................18 10.2 Sight.........................................................................................18 10.3 Hearing......................................................................................18 11 References.......................................................................................19 12 Bibliography.....................................................................................20

Tim Bradley December 2010

Page 2

Quality and Security Management

1 Introduction
In todays economic climate where business competition is fierce, quality has become an important factor in gaining competitive advantage. Software development organisations use metrics to establish the quality of their products in order to improve customer satisfaction by reducing errors and implementing better programming and testing techniques. Software quality assurance drives best practice, ensuring that everyone involved is doing the right things, in the right way at the right time. Lifecycle Quality Management (LQM) enables companies to align priorities and customer expectations with the system/project requirements. This process reduces costs, maximizes the potential for success and improves customer satisfaction and can therefore give an organisation the competitive edge.

2 The necessity of quality assurance

Quality is important to every business whether they provide goods, services or software. Poor quality invariably leads to dissatisfied customers, which reduces the likelihood of new or repeat business from the supplier. This would have direct impact on revenue. Software development companies are under constant pressure to ship products faster, at lower costs and with fewer resources. Within this complex and demanding environment it is often all too easy for organisations to overlook the importance of quality. Within corporate software organisations, only one in three software projects is considered a success, and approximately 70 percent of software projects fail to deliver what was originally intended without going over budget, missing the deadline or sacrificing quality (Standish Group International, 2004). Many development companies employ a testing department which usually verifies quality when coding is complete, or frozen. The testing is often rushed in order to certify the software as soon as possible so that customer and market commitments are met. As the stages of the System Development Life Cycle are put under continual pressure the testing prior to deployment becomes less effective and the possibility of error increases. It is fairly common that during the latter stages of the Implementation phase of the SDLC that new features are requested and modifications made. This adds to the testing workload, often without any additional resource or time factored into the project plan. When organisations neglect the importance of quality they run the risk of increasing their costs and decreasing their efficiencies. The diagram below illustrates how cost can spiral sharply upwards during the latter stages of the SDLC. By implementing better quality processes earlier in the lifecycle companies can reduce their business costs and become more efficient. Tim Bradley December 2010 Page 3

Quality and Security Management

Figure 1- The cost of discovering defects increases significantly during the latter stages of the SDLC.

Borland (2007). Image Available from: http://www.borland.com/resources/en/pdf/solutions/lqm_driving_quality.pdf Accessed 6th January 2010 According to Humphrey (2005) the estimated cost of correcting a defect found in software application after its release is ten times more than the cost of defects rectified during system tests. Organisation can benefit greatly by implementing quality activities earlier in the lifecycle.

3 Quality within the Lifecycle

When quality is fused throughout the software lifecycle, organisations deliver higher quality applications and service, while systematically reducing costs, risk and time to market. Borland (2007). By using appropriate quality processes in each phase of the lifecycle an organisation can reduce overall development cost by addressing quality issues at their root causes. Defining quality can often be complex and difficult because it means different things to different people. However in terms of software engineering quality can generally be defined as a software package that meets the requirements, has correct code, minimal defects and enables the organisation to fulfil their business goals. When organisations concentrate solely on post production testing they eliminate the opportunity for detecting problems earlier in the development lifecycle. By addressing quality issues earlier the organisation can ensure that an application (complete with its source code) is complete, scalable, adaptable and maintainable. The following list of quality issues cannot be detected by testing

Tim Bradley December 2010

Page 4

Quality and Security Management

alone, however they can all be addressed through Lifecycle Quality Management: Poorly defined or ambiguous system requirements can result in costly reworking. Violations in code licensing can increase legal risks. Poor overall design of an application can limit the performance and scalability of the software. Tightly coupled interfaces can make it difficult to integrate software with existing systems. The use of platform specific code can affect the portability of a product. Poorly commented or hard to read code can increase maintenance costs. Security vulnerabilities which can expose users to attacks.

Reactive software testing (at the end of the lifecycle) provides no quality guarantees as the application moves through the development lifecycle. This can result in software that gradually becomes misaligned and does not end up meeting the customers needs. Conversely, proactive quality management enables management to monitor the success, risk and progress of a project. This approach focuses on tracking key indicators throughout the lifecycle to ensure the project is delivered on time, on budget and to the specifications. Testing can be a costly and time consuming process for an organisation and therefore can often be neglected. By investing a more proactive approach organisations can remove defects earlier and therefore spend less time on rework and testing. To ensure quality software there needs to be skilled people using effective processes. When software bugs are found within an application they are often recorded using a defect management system. This tracking software is useful for monitoring a defect until it is fixed, however a greater gain can be achieved by exploring the cause of the defect. Analysis of software defects can allow organisations to train their staff better and improve their processes.

3.1 The Triangle of Objectives

In the mid 1980s Dr Martin Barnes devised a theory to demonstrate that the three primary objectives of cost, quality and time are interrelated. According to Barnes it is often necessary for development companies and customers to prioritise two of the primary objectives at the expense of the third. For example, if a project had to be completed quickly and the owner had sufficient funds, the project manager may recommend a generous budget and plenty of resources in order to deliver the project in the shortest possible time; this would however be at the detriment of quality. To deliver quality software solution the customer must be willing to either invest sufficient funds or be prepared to agree on a longer development schedule.

Tim Bradley December 2010

Page 5

Quality and Security Management

Figure 2 - The triangle of objectives.

4 Quality Management Strategies

There are a variety of different practices an organisation could adopt to instil quality into each stage of the Systems Development Life Cycle. The diagram below illustrates best practices in Lifecycle Quality Management.

Figure 3 - The four key process areas within Lifecycle Quality.

Borland (2007). Image Available from: http://www.borland.com/resources/en/pdf/solutions/lqm_driving_quality.pdf Accessed 6th January 2010

4.1 Plan (Define and Prioritise)

High quality system requirements provide the foundation for successful software projects as they align all stakeholders to clearly defined, common goals. These high quality requirements should be prioritised based on the needs of the customers business. Initial requirements should also contain details of testing and quality measures, both of which should be explicitly documented. Beynon (2010) argues that clear conceptual design is central to developing systems that are understandable and that meet customers expectations and preferences.

Tim Bradley December 2010

Page 6

Quality and Security Management

Analysis and prioritisation should involve IT and business stakeholders in order to ensure that there is a balance between customer needs and business costs. Risk management is the ability to anticipate what might go wrong in a project. Hoffer et al. (2004). All software projects carry certain risks. Through careful analysis plans can be implemented to minimise the potential impact of such risks. Broadly speaking risks can fall into the two following categories. Product related risks Integration with legacy systems, dealing with leading edge technologies, out manoeuvring the competition. Project related risks Managing resource levels, meeting schedules, financial considerations.

Quality measures can be costly and time consuming to organisations; therefore it is imperative that quality initiatives provide a reasonable return on their investment. The project teams should prioritise quality activities according to the needs of the business and the level of risk they are seeking to mitigate. For example, certain aspects of the software may be considered mission critical and thus should be tested carefully with the highest level of priority. Effective communication systems should exist between all stakeholders so that all quality activities are carried out with the correct level of priority. Most software development organisations have detailed policies and procedures that define their Software Development Life Cycle (SDLC). To deliver quality throughout the SDLC a project plan should contain the following: A clear definition of the roles and responsibilities. How resources will be allocated. Timelines, milestones and deliverables. The reports that will be required.

The criteria upon which application quality goals are based should always be aligned to meet the needs of the business.

4.2 Verify and Validate

Once a quality plan has been drawn up and documented the project team can begin to perform the quality activities. Through the systems design phase of the SDLC developers can measure application quality status and quality progress using a variety of methods.

4.2.1 Process audits.

Clearly defined processes enable teams to work efficiently and effectively. They ensure that each member of the team understand their role and responsibility throughout the project. Sometimes it is necessary for an organisation to tailor their processes in order to best fit the requirements of a project. Process audits are carried out to ensure that as the project moves through the SDLC the development teams are following the correct processes and procedures. Feedback from such audits can be used to make adjustments to the Tim Bradley December 2010 Page 7

Quality and Security Management

processes as the project progresses. By reacting to audit feedback proactively the project team can use the data to its advantage in delivery a quality product.

4.2.2 Peer reviews

To ensure software quality peer reviews can be a particularly useful tool. Peers within the project are able to offer feedback on all types of work products (requirements, design code, tests) to ensure that they meet with both customer and project requirements. Once again this proactive approach can help identify issues early and therefore allow rectification before the project moves into the subsequent phase.

4.2.3 Work Product Analysis

Prior to coding aspects of architecture and system design require thorough analysis. Proactive quality approaches could include architecture modelling to provide alternative design approaches. For example, some technologies or platforms may initially appear suitable until detailed modelling indicates that they might not scale to meet the customer needs. This early type of analysis which may include the use of static models or simulations can provide the assurance that the system design can be coded, safe in the knowledge that it will meet performance requirements.

4.2.4 Testing
Tests are designed to measure the difference between observed and expected software behaviour. Stair and Reynolds (2008) assert that good testing procedures are essential to make sure that new of modified software operates as intended. Testing is an effective process for identifying quality issues once the code has been written. Tests can be used to indentify: Functional defects Missed service level objectives Performance bottlenecks Usability issues Security vulnerabilities

Test can be performed manually or through automation. Automated testing is useful for repetitive tasks or where manual testing is inappropriate due to repeatability and scalability (e.g. load or performance testing). The cost of removing software defects grows exponentially as the application move through SDLC phases. Therefore it is important that organisations start testing early in the development phase. There are a variety of techniques that can be used for testing systems. Jones (1996) identifies 18 types of testing, the most common of which are subroutine, unit, new function, regression, integration and system testing. Testing strategies are commonly divided into two categories: black-box and white-box testing.

Tim Bradley December 2010

Page 8

Quality and Security Management

Black-Box Testing Black box testing (as the name suggests) requires no knowledge of the internal logic or code structure within the given system. It is sometimes referred to as opaque testing, functional/behavioural testing or closed box testing. In order to carry out effective black-box testing it is imperative the tester has a thorough understanding of the requirement specifications and as a user should know exactly how the system should behave in response to any particular action. Testing can be divided into two sub categories; a) tests requiring a user and b) tests where a user or role is not required. The following methodologies can be applied where a user is not required. Functional Tests are conducted on the system software, they are written in order to check the application behaves correctly. Stress testing applications is where the tester loads the system with complex numerical values, large quantities of input and complex queries. This is done to ensure the application can withstand high workloads. Load Testing is conducted by applying high levels of input or making demanding requests until the systems performance degrades or eventually crashes. Usability Testing is done to the user friendliness of the system. The user interface should be suited the needs and ability of the end user. Recovery Testing looks at how an application recovers after a hardware or software failure. Applications should be designed to recover quickly with data loss minimized. Volume Testing subjects the system to extreme quantities of data to observe how it reacts.

The following methodologies can be applied where a user is required. User Acceptance Testing, the software is tested by the user to determine whether it meets their expectations and requirements. Alpha Testing is usually carried out at the development centre. Users are invited to perform tasks and any abnormal behaviour is documented and rectified. Beta Testing, at his stage the software is distributed users who test the application at their site. Any abnormalities or defects are documented and reported to the developers.

White Box Testing White-box testing deals with the internal logic and structure of code. White-box testing is also referred to as clear box, glass box or structural testing. To be able to carry out white-box testing the tester must possess knowledge of the coding and logic on which the system runs. Therefore white-box testing is a specialist skill and can only be carried out by trained personnel. The purpose of white box testing is to correct errors, to reduce unnecessary code and optimize the efficiency of the code. The disadvantages of white-box testing are that it is costly

Tim Bradley December 2010

Page 9

Quality and Security Management

due to the high skill level required and that in most complex applications it is almost impossible to check every bit of code. White-box testing may utilize some of the following methodologies: Unit Testing is conducted upon completion of a unit of code or when a particular functionality is built. This testing is carried out at a very basic level, usually by the developer. Static & Dynamic Analysis. Static analysis requires the tester to examine each line of code individually to identify defects, whereas dynamic analysis involves executing the code and examining the output. Statement Coverage tests that every statement within the code is executed at least once. This enables the tester to identify any side effects relating to individual statements. Branch Coverage tests any branches within the code ensuring that all branches link seamlessly and execute correctly. Security Testing requires the tester to ensure the system is protected from unauthorized access, hacks, cracks or other malware. Mutation Testing takes place after bugs have been identified and fixed. It ensures the new code executes correctly and that the modifications have had no undesired side effects.

4.3 Improve (Analyse and Tune)

After a software application has been released the project team can look to improve further developments by analysing the results of their verification and validation activities. Once a root cause has been identified the skill, process or technology can be addressed and improved. This swift action may save a great deal of time and resources on future development projects; it also helps to reduce any finger pointing between team members.

4.4 Manage
Throughout the SDLC it is critical that managers have the right information to make decisions. They should be able to access real time reports on quality status and project progress. These reports should cover the results of reviews, testing results, coverage and find-fix rates. Equipped with this information managers should be able to direct resources and understand release readiness. To minimise project costs all resources need to be managed efficiently. The use of control systems within an organisation will help manage the activities and assets that contribute to quality results. Quality Assurance procedures have been formalised in the British Standard BS5750 Part 1 and the international equivalent ISO 9001. Bocij et al. (2006). These procedures do not guarantee a quality software application but they ensure that the relevant phases of the SDLC, such as requirements capture, design and testing are carried out consistently.

Tim Bradley December 2010

Page 10

Quality and Security Management

5 Conclusion
Software quality should be delivered in every phase of the Systems Development Life Cycle; it should not be an afterthought. Lifecycle Quality Management enables businesses to effectively align their priorities and quality expectations within the project system requirements. By eliminating unnecessary risks inherent in software development organisations can minimise costs and maximise the commercial success of their applications. This approach to quality can enhance the confidence of the development team by combining people process and technology aspects to ensure that quality is built into an application from the initial requirements through to delivery. Software quality assurance is not a black art but a basic good management practice that complements sound technical skill and experience. McManus (2010).

Tim Bradley December 2010

Page 11

Quality and Security Management

6 Cause and Effect (Ishikawa) Diagram

Tim Bradley December 2010

Page 12

Quality and Security Management

7 Priority List
The cause and effect diagram above illustrates the relationship between a given outcome (post graduates failing to find employment) and the factors that influence this outcome. The most likely causes can be indentified and then a priority list can be devised so that the causes can be tackled in a rational order. The following priority list identifies the most important cause of post graduate unemployment in descending order of importance. Motivation An individual must have the motivation to want to work and also the drive to search and apply for employment. Preparation Students should know their prospective employer and prepare themselves for interview. This might included research into the job/company, creating a portfolio, practice interviews, prepare questions and dressing appropriately. Experience Students should try and gain as much possible experience either through art time work, placements or voluntary positions. CV- A well written and presented CV will make an applicant stand out. Willingness to learn new skills Every employer has their own methodologies, processes and procedures. Post graduates should be adaptable, receptive to change and willing to learn new skills. Communication and social skills- Most organisations rely on teamwork to produce a quality end product or service. It is important that applicants are able to fit in, communicate and work effectively with others. Qualifications Applicants should have high quality academic qualifications, but these could also be supported with professional/vendor certifications. Portfolio A interviewee should be able to demonstrate their past work and showcase their talent and ability. Flexibility Organisations need employees how can be flexible in terms of hours worked, geographical location and the types of role they perform. Expectations Most Post graduates will NOT earn 30,000 a year right out of University. They won't be a vice-president with a mansion until they earn both.

Tim Bradley December 2010

Page 13

Quality and Security Management

8 Self Control
In my position as an IT Technician at a secondary school I am required to ensure that the maximum number of workstations are available to students throughout school opening hours. Kineton High School has an attendance of approximately 900 pupils. The school has over 200 workstations in 7 different classrooms. It is important that I can manage the quality of results I attain and that Im responsible for the outcomes I deliver. Through self control I am able to manage my workload and a quality service to my customers (Teachers and students).

8.1 Expectations:
Through a clearly defined Service Level Agreement I aware that at least 95% of the schools workstations should be available during school open/working hours. Non functional workstations are reported via a helpdesk system, which tracks the number of PCs that are unavailable, this can be up to 10 PCs before the ICT department falls behind the targets stated in the SLA. This target is monitored on a daily bases and is a reasonable reflection of the effectiveness on my departments role within the school. The SLA clearly communicates to me what I am required to do and how I can measure my own success. Through coaching and feedback from my Line Manager and colleagues I am also kept informed of what is expected from me. My responsibilities are also documented in my job description. This comprehensive method of communication ensures that I have a clearly defined and unambiguous job role. Expectations are made clear; I know what I should be doing and how to do it.

8.2 Knowledge of Performance

I am able to measure my success and that of my department via a computerised management dashboard. The system tracks the number of malfunctioning workstations, when they were reported, by whom, who is responsible for fixing them and how long they were out of service. Reports can be run that show availability statistics on a daily, weekly and monthly basis. The reports also show patterns in hardware/software failure, causes of faults, remedies, locations, success rates and other trends that are useful in improving the ICT services. Feedback about my performance can also come from Line Management or peercoaching, audits, process reviews and feedback from teaching staff. The feedback provided enables me to review and improve my working practices. I can continually work towards providing higher availability, shorter Out of Service times, better performance, improved service and generally better quality in the services and products I deliver.

8.3 Regulation of Performance

If I am failing to meet the organisations targets it is important that I am aware of the situation. Individuals cannot be held responsible for delivering poor quality Tim Bradley December 2010 Page 14

Quality and Security Management

if they are unaware that they are not meeting the expectations required of them. In my position I will receive feedback from various sources if I fail to deliver quality in my role. The ICT management systems will inform me that I have dropped below the standards specified in the SLA. Management would also be aware and this would be communicated to me immediately. At this point my manager and I would create a plan describing how I improve my performance in order to return to the expected standard. I would be offered support, training and recourses if required. Only a monthly basis I complete a formal review with my manager who measures the success of each task within my role. The review document is used to create a development plan, which focuses on future training and personal improvement. I am also set longer term objectives (6-12 months) which challenge me to improve the over quality of service offered by the ICT team. I also seek 360 feedback from my peers and staff in other departments of the school such as teachers, pupils, support and administrative staff.

Tim Bradley December 2010

Page 15

Quality and Security Management

9 Quality Control
The table below illustrates how as a consumer I can measure the quality of goods, services and software. Each table describes the quality control subject and then provides a suggested method of measurement which the supplier could utilize to improve their performance. Goods Quality Control Subject 1: Fitness for purpose Number of product returns or complaints for every 1000 products sold. Consumer Reviews. Services Success of overall service provided Number of followup visits or callouts required to complete work to customers requirements Software Fitness for purpose Does the product meet the specified system requirements and has in helped to improve the business in the way which was intended Software Customer Support Total number of support calls raised and the % resolved.

Unit of Measure

Goods Quality Control Subject 2: Unit of Measure Product Life Span The time between repeat purchases

Services Customer Care Customer satisfaction questionnaire results. Services Availability of Service At what % of the time is resource available to provide the specified service to the customer? How long do customers have to wait to receive the service they seek? Services

Goods Quality Control Subject 3: Availability of goods Is there a waiting list for the product being produced? At what % of the time is the product available for purchase by the end customer?

Software Delivery on time

Unit of Measure

Has the fully operational, tested software been delivered to the customer within the guidelines defined in the project plan?

Goods Tim Bradley December 2010

Software Page 16

Quality and Security Management

Quality Control Subject 4:

Usability Consumer trial results. Focus group feedback. Reviews by critics.

Speed of service Completion time. How long did it take to provide a satisfactory service from the initial request?

Usability Feedback from users, questionnaires. Reduced system training hours.

Unit of Measure

Tim Bradley December 2010

Page 17

Quality and Security Management

10 Consumer Audit
During a recent visit to an electronic goods retail outlet I observed how customers used their senses to observe the quality of items they were interested in. Because the store sold only electronic goods I was restricted to observing their use of touch, sight and hearing. In another environment i.e. a green grocers I may have observed them using their sense of taste and smell, this would of course be deemed somewhat unusual in and electronics outlet.

10.1 Touch
Most technological products require the user to psychically interact with them. Most items have switches, dials, knobs, sliders, keys, touch input surfaces or other means of interacting. I observed that most customers used their sense of touch to measure two different things. Firstly they measured the products responsiveness and build quality, and then secondly they judged the products in terms of how tactile they were. Customers seemed to prefer controls that felt positive and well made. Flimsy engineering and poor build quality left customers dissatisfied. Positive, responsive controls gave users a sense of empowerment and satisfaction.

10.2 Sight
The overall aesthetics of a product can have a significant impact on products success. Many consumer electronic items now come in a variety of colours and designs in order to meet the ever changing tastes of potential customers. I observed several customers who considered the look of the product to be one of the most important considerations prior to purchase. However, other than pure aesthetics some consumers used their sense of sight to make decisions about the build quality of the item. Some customers viewed the items from all angles to ensure that it met their requirements.

10.3 Hearing
I observed various customers listening to sounds made by the moving parts of an item i.e. buttons being pressed, dials being turned, thing being opened and closed. Hollow, tinny sounds seemed to give the impression of poor build quality whereas nice deep clunky sounds seemed to reassure the customer that they were handling a quality product. This premise has been exploited in advertising campaigns run by Volkswagen, who argue that customers perceive positive, solid sounds to be indicative of high quality.

Tim Bradley December 2010

Page 18

Quality and Security Management

11 References
Beynon-Davies, P. (2010) Designing Interactive Systems, Second Edition. Harlow: Pearson Education Limited. Bocij, P., Chaffey, D,. Greasley, A., Hickie, S. (2006) Business Information Systems (Third Edition) Prentice Hall, London. Borland (2007) Driving Quality Throughout the Software Delivery Lifecycle. Austin, USA. Hoffer, J., George, J., Valacich, J. (2004) Modern Systems Analysis and Design (Fourth Edition). Prentice Hall, London. Humphrey, W. (2005) A Discipline for Software Engineering. Addison Wesley Publishing, Massachusetts. Jones, C. (1996) Software Quality: Analysis and Guidelines for Success. Thomson Computer Press, London. McManus J (2010) IT Now - Quality Assured Oxford University Press, UK Stair, M and Reynolds, G. (2008) Fundamentals of Information Systems, Fourth Edition. Thomson Course Technology, Boston Standish Group International. (2004) The CHAOS Report Massachusetts, USA.

Tim Bradley December 2010

Page 19

Quality and Security Management

12 Bibliography
Anderson, H. Yull, S. and Hellingsworth B. (2004) Higher National Computing (Second Edition) Oxford, Elsevier. Long, L. and Long, N. (2005) Computers: Information Technology in Perspective (Twelfth Edition) New Jersey, Pearson Education. Rudshill, M., Lewis, C., Polson, P., McKay, T. (1996) Human Computer Interface Design Morgan Kraufmann, USA Schneider, G (2003) Electronic Commerce Thompson Course Technology, Boston

Tim Bradley December 2010

Page 20