Lean software Development

Introduction
Lean development process is one of the frequently used terminologies in IT industry in todays world. With economic recession and spend cuts in IT by companies, IT companies (especially the IT service providers) are increasingly assessing their processes to cut down inefficiencies to save cost, thereby passing on some of the cost savings to their customers. Companies are also experimenting ways to improve productivity of their employees to deliver more value to customers. In this article, we try to show how lean processes are adopted in IT companies to minimize waste. We will also see some of the recent development methodologies used to minimize waste in the development process and to improve productivity of the employees1.

What is Lean processing?

Whenever we hear about Lean development process, the instant term that strikes us is JIT (Just in Time). JIT is a technique introduced by Toyota to bring down inventory levels in its assembly line. What is so great about lean development? Is it something that only manufacturing industries can adopt? Can this process be adopted in software industries? This article tries to answer some of the above questions. According to Japanese management, capacity of a system can be defined as following: Capacity = work + waste In general, Work relate to those activities performed in an organization towards making a product that the customer demands and eventually results in cash realization to the organization. Waste relate to those activities performed in an organization towards making products that are not demanded by customers. Waste also refers to those activities that are performed on a defective component or product. Toyota Production Systems, the forerunner in implementing lean development model in their plants, focused on removing wastes in their unit by implementing two things

The article is written purely based on the knowledge and experience of the author working in IT industry and understanding of lean concepts

1) JIT Arriving at what to produce, how much to produce and when to produce depending on demand. Excessive production results in unnecessary WIP, Storage Space, etc. 2) Jidoka A self-evident system that warns and stops the line immediately when a product is not of sufficient quality. Removing waste from a system naturally increases work and ensures correct use of the capacity. The above capacity equation perfectly suits a manufacturing industry especially an assembly line. However the concept of reduction in waste holds good and can be applied to any industry. The capacity equation though, it makes sense, requires some customization to suit the model/processes used in IT industry. So, we modified the capacity equation slightly to fit the processes/procedures followed in Software industry. Total Person Hours Where, Employee working hours = Hours put by employees at work (in projects to build products). Training/Skill development hours = Software Industry is a knowledge industry marked by high technology obsolescence. Employees need to continuously hone their skills with recent updates. Training could be on domain, technology or processes. Employee non-available Hours = Time spent on other non-working activities such as for business development, consulting, organization specific activities (unrelated to client). Each organization mandates minimum hours per employee on training/skill development and other organization specific activities. (Organization specific may be relevant only software services companies). So, let us look at Employee working hours to see if there is scope to remove waste. Employee working hours may again be split into smaller components as follows: Employee Working Hours Where, Production Hours = Time spent on actual development of the software product (Design, Development, Testing, etc.). Rework Hours = Time spent on product due to change in requirements, misunderstanding of requirements and defective functionality developed. Software Configuration Management (SCM) Hours Software Development Process Hours = Employee Working Hours + Training/Skill Development Hours + Employee NonAvailable Hours

Production Hours

Rework Hours

SCM Hours = Time spent on tracking and controlling the changes in the software such as baseline, revision control, code merges, etc. Process Hours = All software projects have a development process. In addition, organizations are certified by ISO, CMMI, etc. These certifications mandate certain additional processes to be followed in software development life cycle.

Analysis of each component in Employee Working Hours

Let us take and analyze each element of employee working hours. We will try to find out if there are potential wastes that can be removed in each of the process.

Production Hours:
Production hours refer to the time spent on actual development of the software product. Software development comprises the following elements Analysis of a requirement, Design, Development (also called coding), unit testing, integration testing, functional testing, load testing and deployment. Each step may also have a review component to make sure the process is good. Overall, there are eight stages in development, in which half (four) are testing phases. If we assume that each step takes same amount of time, we could state that we spend 50% of the time in testing the product we build2. In addition, there is a review step present in each activity which is also an appraisal effort like testing. In general, there is very high effort put on review and testing processes to make sure the finished product has no defects. Let us see the function of each testing step. Unit testing Integration testing Basic (modular) testing performed on a module of code to ensure that it is performing its intended function. Second level of testing that is performed once individual modules are integrated into a component. Testing a developed system to ensure that it is performing its intended need. It is sometimes referred to as black box testing because the tester is not interested in how the request is handled (input is processed). An advanced functional testing, where the system is loaded with simultaneous requests or hits, to see if it can handle the load when the product is made online.

Functional testing

Performance testing

IT Product managers believe that having defects in a finished product is much costly than the effort and hence managers live with cost incurred on testing activities to avoid defects. There are test automation products available that are used by testers to automate and speed up the testing process. However if we look at it from lean perspective, there is a high amount of waste
2

In reality, the testing effort in a software product development is not 50% of overall efforts. It is around 30%.

effort involved in product development. System studies can be done to minimize the number of testing phases required without compromising on quality. By applying the principles of Jidoka (empowering employees to raise flag when a potential defect is identified), the development process could be halted, problem rectified and then development process continued. Please note that testing is not a value-added activity, it only adds credibility to the product/system we build. There should be ways to minimize the testing efforts in a software product development process.

Rework Hours:
As mentioned in the earlier section, rework effort can be due to change in product requirements, misunderstanding of requirements and defective product developed. Let us first understand in detail each category of rework and then we will look at how to avoid the same. Software product development many of the time is marked by high change in requirements/specifications. This is essentially applicable when a new product is launched in the market. For maintenance projects, however changes in requirements are low after a release starts3.

Rework due to change in requirements

A change in requirement results in partially completed work the efforts for which are waste. Requirements are mostly specified in English documents. English is a descriptive language, in the sense, there is no single word to describe Rework due to most of the details, a collection of words will have to be used and there misunderstanding are chances of miscommunication in this process. of requirements A misunderstanding of a requirement results in a defect in the product that is developed resulting in defects/rework at a later stage. Every a defect is raised it requires rework followed by another round of Rework due to testing to ensure the product is functioning as per specifications. defective product developed Also, later a defect is identified more the rework and associated retesting. Any rework effort is a waste effort. A lean system should reduce rework effort to zero. Basically, all the three teams analysts (who gather business requirements and generate specifications), developers (who build the system by writing code and do unit & integration testing) and testers (who perform functional and load testing) should work together and in harmony to avoid rework effort. Often, three different companies work on each of the above customers themselves write specifications, one vendor does development and another vendor does testing; hence there are more challenges in close interaction.

Software projects can be broadly classified into two categories development projects, where a new product is developed from scratch and maintenance projects, where enhancements, improvements are done to an existing product.

SCM Hours
Software Configuration Management Efforts are those involved in tracking and controlling the changes in the software such as baseline, revision control, code merges, etc. Typically third party products are available and used to perform the operation. Most of the efforts are automated.

Process Hours
Many of the customers, who have huge IT systems, outsource to multiple vendors to maintain their systems. To ensure proper communication between vendors and teams, customers stress on a lot of processes. Also, the IT service providers are certified by standards such CMMI, ISO, etc. and they also have a lot of in-house processes to ensure continuous improvement with time. With time, there are chances that some of the processes are unnecessary and redundant; so it is important to revisit the processes year on year or based on maturity of the system to remove unnecessary ones. As described in each section above, there are chances of waste activities at various points in a software development process. The development processes are even more complicated when multiple vendors are involved and projects are executed in parallel.

Can all waste be removed?

It may not be possible to remove all non-value added activities. Take for example, the quality inspection aka the testing process. Though testing does not generate revenue, it helps in delivering a product of high quality. So, testing cannot be ignored altogether that it is a waste. A high quality product builds credibility to the company/person. No company wants to deliver an uninspected product and lose credibility before its customer. So, it may not actually be possible to remove all testing, but we should always strive to minimize efforts.

Note: Customer pays for testing activities, so how can one state that testing does not generate revenue? Customers play a major role in services companies. The degree of interfacing may vary based on the company policy. A customer can be involved superficially the requirement is taken from the customer and the final product is delivered at the end. On need basis, the customer is contacted for any queries, if any. (Fixed Bid projects). A customer can be involved to a greater degree throughout the period. Frequent updates are provided; customer is fully aware of the internal processes and can track the status almost every day (Time and material projects).

More the customer involvement, lesser the productivity of the service as additional time is required to educate the customer of the processes. In case of FB projects, customer does not care (to that extent) how much testing efforts are spent on the product as long as the product is of high quality. But the services company puts that effort to ensure a high quality product. Alternatively, in a T&M project, the customer understands and funds the time & money put in testing. The customer also knows that testing costs does not add any value and hence there is always pressure to bring down the testing costs.

Minimizing waste:
There is no single mantra or process to minimize the wastes in software development because of the following reasons: Every team is heterogeneous what is applicable to one team may not be applicable to others. Heterogeneous software product platform (Microsoft, Mainframe, JAVA, etc.) and product usage (online using a user interface, batch process, etc.). There are different categories of projects o Development projects - a new software product is built from scratch based on a need. o Reengineering projects - a product is migrated from an existing platform to a new platform. o Maintenance projects - enhancements / new features are added to an already working established software product. o Upgrade projects - the platform (server, machine, language version) in which a product is running is upgraded to a newer version for better performance or the older version of the platform is no more supported. Each project has its own set of challenges and requires different processes. Processes, procedures for one may not be applicable for the other.

Each team has to do its lean analysis separately. To perform the analysis, it is necessary to know the relative strengths and weakness of the team and the product, so as to exploit the internal strengths while protecting the areas of weakness. This exercise will consume time and

is a non-value added activity. However the relative benefits compared to the cost are high, making this an overall beneficial exercise. Above all, this exercise should be a collective effort every member in the team should buy-in to this idea and contribute, a top-down push will never work. Similarly benefits of the exercise should be shared by the whole team to sustain the model. The role of management is very important and critical in lean exercise. Not all customers may be co-operative to the exercise. Managers need to be innovative in driving and sustaining this exercise. Some activities ought to be taken up at the organization level also such as the following: Data points on the steps taken to minimize waste can be collected and analyzed across projects to arrive at a broad rule or framework. Such rules can be published and used as inputs by project teams. These rules may not be applicable to all teams as the framework is formulated based on historical data. But these rules can be used as guidelines to assist the managers. The guidelines change with time. So, this will be an ongoing exercise and not a one-time activity. Lean processing technique provides a new perspective in looking at a process. It is tough, takes time, requires a radical shift in execution, but provides huge benefits.

Generic Lean models/frameworks

In the previous section, we said that there is no single solution in adopting lean in IT products and that each team has to do its analysis individually to minimize wastes. However there are certain frameworks available and currently picking in the market that help adopt lean approaches in organizations. We will see some of them below.

SCRUM Agile framework

The SCRUM model/framework is built on the assumption that there are always uncertainties when a business requirement is converted into specifications and that the uncertainties can be resolved in a specific period of time. Human creativity cannot be boxed and forced to unleash within time frames. So, new product developments are always bound to change in requirements as the product is developed slowly. The framework suggests that teams work in boxed-time intervals say four weeks (also called Sprint). In each week, the team picks up certain requirements from a backlog of product specifications and strives to complete the committed items. The team will be collocated or highly virtually connected to interact. At the end of each boxed period, a fully tested and deployable product is developed. So, as the product is enhanced in the next sprint, efforts spent in the developed version are not wasted. The framework is applicable when product requirement is uncertain and bound to have frequent change requests.

Test Driven Development

The test driven development (TDD) is a development model where test cases4 are written prior to development unlike conventional models where development happens before testing. The developer picks up the test cases and executes them initially all test cases fail. Then the developer sequentially builds the product and the number of failures decreases with more development effort. The development process is complete when all test cases succeed. TDD can be implemented in projects where the number of inputs to the product is very high. A conventional model requires test data preparation to be done by both developers and testers separately. With higher number of fields, time taken to prepare test data considerably increase. TDD helps remove testing efforts considerably in such cases.

Continuous Integration
Continuous Integration (CI) is a continuous process of applying quality control to the software product being developed. Third party tools or products are available for CI that can be integrated with Software Configuration Management tools. The tool can be configured to build the product based on the latest version of the code available in SCM, deploy the product, execute all test cases for the product and share (email) the test results in specified intervals of time. This model is applicable when a huge software system is being built and the project is executed by multiple teams from different vendors and/or different locations.

Conclusion
Lean as a technique to minimize waste in IT product development is only bound to increase. The era of customers realizing costs due to outsourcing and offshoring is over and software product development is becoming more commoditized today. New software development models to remove waste and increase productivity are on the rise and more products will be available in the market in the future. Many companies have started adopting new models for their project execution. The primary setback with these models is that they can be easily adopted for a new project. In a maintenance project of an existing (huge) system, the cost of developing a framework may be very high if automated tools are not already available. Under such circumstances, the conventional style of system study by the team to minimize waste may be more realistic and easy than switching to generic frameworks. It is important that customer and vendors are not carried away by the hype and attractiveness of the new models and enough rationale is applied in choosing the best model for project execution.

Test case is a set of conditions with which a tester will determine whether an application is functioning correctly.