The patterns of success in product development: a case study

Petri Suomala and Ilkka J okioinen

Introduction
Industrial research and development (R&D) utilizes science and technology to construct new or improved products or processes for pro t-seeking companies (IRI, 2000). Product development, which is an essential part of R&D, can be seen as an activity that is expected to improve a companys competitive advantage and future success in terms, for example, of pro tability and market share. Based on the hope and trust that tangible returns will be greater than expenditure, considerable sums of money are spent on R&D (Batty, 1988). According to IRI, in US companies alone, representing over one-third of the entire worlds allocation in R&D, US$185.9 billion was invested in industrial research and development in 1999 (IRI, 2000). For comparison, US R&D expenditure in 1950 was US$2.5 billion (Jackson and Spurlock, 1966). With the increase in investments, the question of R&D management and measurement has gained in interest. Performance measurement has been considered a relevant aid in the pursuit of success in product development. Therefore, a wide body of literature concerning the measurement of R&D performance and success has been produced. Some writers (McGrath and Romeri, 1994; Chiesa and Masella, 1996; Kerssens-van Drongelen and Bilderbeek, 1999; Szakonyi, 1994a, b) have established holistic approaches for the assessment of R&D effectiveness, while others have concentrated on project level (Ormala, 1986; Rouhiainen, 1997). There are advocates of continuous or in-process monitoring of development as well as end-ofprocess evaluation (Schumann et al., 1995). There is a variety of methods available for R&D project selection (Cooper, 1985; Hollander, 2000), performance evaluation for managerial purposes, customer perspective (Hirons et al., 1998; Nixon, 1998) and benchmarking. Many creditable reports that describe the state-of-the-art of R&D measurement have been published (Brown and Svenson, 1998; EIRMA, 1985, 1995; Werner and Souder, 1997a, b; Grif n, 1997). While several decades of R&D studies have produced a good deal of data with respect to variables associated with the success and failure of new products, the research has not been able to resolve a practical problem: how should R&D actually be managed in order to

The authors Petri Suomala is Senior Researcher, Institute of Industrial Management, Tampere University of Technology, Tampere, Finland. Ilkka Jokioinen is based at Metso Paper Inc. Turku, Turku, Finland. Keywords Research, Product development, Performance measurement (quality), Case studies, Metalworking industry Abstract Success in product development can be considered a general aim for any R&D activity. Unfortunately, success is very multidimensional. Which dimensions of success one should include and how one can measure these dimensions is an essential question that must be resolved within R&D management. The aim of the paper is to analyse the multidimensionality of success at R&D project level. Using the evidence from three industrial cases, the paper shows the versatility of the concept of success in product development. On the basis of the evidence, the paper concludes that there is often only a vague correlation between three aspects of R&D success nancial, technical and project management. The focus of the paper is on the development of investment goods for the paper industry. Electronic access The Emerald Research Register for this journal is available at http://www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at http://www.emeraldinsight.com/1460-1060.htm

European Journal of Innovation Management Volume 6 Number 4 2003 pp. 213-227 q MCB UP Limited ISSN 1460-1060 DOI 10.1108/14601060310500931

213

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

promote high new product success rates. Poolton and Barclay conclude that managers are still relying on gut-feelings regarding best practice in new product development (NPD). Analogously, research has tended to be theory-driven instead of being applicationsbased (Poolton and Barclay, 1998). Driva et al. conclude that in most cases companies do not measure R&D activity very well, but that they are striving to determine how to do so effectively (Driva et al., 2000). In this respect, it seems fair to claim that a good deal of work is still needed to improve the ef ciency of the interface between industrial R&D management and academic R&D research. Success in product development can be considered a general aim for any R&D activity. Unfortunately, success is very multidimensional. The question of which dimensions of success one should include and how one can measure these dimensions is an essential question that must be resolved within R&D management (Hultink and Robben, 1995). However, relatively little discussion has focused on the resolution of this question. Yet, as Hart (1993) puts it:
Clearly, the way in which NPD success is de ned in uences the ndings which describe the factors contributing to NPD success.

Figure 1 The process of R&D

product development. In Matthews model, R&D is a process that consists of three main phases. It starts with technology development and ends with actual product development that results in a marketable product. Two essential dimensions are established: (1) technological uncertainty; and (2) the amount of allocated resources. If the R&D process proceeds, it is assumed that the technological uncertainty will decrease i.e. the prospects of new product success should be improved. Typically, as the product gets closer to the market more resources are allocated to the development project. According to Matthews, completing the whole R&D process requires answering ve questions: (1) Is it possible: (2) Is it attractive? (3) Is it practical? (4) Is it desirable? (5) How do we do it? The challenges of R&D are likely to differ from phase to phase. Projects belonging to category C have to be managed differently than projects belonging, for instance, to category A. This is because research projects (category C) typically have somewhat different goals, objectives and restrictions than speci c product (category A) or concept development projects (category B). Research projects may often deal with technological issues that cannot yet be attached to certain industrial and marketable products they are mainly serving as a breeding ground for promising tadpoles. Only time will tell whether these technologies will lead to commercial products that is, technological uncertainty is rather high. According to

Grif n and Page recognize that success is elusive, multifaceted and dif cult to measure. Still, companies and academics use over 75 measures of success in product development (Grif n and Page, 1996). Basically, hand in hand with determining and selecting R&D performance measures for a company, one should also consider the concept of success. What is the form of success that is primarily pursued? Are there any other success dimensions that would be important for us? Knowing the type of success pursued would likely be helpful in choosing the appropriate set of R&D metrics.

R&D processes contributing to new product success

From the perspective of R&D management, in practice, seeking new product success might be challenging, seeing that prospects of success likely vary from phase to phase in the R&D process. In this study, the model presented in Figure 1 (Matthews, 1991) has been adopted as a basis that describes the fundamental processes of research and

214

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

Matthews (1991), the costs of these types of projects are typically seen as overheads, and the projects are allowed to yield results over a rather long time scale, if ever. On the other hand, the technological uncertainty of product development projects (category A) should be signi cantly lower, but at the same time the objectives of the projects should have more concrete goals, including the nancial ones, than research projects. As Matthews (1991) puts it: short-term development (in a typical company) is seen as investment. The most problematic area, however, is the gap between overhead and investment. These category B projects may often fail to show suf cient justi cation for funding, since they neither represent a pure breeding ground any more, nor have they yet reached the status of investments that could be assessed using sound nancial measures. These projects can be seen as strategic options and the challenge is to identify the most promising and practical ones that have the potential to proceed to actual product development. Matthews work provides one valuable insight into the classi cation of R&D projects or activities, but traditional classi cations need to be discussed as well. Batty, for example, has provided a traditional de nition of R&D by recognizing three elements: (1) basic research; (2) applied research; and (3) product development (Batty, 1988). McLeod (1988) also adds a fourth element design, which starts after development. A slightly different categorization is proposed by Jackson and Spurlock (1966). They have identi ed fundamental research, applied research and developmental research (Jackson and Spurlock, 1966). R&D projects, as seen in this case study, mostly belong to the development, design or applied research category. Fundamental or basic research is not so typically conducted at company level. However, many of the applied research or development projects are originally triggered by co-operation with a company and a university or research institute in the basic research area. De ning a detailed research and product development process is a complicated task. Sometimes it is dif cult to distinguish which activities should be conducted in a certain R&D process step. However, Ulrich and Eppinger (1995) have established a generic development process that consists of ve

phases. The process starts with concept development, as the writers exclude research activities from the actual development process. The four sequential phases are: (1) system level design; (2) detail design; (3) testing and re nement; and (4) production ramp up (Ulrich and Eppinger, 1995, p. 15). According to Ulrich and Eppinger (1995), in the concept development phase many functions of a rm are allotted several tasks and responsibilities. The following concept development related tasks are recognized by function (Ulrich and Eppinger, 1995): (1) Marketing: . de ne market segments; and . identify lead users and competitive products. (2) Design: . investigate feasibility of product concepts; and . develop industrial design concepts. (3) Manufacturing: . estimate manufacturing costs; and . assess production feasibility. (4) Other: . nance, facilitate economic analysis; and . legal, investigate patent issues. Concept development, as seen from the previous list, should put emphasis on economic issues in analyzing and resolving the potential of a product or solution for the companys business. In this analysis, customers and production are to be taken into account. Issues not explicitly mentioned above, but still worth investigating in the concept development phase, are corporate strategy and business as a whole. Questions such as Does the product t well into our strategy? and How exactly will the product strengthen our overall business? should be the concern in concept development. If a development project passes the assessments conducted in the concept development phase, it will proceed to product development. The main input to product development is the outcome of concept development, the main objective of activities being a commercial and marketable (pro table) product. Thus, product development resolves the detailed nal design and prepares the whole company for the

215

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

product launch. In real life, organizing an R&D process may not be as straightforward as described above. Phases are likely to be mixed and partly overlap each other. Importantly, new product success is not something that is considered only in the product development phase. Success should be seen rather as a goal that is striven for in all the phases that constitute the R&D process in the company. Each phase may make a contribution to the overall success.

Selecting development projects that will lead to success

Often the number of potential research, development and design projects is greater than it is possible to carry out. The limited resources and skills compel the managers to select projects from those proposed. A comprehensive description of different selection methods is presented by Martino (1995). Ellis (1997) argues that without objectively measuring the process of innovation one is not able to determine whether the expenditures on R&D are bene cial or not. Both the desired outcomes as well as the inputs and R&D processes that contribute to these outcomes should be measured (Ellis, 1997, p. 3). Cooper (1985) has developed the NewProd model for separating probable successful projects from probable losers. He remarks that project selection is pivotal to effective risk reduction in product development. A scoring model could be a valuable tool in screening proposals. According to the NewProd model, product superiority/ quality, market need, growth and size and product scope are the factors that have the strongest impact on probability of success (Cooper, 1985). Hollander (2000) has reported the potential of the Genesis model for project assessment. His study is based on Coopers NewProd studies. The objective of both of these models is to provide support for the product development team, especially for go or no go decisions. The Genesis model is focused on development projects and teams; the question is does the team have the necessary resources and skills and how the product is positioned in respect of markets and competitors products (Hollander, 2000). McLeod (1988) suggests several factors that should be taken into account when

selecting R&D projects, including: probability of success; time to rst sales; pro tability; and compatibility with the companys long-term plans (strategy). However, it is argued that there is little point in trying to give the factors any order of priority. According to McLeod (1988, p. 254), all these factors should be considered as a whole. The construction of numerical indices, struggling with gures and scoring the projects is not seen as bene cial. Cooper and Kleinschmidt (1995) point out that new product performance is a multidimensional concept. Therefore, a single measure for NPD performance monitoring may not be enough. However, many companies still use only one or two measures. Having said that multiple measures are better than a single one, Cooper and Kleinschmidt (1995) continues that many of the measures may not be totally independent of each other. A holistic view of new product performance can be reached with a limited number of measures. The body of literature on new product success suggests that there is a rather versatile set of factors affecting this success. To structure the eld, different domains of groups of factors related to NPD success are illustrated in Figure 2. The brackets refer to literature that supports a particular perspective. In this synthesis, the success factors are collected into four perspectives that demonstrate not only the elusive nature of success factors but also the different viewpoints of various stakeholders in the NPD process. The top management of the company, for instance, may assess the NPD activities and outputs through a strategic framework: is there a suf cient t between the NPD and the corporate or technology strategy, or does the NPD produce essential strategic value for the company. At the level of business units the concerns of management
Figure 2 Perspectives having a contribution to new product success

216

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

may include the consistency between the present business and potential produced by NPD: what are the anticipated business impacts caused by or the risks posed by the NPD. Furthermore, inside the technology and product development unit there might be pure technological interests in addition to the economic ones. Throughout the company there are or at least should be an interest in maintaining and developing the capability of NPD to produce value for the customer through innovative and competitive new products and services. With regard to the practical organizational context, strategy is one of the core issues related to the success of NPD. The management of NPD has to take into account the corporate strategy and its implications. Projects that are consistent with the strategy are likely to receive the most support from top management. On the other hand, strategy should be exible enough to enable the initiation of projects that seem to be very promising but that may not yet be at the core of corporate strategy. In addition to corporate level strategy, an R&D strategy may provide arguments for internal marketing of a project, thus facilitating, e.g. a suf cient resource allocation for the project. Second, a products relative position with respect to the target market seems to be an important determinant of success. The new products competitive position the products competitiveness against competitors products should be therefore carefully considered. Timing of the product launch is one of the things that affect the products competitive position. The third perspective affecting the products success is technology. Is the new technology mature enough to be manufactured and delivered? Does the customer have the ability to adopt the new technology, and what is the additional value of innovation for the customer? The fourth area is company business. A development project may have synergies with other projects, or it may lead to cannibalizing some existing business. Thus, the business impact of the new product should also be considered. Also, in order to be able to extract fully the success potential of the new product, some t between the new product and the companys present business is required. Otherwise, the supply chain, manufacturing process and customer base would have to be subjected to a dramatic adjustment for the new product.

It is suggested that there is a relevant link between the ideas presented in Figures 1 and 2. As Matthews (Matthews, 1991) has stated, for the projects that are neither pure research (category C) nor actual product development (category A) it might be challenging to establish sound arguments that either support or oppose further development efforts. As one solution for this managerial problem, a holistic framework such as the one presented in Figure 2 including the identi ed success factors of NPD might be a useful tool for structuring the issue. Especially in the assessment and in the prioritization of the category B development projects, a decision-maker should consider various factors that represent the viewpoints of different stakeholders in the NPD process. If each of the projects at hand is evaluated from the perspectives presented in Figure 2, it should be possible to more rationally or at least more explicitly rank the projects based on the anticipated success factors. Of course, this kind of assessment can probably never be totally objective or de nite; however, it is supposed to be an attempt to systematize organizational discussion and decisionmaking regarding projects: . whose justi cation is not self-evident; and . that cannot be (yet) evaluated using direct nancial criteria.

Objective and methods

The aim of the paper is to analyse the multidimensionality of success at the R&D project level. Success is considered as a variable that is connected with the NPD process and its phases. That is, success as a goal of NPD can be seen very differently, depending on the phase of the R&D process. The paper also addresses the issue that there are practically meaningful patterns of success, which are also very different from each other, that pose challenges for R&D managers who are striving for effective and generally successful NPD. A core problem is related to the correlation between different success domains: good performance in one success domain may or may not be related to success in another. The paper is based on a case study conducted in the metal industry. All three cases studied are product development projects involving investment goods products of the metal industry companies A

217

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

and B serving customers in the papermaking industry. In the customers business the growth rate is typically between 2 and 4 percent. Big investments in production lines affect the market demand for paper grades, which results in some cyclical uctuations in the market demand for production machinery. Another essential characteristic of the business is the length of product life cycles (PLCs), which typically vary between ten and 15 years. The challenge to PLC management in this industry is underlined by the fact that the introduction of totally new technology can typically take ten years or more, mainly because of the risks of ruining the payoff of the entire production process when technology is introduced too early. The dif culty and complexity of the technology transfer process from phase C to A in this particular industry area is reviewed with several cases in the paper by Crotogino (2000) in IDS 2000. New technological solutions are commonly introduced in the industry rst by rebuilds of paper machines and then nally by new production line deliveries. Thus, feedback from markets and learning by experience tend to be slow and time to market long, at least in the case of major components of the production line. The learning process and the process of managing the risks in the development of components for the papermaking line are improved by demonstrating the potential of equipment on pilot machines. All the product development cases in this paper were carried out in the 1990s and the market launch of products took place in the late 1990s. In each case the following components of success discussed by Rouhiainen in his dissertation focusing on the management of NPD project implementation in the metal industry (Rouhiainen, 1997) were evaluated: . project management success; . technical success (businesspersons, engineers and clients point of view); and . commercial success estimate. Project management success has been estimated utilizing the PIP questionnaire constructed by Slevin and Pinto (1986). Slevin and Pinto (1986) have identi ed ten success factors, including for instance project mission, top management support, client consultation, communication and troubleshooting, that are related to a projects success. By studying 82 successful projects,

Slevin and Pinto have provided a reference score scale for each success factor that enables benchmarking a certain project performance with the success pro les of known successful projects. (For instance regarding the factor of communication, the 0th percentile of 12 points refers to the fact that none of the studied success projects scored less than 12 points, and the 100th percentile of 99 indicates that the full score of 100 was not achieved by any project.) If a projects performance in the case of any factor is below the 50th percentile, one should according to Slevin and Pinto (1986) devote extra attention to that factor to improve the odds of success. As regards the other success areas, technical as well as commercial success has been analysed in this study by interviewing the key development personnel and management of NPD projects as well as by studying the companys NPD reporting and documentation. Concerning the theoretical representativeness of the study, the cases studied are quite different from each other. Case 1 was basically a new technology development project, the outcome of which was a new product utilizing technologies known in company A. Case 2 was a product development project of company A aimed at introducing a product similar to but better than existing ones in the marketplace. Case 3 involved introducing an incrementally improved product using technologies known in the companies participating in the development project (companies A and B). In case 3, company B is a supplier of company A. In Table I the cases are outlined in terms of main development characteristics. The key parameters utilized in Table I are de ned and discussed by EIRMA (1995); it also illustrates the diversity of the cases in compact form. The rationale for using a case study approach is to provide holistic and meaningful characteristics of real-life events (Yin, 1994). According to Yin (1994), a case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context. Typically, in case study research, the boundaries between phenomenon and context are not clearly evident. In this study, a product development project is the phenomenon to be investigated and the context is a companys product development organization and its environment. A holistic

218

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

Table I Characteristics of the cases as they were seen in the phase B when development decisions were made Case 1 Technologies applied Type of R&D Market Feedback from market Time to market Competition Innovation driver Known by company A Radical Known Low Long Medium Technology push Case 2 Known by company A Incremental Less known Medium Medium High Regulations Case 3 Known by companies A and B Incremental Known Medium Medium High Market price/Technology push

view is needed when making R&D decisions because the results of development work would greatly in uence the whole company and its business potential in the future. Therefore, the interests of various stakeholders should be somehow considered in the research and development process.

Case evidence
Case 1 Case 1 was originally a technology development project established to reduce the investments in a paper production line. The result of the technology and product development project was a commercial product making possible up to 30 percent savings in machinery investments when the product is utilized in full scale. The development history (phases C, B, A described by Matthews (1991)) can be summarized as follows. Phase C lasted four years. It can be described as applied research aiming to study if the technology could be utilized in an application area. The research was conducted both by the company and by external research laboratories as well as technical universities. This phase was focused on minimizing the risks of choosing the wrong technology for the concept and development phase. Phase B lasted three years. The objective of the concept development phase was to evaluate the feasibility of the product concept as a strategic option. During this period, pilotscale equipment for this technology was built and tested. This was done to further minimize the risk associated with the new technology and to evaluate its applicability for several customers. The most critical components of the technology were tested and designed in phase B. Phase A lasted two years. It ended with the rst customer delivery of the new technology

product. During this stage the product was developed and designed mainly within the delivery project. However, many design features and solutions were based on the detailed studies made in phase B. Generally, passing a development project from concept development to the actual product development phase requires an estimation that there is enough potential in the concept in respect of risks associated with it to achieve an acceptable level of success. In this case, concept development for the product included many (technical) risks that were also identi ed: . the effect of the technological solutions characteristic of the concept on the quality of paper grade; . the implications of the concept for runnability issues; . the effects of the concept especially on the durability of core peripherals that are in interaction with the product. However, despite the risks, concept development was encouraging to the extent that it was decided to proceed to product development. The selection of the project for actual product development was mainly based on the following criteria (estimated bene ts of the product): . Possibility for the customer to have lower investment costs. . The quality and runability of the paper grade remains at the very least the same. . Positive impact on the ef ciency of the production (customer process). . Simplicity: good product architecture, low number of components. . Lowering energy consumption. Project management success in case 1 was studied by implementing a reduced PIP questionnaire (see Slevin and Pinto, 1986) among the key people of the development personnel from phase B to phase A. The questionnaire was supplemented by

219

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

interviews conducted in the same target group. The PIP questionnaire revealed the perception that the project was fairly well managed. The weakest identi ed critical success areas were (see also Figure 3): . Client consultation and client acceptance; evidently, this domain should have been studied better and more comprehensively in phase B. One reason for the poor or absent customer consultation was the con dential nature of the project in this phase. . The schedule/plan, which can be partially explained by the length of the project and changing personnel during the project. . The project mission, which was to some extent unclear. This was mostly due to the fact that the project dealt with fairly radical development, which caused dif culties in specifying and operationalizing the exact goals for the project personnel. What can be said about the technical success on the basis of the inquiry and interviews? From the technical point of view, the product developed was a success in the rst delivery in almost every respect. The technical speci cation and process performance targets set by the customer were met. The new product created in the project made it possible to cut the investment costs by 30 percent for the production line when utilized in full scale. From the businessmans point of view the manufacturing costs of the product could have been lower in order to be able to meet the market price of competing technology.
Figure 3 PIP pro les of the studied cases

The commercial success of the new technology product so far has been poor: no further references have been obtained two years after the rst one. However, interest among customers has been fairly high. The lack of commercial success is also due to competing products. Another reason is probably the timing: the release of the new technology product in case 1 took place too early and was not properly synchronized with other product releases. The product as such is an outcome of long-term technology strategic choices of the company as the technology leader in the industry. The company is still considering additional investments in order to attain the strategic targets set many years ago for the product. Observations from the interviews (n 4) with key development personnel and product development management can be summarized as follows: . Strategy implications: in line with the established strategy, but payoff is still a question, even if the technology project has lasted a very long time. . Customer and competition: price competitiveness of the product is the major concern. . The business impact of the product is rather fuzzy; there are also some competing products within the same company. . The development project leader feels unsure regarding the customers ability to derive full bene t from the product.

220

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

.

Technology: the pro tability of the project appears weak although the technology is considered mature enough.

Case 2 Case 2 was a spin-off of an applied research project intended to reduce the environmental impact of the papermaking process. The partners of the research project were: . a machine supplier; . a customer in the paper industry; and . a commercial research centre. During the project the machine supplier got an opportunity to deliver new process equipment tailored to the speci c needs of the client. The decision to develop and deliver was, however, not very spontaneous because the product concept was elaborated quite a long time in the concept development phase. At the time, market value and the major competitors were still being evaluated. Some early signals from customers had been received, showing an emerging market demand for the product. The development history of case 2 can be summarized as follows. Phase C lasted two years and it was more or less a process development using simulation tools. At this stage no equipment development was undertaken. During phase C the equipment developer and supplier became familiar with a new kind of production process environment for the company. Phase B lasted two years. At this time, the product concept was planned and tested on the pilot scale. The dimensioning of the equipment was also studied for scale up to industrial scale. The customer-partner worked with the development team to tailor and specify the product according to his needs. Some other customers were also consulted, but their in uence on the nal product speci cation was minor. Phase A lasted for one year and during that time the rst customer delivery of the case product took place. What was characteristic of the concept development of this product? The development project was very strongly customer-driven: although there seemed to be a general need for this type of product, the concept was developed in close collaboration with a customer. Thus, the product was primarily tailored to ful l the customer-speci c needs. It can also be said that the very good customer collaboration pushed the project forward by giving a suf cient justi cation for

the work. Other grounds and criteria to continue were not so much needed. A market study was also conducted at the concept development phase and a lot of effort was expended to select the most appropriate materials for the product. Still, the project resulted in a product that was too expensive to manufacture and whose market was inadequately studied. Particularly, the market for the product does not seem today as tempting as when it was assessed and perceived in the concept development phase. Afterwards, R&D management has pointed out that the concept development might have been too rapid, thus preventing a versatile assessment of different construction alternatives. Project management success according to the stakeholders in the case project was reasonably good (see also Figure 3). Generally, the project was thought to be well-managed and organized. According to the project manager, the weakest area was the lack of comprehensive client consultation and marketwide client acceptance, which was misleading for the product concept choice. Also, the project did not have fully satisfactory and speci c goals that would have enabled wellfounded project assessment and monitoring. Regarding the technical success of the case, two customer requirements were met in the rst delivery both as regards the technical speci cation and the service provided for the customer within the delivery project. From the businessmans point of view, the manufacturing costs were too high in the rst customer delivery. This was mainly due to the heavy construction with expensive construction materials. Also, there was only minor experience of the type of manufacturing involved. After the rst delivery the technical speci cation of the developed product was benchmarked more carefully than in concept phase B. The conclusion was that some technical improvements had to be made to reach the level of competition in the market segment within the scope of company A. Commercial success The case product was marketed for a one-year period. There were some customer inquiries but no further discussions with the customers because of the selling price, which seemed to be too much above an acceptable market price level. After positioning the new product in respect to the main competitors and studying

221

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

the market segment volume and availability once again, the product launch was stopped. The product concept was not competitive enough and the best option was to withdraw from the market. Catching up with the market price level was not considered to be feasible. The learning process would have been too long and expensive compared to the expected returns. Observations from the interviews (n 4) with key development personnel and product development management can be summarized as follows: . Strategy: no clear support from strategy in this project. . Customer and competitive position: no clear value added from the business or customer perspective. . Companys business: Some additional business and synergy contribution seen; however, poor cost ef ciency of the product. . Technology: customers capability to exploit technology is fairly good. . Reliability of the technology was good. . Overall: one good area in scoring does not make the product and project lucrative from the perspective of business and strategy. Case 3 The case 3 product development was a cooperation project between companies A and B. The outcome of the development project was a merger of two proven technologies in the industry. Company A had the supporting technology know-how and experience, while company B had the main technology regarding the new product. Concerning the development history of case 3, phase C is not applicable. Phases B and A partially overlapped in the development project. During phases A and B the product concept was tested in the laboratory and on a small scale on a pilot machine. This phase lasted about one year and it ended with the rst commercial order. With regard to perspectives of future success in this project, it was particularly interesting that the concept development was started in the belief that a potential new killer product could be constructed by combining two technologies. Therefore, company A could not take the risk of totally opting out of the project. In that case, if the product succeeded, it would have provided the competitors with an essential competitive

advantage. However, the R&D collaboration between the two companies is a dif cult sport: different roles and interests of companies, the question of sharing future revenues, product management and marketing principles, and the combination of different organizational cultures set major challenges. In line with this, the two companies, A and B, did not totally agree on the performance (e.g. breeding ratio) of the product. This caused dif culties since it led, for instance, to different sales promotion argumentation. Among the project personnel opinions varied widely regarding the project management success. Although the technical results were satisfactory, there seemed to be tension between the business interests of companies A and B. Both companies admitted that they had learnt a lot from each other and the development group was committed and knowledgeable. Another common conclusion was that the development project was fairly well managed regarding schedule and budget. However, on average, project implementation pro le of this project is far from excellent (see Figure 3). The technical speci cation of the case 3 product was set at the beginning of the project in phase B. A compromise speci cation was nally accepted by both companies. Company Bs expectations were higher before and during the development project because it was to be the owner of the product concept to be developed and the supplier of the new product. The speci cation set for the product was not totally met in the development project, but the gap was not expected to be crucial marketing-wise. Commercial success Two years after the end of the development project, company B has sold some 20 units of the new product. Part of them were sold and delivered through main supplier A. Time to market was short and penetration in the rebuild market was successful. The question of whether the product can be exploited fully on the market for new machines is still open but the prospects seem encouraging. Observations from the interviews (n 5) with key development personnel can be summarized as follows: . Strategy: the link to the strategy is questionable or weak and not discussed properly inside company A, in contrast with the situation in company B.

222

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

.

Customer and competition: the product is not believed to provide additional value in a proven way in company A and uncertainty also exists about the price competitiveness. Companys business perspective: there is some synergy but the product does not provide new business for company A. The total economic effect is somewhat unclear. Technology: the customer is ready to implement the technology and it is manufacturing-wise all right. Reliability in wide machines is still unsure after implementation by several customers.

As a result of the development project, no superior product was created compared to the previous product generation from the perspective of company A. Company B has in any case managed to differentiate the new product in competition and it has sold a considerable amount of case 3 products. The solution developed has therefore demonstrated its potential. It seems to have a certain place in the market applications.

Analysis of cases
Figure 3 summarizes the project implementation pro les of the three cases studied. The x-axis represents the important parameters or success domains of a project that are evaluated; and the y-axis illustrates the percentage of known success projects (see Slevin and Pinto, 1986) that perform more weakly in that respective domain than projects now studied. According to the PIP method, the projects seem to be clearly different from each other: the pro le of case 2 is the most positive, while that of case 3 is the worst. Case 1 lies in the middle. This observation suggests that the PIP method, despite the fact that it has proved to be a valid measure of success in a larger data set, fails to interpret reliably the odds of success concerning an individual project. In an individual project, it seems that a majority of necessary ingredients of success might be present and still the nal success remains poor, or vice versa. This would be one reason to advocate holistic and versatile NPD performance measurement, which is discussed, e.g. by Cooper and Kleinschmidt (1995). The summary of all success perspectives studied in three cases is presented in Figure 4.

In each case, the success pro le consists of three different success perspectives: R&D project management, technical and commercial success. Case 1 represents a case that is project management-wise rather neutral. Management of the R&D project included some clear weaknesses but also clear strengths. In terms of technical success, case 1 can be considered a positive project; no serious doubts exist, except perhaps concerning price competitiveness. Importantly, however, the commercial success of the product has been at least so far too weak. The volume of business has remained small. In case 2 the project management received good grades. There were no severe weaknesses identi ed regarding the project management. Also the technical success seemed to be more or less all right. Concerns mainly lay with the manufacturing costs. In contrast to this, there was practically no commercial success at all with this product. Due to the reasons discussed in the case section of this paper, the product was nally found to be not competitive enough for the market. Case 3 represented R&D collaboration between two companies. That resulted in a project, whose management proved to be to some extent problematic. Although the technical success of the project was not according to the most optimistic expectations, it was still acceptable. Most importantly, the commercial success of the product has been rather positive, especially from the perspective of company B. What is the picture of success of NPD drawn by these cases? First, the three cases most certainly do not constitute a homogeneous pro le. One of the three projects has led to fairly successful business, while the other two have remained somewhat disappointing efforts for the companies. The only commercially successful project (case 3), however, was not a result of a development project that ran smoothly and as planned. Instead, the development work proved to be rather problematic and even the technical result of the project was to some extent a compromise. Furthermore, the projects connection with the strategy was questionable and it was not discussed thoroughly either before or during the active development. In the light of typical success factors discussed in the literature, including product superiority/ quality, market need, product scope, highquality new product process, clear and well-

223

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

Figure 4 Summary of case results

communicated new product strategy for the business unit and positioning in respect of markets and competitors products (see Cooper, 1985, 1996; Hollander, 2000), this project is certainly interesting. The product cannot be characterized as superior, but it may be labelled as a fairly good one that has found its place in the market, indicating some success in positioning the product. Also, reasonable commercial success was achieved without high-quality new product process and a strategy connection. In contrast, case 2, which proved to be a commercial failure, was from the perspective of project management an excellent example. Also the technical success seemed to be all right. An important insight that helps to understand these contradictory observations made in case 2 is provided by Ottum and Moore (1997) and Campbell and Cooper (1999). Ottum and Moore (1997) have investigated the role of market information in new product success or failure in their study and have shown that there is a strong relationship between market information processing and new product success. They also stress that effective market information processing requires not only the gathering of good quality information but also the sharing and using of that information (Ottum and Moore, 1997). However, regarding customer involvement in NPD there is also somewhat surprising evidence available. Namely, it is argued that there is no automatic short-term commercial bene t associated with customer partnering compared to in-house development (Campbell and Cooper, 1999). Indeed, one should be very careful in gathering market information and integrating it in the NPD process. Especially in circumstances analogous to case 2, where industrial investment goods were being

developed, on the basis of a very limited number of customer opinions, it is too easy to get the impression that the product and its characteristics are based on market needs even when this is not the case. One or two active customers in the development phase may, as happened in case 2, lead the developing company into areas which are far from real market need. Furthermore, case 1, which clearly succeeded in terms of technological goals and did not fail from the perspective of project management, did not make a clear commercial difference. In this light, it seems that neither project management nor technical success alone are measures that should be used to anticipate commercial bottom-line success in this particular environment. Partly this is due to a situation analogous to that in case 2 regarding market or customer information. Instead of being the result of a market study, the product speci cation was created mostly on the basis of a strategic de nition of company policy. Still, the product seems to be able to deliver value for the customer. It is, however, noteworthy that the company has failed in the task of value/cost assessment regarding this product. To be able to compete in the market successfully, product cost and value have to be in balance. So far, this has not been the case. As regards the problem presented by Matthews (1991) according to which category B projects may often fail to receive justi cation for funding since they neither represent pure breeding ground any more nor have reached the status of investments that could be assessed using nancial measures, the cases in this paper provide one though a limited insight. Especially when examining the nancial success of the project, it is necessary to assess and to measure the

224

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

project from the perspectives of all relevant stakeholders. Technological success or strategic t is not enough if there is, for instance, lack of understanding concerning customers value creation or competitors positions and offerings. Finally, to turn an otherwise successful project or product into good business, one has to consider also issues such as cost ef ciency or manufacturability. As Neely et al. (1995) among others, point out, the main sources of inappropriateness of performance measurement, in general, are short-termism, lack of strategic focus and local optimisation. A holistic view seems to be required. In the light of the cases presented in this paper, NPD is no exception to the other application areas of performance measurement: good performance measurement captures a number of dimensions that are associated with success in a particular case. Regarding the perspectives introduced earlier which make a contribution to new product success, the observations of the study can be summarized as follows: . Strategy: without some kind of strategy connection, an R&D project is not likely to receive the resources and support that are an imperative in assuring the proper conduction of a development project. However, on the basis of all this case evidence, a strategic t or strategic support does not seem to be able to secure any success if the other fundamental issues such as customer acceptance, right timing or feasible costs are neglected. The cases reported in this paper serve as good examples of this. . Company business: the implications of the project for the companys business and competitive situation should be evaluated well. Even if the primary target is not the cost leadership, cost ef ciency has to be at an acceptable level. A technically good product with the wrong price tag on it is not at the end of the day a lucrative product. . Technology: the technologies used in the products under development should be well proven both separately and together in the end-user environment. In the development process itself it could be wise strictly to separate different phases from each other. That is, the development of the product should be carried out separately from the development of the technology. It might also be very valuable

to explore fully the potential of existing product technology before introducing new technology. Customer and competition: also, it is very important for a better understanding of customer expectations to consult several customers in the selected market segment during the product development phase. Apparently, customer-driven R&D may well be to some extent misleading if it is based only on the identi ed needs of a single customer.

This paper is not in a position to give de nite recommendations regarding the means for implementing performance measurement or evaluation. At least there is no lack of different available methods (see Martino, 1995). Whether the measurement is done by construction of numerical indices, struggling with gures and scoring the projects (McLeod, 1988) or by more subjective assessments, the main challenge is not the actual measurement but the process of making the right decision on the basis of the information available.

Discussion
The objective of the paper was to analyse the dimensions of success at R&D project level. Success was seen as a variable that is connected with the NPD process and its phases. An assumption behind the paper was that there are different patterns of success for R&D projects, which are also very different from each other. A core dilemma was related to the correlation between different success domains: a good performance in one success domain may or may not be related to success in another one. In line with the literature (see, e.g. Hart, 1993; Grif n and Page, 1996), the cases investigated in the paper showed that success in NPD is indeed a multifaceted and elusive issue. Project management success, technical success and nancial success seldom go hand in hand as unwelcome as this may be from the management point of view. In the process of specifying and interpreting the meaning of NPD success, this fact reinforces the perception that success is both a context- and stakeholder-speci c issue. The very same project can even with sound argumentation be interpreted either as a failure or as a success story, depending on who is evaluating

225

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

the project, the situation or the timeframe within which evaluation takes place. Also, different product development projects often represent quite different patterns of success: one project may perform very well in nancial terms, another may be pro tability-wise a failure but in the technical sense by meeting the state-of-the-art technical requirements a big success story, while a third projects outcome may be poor almost in every respect but the project management has been ef cient and well organized, thereby providing the company with a good example of outstanding R&D project management also for the future. The existence of different patterns of success challenges R&D management. Striving for success might include seeking compromises between different forms of success. If a project cannot be successful in every domain, it still may be able to contribute in some areas. Taking into account that investments in R&D are expected to produce returns, especially in the long term, an individual project can be considered a success to some extent if it can either produce a direct impact or indirectly produce at least the preconditions for future returns. Therefore, a systematic demand for direct payoffs in each NPD project that refuses to take into account the importance of projects indirect impacts on success may even be harmful in terms of the overall success of NPD. Although the three success perspectives studied suf ce to demonstrate the elusive and varied nature of NPD success, this analysis has excluded other issues that are also of interest. For example, as seen in the context of knowledge creation, the concept of product development success receives one additional interpretation: R&D activities can be perceived as successful when they facilitate or generate valuable knowledge. This seems to be reasonably well in line with Lewis (2001) notion that NPD can be seen as a speci c illustration of organizational learning. Lewis (2001) argues that the interaction between resources and processes is not unidimensional: resources create value when they are utilised in processes, which in turn helps to create new resources or extend existing ones. In NPD this means not only that skilled resources are one of the prerequisites of a good NPD process, but a good process by itself also generates valuable information, knowledge or experience i.e. resources. Drejer and Riis (1999) view competencies as a

system of human beings using technology in an organized way and under the in uence of a culture to create output that yields a competitive advantage for the rm. Both of these views are essentially similar: they perceive competencies to some extent as systems action and interaction are strongly present, and both link competencies closely to the creation of competitive advantage. Competencies are not just any ability to respond to random demands, but rather building blocks for a rms competitive advantage. Since the concept of success is itself dif cult to de ne, it is a dif cult task indeed to try to describe the road leading to success. Therefore, on the basis of this study, it would be questionable to present any standard formula for new product success. However, consistently with the ndings of this paper, many variables that have an in uence on success have been identi ed also in prior studies: for instance, product superiority, market need and competitive position (see, e.g. Cooper, 1985; Cooper and Kleinschmidt, 1995; Hollander, 2000; Hultink et al., 2000). Also, concerning the generalization of the results, it is not claimed that all development projects behave similarly to the projects analysed in this paper. That is, a project could possibly succeed in all domains or it could fail in all three. However, it seems to be a common of product development projects that success often has something to do with coincidences or chaos. Perhaps everything cannot be rationalized.

References
Batty, J. (1988), Accounting for Research and Development, 2nd ed., Gower Publishing, Aldershot. Brown, M.G. and Svenson, R.A. (1998), Measuring R&D productivity, Research Technology Management, Vol. 41 No. 6, pp. 30-6. Campbell, A.J. and Cooper, R.G. (1999), Do customer partnerships improve new product success rates?, Industrial Marketing Management, Vol. 28, pp. 507-19. Chiesa, V. and Masella, C. (1996), Searching for an effective measure of R&D performance, Management Decision, Vol. 34 No. 7, pp. 49-57. Cooper, R.G. (1985), Selecting winning new product projects: using the NewProd System, Journal of Product Innovation Management, No. 2, pp. 34-44. Cooper, R.G. (1996), Overhauling the new product process, Industrial Marketing Management, Vol. 25, pp. 465-82. Cooper, R.G. and Kleinschmidt, E.J. (1995), Benchmarking the rms critical success factors in new product

226

The patterns of success in product development: a case study

European Journal of Innovation Management Volume 6 Number 4 2003 213-227

Petri Suomala and Ilkka Jokioinen

development, Journal of Product Innovation Management, No. 12, pp. 374-91. Crotogino, R.H. (2000), The mysteries of technology transfer luck versus good planning, paper presented at the International Drying Symposium IDS2000, Noordwijkerhout. Drejer, A. and Riis, J.O. (1999), Competence development and technology. How learning and technology can be meaningfully integrated, Technovation, No. 19, pp. 631-44. Driva, H., Pawar, K.S. and Menon, U. (2000), Measuring product development performance in manufacturing organisations, International Journal of Production Economics, Vol. 63, pp. 147-59. European Industrial Research Management Institute (EIRMA) (1985), Evaluation of R&D Output, Working Group Reports. Report No. 29, EIRMA, Paris. European Industrial Research Management Institute (EIRMA) (1995), Evaluation of R&D Projects, Working Group Reports. Report No. 47, EIRMA, Paris. Ellis, L. (1997), Evaluation of R&D Processes: Effectiveness through Measurements, Artech House, Boston, MA. Grif n, A. (1997), PDMA research on new product development practices: updating trends and benchmarking best practices, Journal of Product Innovation Management, No. 14, pp. 429-58. Grif n, A. and Page, A.L. (1996), PDMA success measurement project: recommended measures for product development success and failure, Journal of Product Innovation Management, No. 13, pp. 478-96. Hart, S. (1993), Dimensions of success in new product development: an exploratory investigation, Journal of Marketing Management, No. 9, pp. 23-41. Hirons, E., Simon, A. and Simon, C. (1998), External customer satisfaction as a performance measure of the management of a research and development department, International Journal of Quality & Reliability Management, Vol. 15 No. 8/9, pp. 969-87. Hollander, J. (2000), Genesis, a product assessment instrument used during the product development process, paper presented at the 7th International Product Development Management Conference, EIASM, Leuven. Hultink, E.J. and Robben, H.S.J. (1995), Measuring new product success: the difference that time perspective makes, Journal of Product Innovation Management, No. 12. Hultink, E.J., Atuahene-Gima, K. and Lebbink, I. (2000), Determinants of new product selling performance: an empirical examination in The Netherlands, European Journal of Innovation Management, Vol. 3 No. 1, pp. 27-34. Industrial Research Institute (IRI) (2000), R&D Facts 2000, Industrial Research Institute, Arlington, VA. Jackson, T.W. and Spurlock, J.M. (1966), Research and Development Management, 2nd ed., Dow JonesIrwin, Homewood, IL. Kerssens-van Drongelen, I.C. and Bilderbeek, J. (1999), R&D performance measurement: more than choosing a set of metrics, R&D Management, Vol. 29 No. 1, pp. 35-46. Kulvik, H. (1977), Uusien tuotteiden onnistumiseen tai epaonnistumiseen vaikuttavat tekijat, Helsinki University of Technology, Helsinki (in Finnish), 93pp.. Lewis, M.A. (2001), Success, failure and organisational competence: a case study of the new product

development process, Journal of Engineering Technology Management, No. 18, pp. 185-206. Lindman, M. (1977), Managing industrial new products in the long run, Acta Wasaensia, University of Vaasa, Vaasa, 336 pp.. Makinen, S. (1999), A strategic framework for business impact analysis and its usage in new product development, Acta polytechnica Scandinavica, Tampere University of Technology, Espoo, 213 pp.. Martino, J.P. (1995), R&D Project Selection, John Wiley & Sons, New York, NY. Matthews, W.H. (1991), Kissing technological frogs: managing technology as a strategic resource, European Management Journal, Vol. 9 No. 2, pp. 145-8. McGrath, M.M. and Romeri, M.N. (1994), The R&D effectiveness index: a metric for product development performance, Journal of Product Innovation Management, Vol. 11 No. 3, pp. 213-20. McLeod, T. (1988), The Management of Research, Development and Design in Industry, 2nd ed., Gower Technical Press, Aldershot. Neely, A., Gregory, M. and Platts, K. (1995), Performance measurement system design: a literature review and research agenda, International Journal of Operations & Production Management, Vol. 15 No. 4, pp. 80-116. Nixon, B. (1998), Research and development performance measurement: a case study, Management Accounting Research, Vol. 9, pp. 329-55. Ormala, E. (1986), Analysing and Supporting R&D Project Evaluation: An Applied Systems Analytic Approach, Helsinki University of Technology, Espoo. Ottum, B.D. and Moore, W.L. (1997), The role of market information in new product success/failure, Journal of Product Innovation Management, No. 14, pp. 258-73. Poolton, J. and Barclay, I. (1998), New product development from past research to future applications, Industrial Marketing Management, No. 27, pp. 197-212. Rouhiainen, P. (1997), Managing New Product Development Project Implementation in Metal Industry, Tampere University of Technology, Tampere. Schumann, P.A., Ransley, D.L. and Prestwood, D.C. (1995), Measuring R&D performance, Research Technology Management, Vol. 38 No. 3, pp. 45-55. Slevin, D.P. and Pinto, J.K. (1986), The project implementation pro le: new tool for project managers, Project Management Journal, September, pp. 57-71. Szakonyi, R. (1994a), Measuring R&D effectiveness I, Research Technology Management, March-April, pp. 27-32. Szakonyi, R. (1994b), Measuring R&D effectiveness II, Research Technology Management, May-June, pp. 44-55. Ulrich, K.T. and Eppinger, S.D. (1995), Product Design and Development, McGraw-Hill, New York, NY. Werner, B.M. and Souder, W.E. (1997a), Measuring R&D performance state of the art, Research Technology Management, Vol. 40 No. 2, pp. 34-42. Werner, B.M. and Souder, W.E. (1997b), Measuring R&D performance US and German practices, Research Technology Management, Vol. 40 No. 3, pp. 28-32. Yin, R.K. (1994), Case Study Research: Design and Methods, 2nd ed., Sage Publications, Newbury Park, CA.

227