Sell First, Fix Later: Impact of Patching on Software Quality

Transcription

1 Carnegie Mellon University Researc CMU Heinz College Researc Heinz College Sell First, Fix Later: Impact of Patcing on Software Quality Asis Arora Carnegie Mellon University Jonatan P. Caulkins Carnegie Mellon University Raul Telang Carnegie Mellon University Follow tis and additional works at: ttp://repository.cmu.edu/einzworks Tis Working Paper is brougt to you for free and open access by te Heinz College at Researc CMU. It as been accepted for inclusion in Heinz College Researc by an autorized administrator of Researc CMU. For more information, please contact researcsowcase@andrew.cmu.edu.

2 Sell First, Fix Later: Impact of Patcing on Software Quality Asis Arora Jonatan P. Caulkins Raul Telang {asis; caulkins; H. Jon Heinz III Scool of Public Policy and Management Carnegie Mellon University Abstract We present an economic model of fixing or patcing a software problem after te product as been released in te market. Specifically, we model a software firm s trade-off in releasing a buggy product early and investments in fixing it later. We first sow tat patcing investments and time to enter te market are strategic complements suc tat iger investments in patcing capability allow te firm to enter te market earlier. Just as te marginal cost of producing software can be effectively zero, so can be te marginal cost of repairing multiple copies of defective software by issuing patces. We sow tat due to te fixed cost nature of investments in patcing, a vendor as incentives to release a buggier product early and patc it later in a larger market. We contrast tis result wit oter pysical good markets. Tus, we sow tat a monopolist releases a product wit fewer bugs but later tan wat is socially optimal. We extend our model to incorporate duopoly competition and sow tat in competition, te ig value firm always enters earlier tan te monopolist. Ironically te firm offering greater value to customers releases a product tat initially is of lower uality (more bugs), but provides te greater value by releasing early (so customers can use te product sooner) and by investing more in patcing so it can provide better after-sale support to its customers. July 003

3 1. Introduction Software as become an intrinsic part of te very fabric of our lives. Virtually every commercial as well as government and non-profit enterprise depends on software to provide production, operations and marketing services to its customers. In 00, te U.S. software market was wort almost $180 billion, wit 697,000 employed as software engineers and an additional 585,000 as computer programmers (NIST, 00: 1). As software becomes ingrained in daily business activities, its failures become even more critical. Media reports of catastropic effects of software failure abound. In addition to wellknown failures suc as te buffer overflow bug responsible for te Ariane 5 rocket blowing up 40 seconds after liftoff in 1996 (estimated loss of over $500 million), tere are less widely known instances as well. For example, a software failure interrupted te New York Mercantile Excange and telepone service to several east coast cities (Wasington Post, 1998). AT&T lost significant revenues due to faulty software in one of its CISCO switces (Business week, 1999). 1 Altoug uantifying te economic cost of faulty software is an open empirical uestion, estimates range in te tens of billions of dollars every year. For instance, a recent study suggests tat even 1% downtime in one small ospital translates into $1.5 million of economic cost (Anderson, 00). Te business community pays close attention to software malfunctions. Fully 97% of te 800 managers surveyed reported software flaws in teir systems in te past year. More tan 90% blamed faulty software for lost revenue or iger costs. Some 6% said tey believed te software industry was doing a bad job of producing bug-free software (Information 1 More examples, some possibly apocrypal, can be found at ttp:// A recent NIST report claims tat major software bugs cost te US economy over $60 billion per year.

4 Week, 00). Software uality as become even more important recently due to eigtened security concerns arising from software vulnerabilities. 3 Te final uality experienced by a software user is a combination of te bugs or defects in a software product wen it is sipped plus te efforts te vendor makes to fix te bugs after sipping. Indeed, most models of te software development cycle explicitly incorporate te possibility of improving te product after release (Evans and Reddy 00). In tis paper, we analyze ow a software vendor decides to allocate effort along tese two dimensions. Terefore, te software vendor in our model can delay product release to reduce bugs or invest in ex-post (after te product is sipped) remediation of te bugs. Tere is ample evidence tat many leading software vendors routinely release patces, at least in certain application domains. Sun released more tan 00 patces for Solaris 9.0 to fix more tan 100 bugs in a six-mont period (downloaded from ttp://sunsolve.sun.com). Similarly, Microsoft issues patces on an almost daily basis (Te Economic Times, 00), and Hewlett-Packard puts out an average of 60 to 80 HP-UX patces per week (InterWorks, 000). We assume tat bugs can be reduced only by delaying te release of te software product. It is a commonplace about software vendors tat time to market pressures are an important reason, if not te most important reason, wy software as bugs. In our model, delay is costly because, all else eual, users would rater get a product sooner so tey can use it longer. It is tempting to assume tat firms can reduce te time to market witout affecting te 3 Te sutting down of EBay and Yaoo! websites and te Code Red virus affecting more tan 300,000 computers due of vulnerabilities in Microsoft products are just some of te instances wic ave appeared in popular press. As software products get integrated and more organizations use networks for critical business operations, security vulnerabilities become serious treats. Microsoft as invested more tan $100 million to improve te security of its products. It is retraining more tan 8500 programmers to improve te security of its products (Standage, 00). 3

5 product uality by simply increasing te number of developers. 4 However, in software, tis assumption is uestionable. Indeed, Brooks famous study of te IBM OS/360 suggests te opposite: Te greater costs of coordinating among more developers may outweig any gain in efficiency, so tat adding people to a project may actually delay it furter witout any improvement in uality (Brooks, 1995). Furter, as Amdal s law implies, tere may be a natural limit to te size of te product development team, especially for new software products. In fact, industry leaders ave recognized tat development teams sould be kelp small, constant and manageable for optimal performance (Te Wall Street Journal, 1991). To capture tis notion, and to focus on te time-to-market pressures, in our model, a vendor can reduce te number of bugs, albeit at a diminising rate, only by delaying release. In addition, te vendor commits to an investment in patcing tat may alleviate some of te disutility due to bugs. Te iger tis investment is, te greater a user s willingness to pay for te product. 1.1 Researc Question We formally analyze ow te ability of a vendor to patc te bugs later affects its decision about wen to enter te market, and ow buggy te product is. We analyze tese coices under different market structure; i.e. monopoly and duopoly, and compare to a socially efficient bencmark. Finally, we note ow tese decisions cange wen te product is a tangible good like an automobile instead of software. 4 Organizations can, and do, reduce bugs by adopting processes suc as CMM or ISO9001. Ultimately, owever, aving fewer bugs at te time of release depend on careful design, coding and extensive testing under different conditions and for different tasks. Tis takes time. 4

6 A key finding is tat te ability to fix bugs later always leads to an earlier release of te product. But a more striking finding is tat it is indeed socially optimal to release te product early and to patc extensively later. In oter words, in market euilibrium, a monopolist actually releases products later and wit fewer bugs tan is socially optimal. But te monopolist is also less willing to patc as extensively. Interestingly, a larger market increases incentive to release te product earlier wit more bugs. Competition does cause a firm to release earlier, but still later tan is socially efficient. Ours is a full information model were users are fully aware of te uality as well as te producer s ability to fix it later. Hence, any sub-optimality is not due to incomplete information. Furter, since a vendor is assumed to be able to commit to patcing, we ignore uncertainty. Moreover, since te investment in patces is made (or committed to) before te user buys te products, our model is valid even if te number of bugs is stocastic. It is important to ask weter tere is someting special about software tat may lead to systematic market failure. Tere are indeed important differences between software and typical pysical products. Te ability to fix or oterwise mitigate software defects after release is an under-appreciated difference and is te focus ere. Pysical goods can of course be recalled to fix defects. However, te key distinction is tat wile te cost of fixing a software bug ex post is rougly independent of te number of copies of te software already sold, recalling and fixing an auto model to fix a uality problem can be very expensive, and more pertinently, will vary directly wit te number of cars of tat model tat ave been sold. In contrast, software vendors systematically rely on being able to fix bugs later. Altoug te cost of actually identifying a bug and writing and testing te code to fix it can be significant, it often as a fixed-cost nature in te sense of being independent of te number of 5

7 copies sold, particularly wen patces can be distributed via te Internet. Of course, sometimes patcing costs may vary by number of users (copies sold). For instance, if te software vendor sent a consultant to eac user s site to elp install te patc, ten te marginal cost per user would also be non-negligible like remediation of YK problem reuired programmers to go troug eac organization s systems to fix code. It may also be true for some custom-developed or very complex software. However, a more common practice is to te patc to customers or simply make it available on te vendor s website wit no customer specific adaptation or and-olding. So, to caricature, in our model, te software aftermarket repairs entail only fixed costs; for pysical goods, tere are bot fixed and variable costs. Te fixed cost nature of patcing investment implies tat te effective market size plays a crucial role. Indeed, we formally sow tat all else eual, increase in market size implies earlier sipping (longer time in market ) but also more investment in patcing (after-sale suppor, and iger overall value to te user. Tis result provides intuition for te oterwise unexpected result tat, relative to te socially efficient outcome, a monopolist sips software products later, wit fewer bugs, but commits to less patcing later. In contrast, we sow tat for pysical products market size is irrelevant; bot te monopolist and social planner provide te same levels of defects and patc post sale remediation. We sow tat tese results arise due to interaction between differences in market coverage, te fixed cost nature of patcing, and strategic complementarity between te patcing tecnology and te time in market. We extend te model to duopoly competition. First, we sow tat competition forces bot firms to differentiate teir products and offer different times of entry and levels of patc support. Interestingly, we find tat wen patcing is a viable alternative, competition forces te ig value firm to enter te market early wit buggier products. But it provides iger value by 6

8 investing eavily in ex-post remediation. Tis incentive increases wit market size. On te oter and, a low value firm tends to provide little or no after sales support. Eventually, if patcing is viable for bot firms, bot will enter a larger market earlier. Te paper is organized as follows. We provide a literature review in section. We outline te model, including te monopolist s and social planner s decisions, in section 3. In section 4, we provide te comparative analysis wit a tangible good. We extend our model to incorporate competition in section 5. Finally, we present managerial insigts and concluding remarks in section 6.. Prior Literature Muc work on software uality and reliability as been done from a computer science or software engineering point of view. Te focus as been on process models of software development (e.g., Buckly 1984, CMM; Capability Maturity Model). It as been argued tat better process leads to iger uality and lower costs and maintenance (Krisnan et al. 000; Banker, Davis and Slaugter 1998). In fact, CMM as become a de-facto industry standard to improve te processes tat lead to ig uality software production. Tere is also unanimity tat catcing a bug late in te software development cycle is costly and tat maintenance activities are a significant part of total cost over te software life cycle (Artur, 1988). Firms are also investing in new testing and uality assurance programs to improve te uality of teir software (Buckley 1984; Abdel-Hamid 1988). However, as noted in te introduction, our focus is not on process economies; we assume tat firms are using te best available practices to produce teir product. We are more interested ere in analyzing te trade-off between delaying product release to reduce te bugs and entering te market early. 7

9 Coen et al (1996) analyze te problem of performance and time-to-market tradeoff for pysical products wen repairing defects after product release is exorbitantly costly. Moreover, unlike tis paper, tey do not analyze te impact of competition nor provide comparisons to te social optimum. Te literature sows tat iger performance provides iger value to customers (Zirger and Maidiui 1990; Dyson 1991), but it also leads to significant development delay (Griffin 199; Yoon and Lilien 1985). Coen and Wang (1997) provide a model for products and after-sale support. But in teir model, after-sale service is independently sold to users, and terefore, an independent source of profit for te vendor. Our analysis differs from muc of te work to date in tat we allow te vendors to issue patces after te product as been released, as te empirical evidence overwelmingly sows is done for software (Viega and McGraw 001). In marketing parlance firms adopt a penetrate-and-patc strategy. Moreover, patcing as become so common tat many organizations ave patc-management systems in place to efficiently integrate patces in existing software (InterWorks 000). 5 We also draw from product differentiation models of economics. We build on te models of vertical uality differentiation, (e.g., Mussa and Rosen 1978; Saked and Sutton 1983; Moorty 1983; Ronnen 1991) altoug unlike tese models, overall uality (or value, as we call i depends on te interplay between time of product release (wic determines te number of bugs) and patcing intensity of te producer. 3. Model 3.1 Consumer s Utility All else eual users prefer software products tat are released earlier and ave fewer defects. In oter words, a product will become obsolete on a given date, ten sooner te 5 See for recent papers on patcing and patc management. 8

10 customers get te product, te longer tey can use it and te more utility tey get. We define user utility as follows: U = θ [ V k( B( ] t p (1) V is te baseline utility per unit of time if te product is bug free, B( is te number of bugs wen te product is released, and t is te amount of time a user uses te product, were for simplicity we ignore time discounting. Time t is measured backwards, so a ig t means te product is released early giving te user more time to derive utility from te product wile lower t means tat te product is released late. Note tat te number of bugs, B(, is modeled as being driven by te speed wit wic te product is released to te market. In particular, rusing to release results in more bugs, and at an increasing rate, so we assume tat B( is increasing and convex in t, i.e., B > 0 and B > 0. Te price paid by te customer is p, and te coefficient θ determines ow muc utility a customer derives per unit time from te software s functionality. Higer θ users derive more utility from te product, but tey also suffer iger disutility due to failures. Tus, freuent users derive more utility but also incur iger loss if it fails. For analytical tractability, we assume tat θ is uniformly distributed between [0, 1]. A key parameter of te model is δ wic captures te firm s investment in after sales patcing support, and k( is te proportion of defects costs (to te user) reduced by te firm troug remediation of one sort or anoter after te product is released 6. Since k ( < 0 and k ( > 0, te utility loss decreases as patcing investment increases, albeit at a diminising 6 One easy interpretation of k( is tat k( = [1 - g(]*d were D is te disutility per bug and g( is te proportion of te potential disutility due to bugs tat is ameliorated by te investment in patcing d. Naturally, g( will be increasing and concave. Terefore, k( is decreasing and convex or k ( <0 and k ( > 0. 9

11 rate. Te proportion of defects costs reduced, k(, ougt to be interpreted as an average over time or in an expectation sense. For some products, a proportion of defects migt be fixed immediately via a patc available for download sortly after te purcase. For some, k( could be interpreted as te proportion of te cost of te defect defrayed by good elp desk support. For oters, te proportion of defects remediated migt initially be zero, but migt increase over time so tat on average, over te time period t during wic te product is consumed, tat portion of defects costs remediated is k(. One can possibly tink of at least tree distinct types of costs of defects to users: (i) Ongoing degradation in performance because te patc is an imperfect remedy relative to te product aving been designed rigt in te first place, 7 (ii) Te cost of failures tat occur between te time te defect is discovered and wen it is fixed or patced, and (iii) Te cost to te user of installing te patc (e.g., te cost of redoing systems integration testing wen one software component is updated or patced). So our model focuses on first category and te second category if te time between defect detection and repair looks more like a proportion of te total time te product is used tan a fixed constant. Te tird category could be accommodated by adding a negative fixed cost per patc. 3. Vendor s Profit Function A general formulation of te software vendor s objective function is π = Q ( p fδb( ) G( δ, B(, () 7 One example would be Microsoft Outlook s vulnerability to viruses. Te basic problem is built in to te arcitecture and cannot be fixed even now tat it is recognized. Hence, users need to freuently update teir virus protection software. Microsoft s aving a large team of people wo make virus updates available very uickly would be like aving a large δ. Anoter example would be if te patc just lets te system cras more gracefully, but does not reduce te freuency of crases. Te cost of te bug is partially defrayed (k( < 1), but te bug continues to reduce te utility provided by te software even after it is detected and patced. 10

12 were Q is te volume sold, p is te price, and te oter terms represent costs of patcing. As we noted in te introduction tat firms can only reduce te bugs via delay, we can assume witout te loss of generality tat initial product development costs are fixed costs and do not affect te firm s optimization problem. In general, patces can involve bot fixed and variable costs to te vendor. Te constant f represents te per unit variable cost per defect remediated. Te second term in (), G(δ, B(, is te fixed (relative to volume of sales, Q) cost of remediating defects. Tis term would include, for example, te cost of doing te researc necessary to figure out ow to remediate a given defect. For an auto manufacturer, tis would be te cost of designing a substitute for a defective part. For a software vendor, it could be te cost of writing te code for a patc. Note tat tese costs vary in tat vendors may coose to patc defects more promptly, reuiring greater investment up front. 8 Tey may also vary wit te number of defects. Wat matters is tat tey do not vary by te number of users, Q. One focus of te analysis will be contrasting tangible products, for wic f is large, and software products for wic f is smaller, if not negligible. To make te contrast between tangible and software products as stark as possible, and for analytical convenience, for te tangible products case (f > 0) we will assume tat te fixed costs of defect remediation are negligible (G() = 0) and for software products tat f is literally zero and te fixed costs are significant. In reality, even downloading patces from te vendor s web site imposes some (trivial) marginal cost on te vendor in te form of server utilization. But te polar cases are stylized renderings of te typical conditions for manufacturing and software development. 8 In tis sense, investments in patcing are an instance of an endogenous fixed cost. Sutton (1984) provides a more general analysis and implications for market structure. 11

13 At tis point, to continue te analysis, it is necessary to get more specific about ow G(.) depends on its arguments. One vision of defect remediation is tat for every defect a solution must be invented. Tus, patc development costs are proportional to te number of patces. 9 Terefore, we coose te cost function G(.) = FB(, wit increasing and convex in δ. One can see tat G(.) is increasing in δ and t, at an increasing rate. Our results are robust to oter plausible formulations Market Euilibrium under Monopoly Given Euation (1) for te users utility and te uniform distribution of θ, demand for te software product facing a monopolist vendor is p D( p, t, = 1. [ V k( ] t Hence te profit Euation () for te monopolist profit becomes p π ( p, t, = 1- p FB(. (3a) [ V k( ] t It can be readily verified tat after substituting for te profit maximizing price, 1 π ( t, = [ V k( ] t FB(. (3b) 4 9 Many large firms respond to te bugs in teir products by releasing appropriate patces. For example, CISCO specifically responds to all te bugs discovered (ttp:// 10 Alternate models would view te vendor s commitment in terms of effort budgeted for responding to defects: te vendor may assign γ programmers to develop patces. Hence, te cost to te vendor is F γ t, were F is te cost per programmer per unit time. Te average proportion over time of defect costs remediated is presumably an increasing, concave function of γ, i.e., δ = f(γ) wit f > 0 and f < 0. So, expressing te model in terms of δ, te cost function becomes F c( t, were c > 0 and c > 0. Alternately, te budget for developing patces could be set simply in terms of te total programming effort. In tat case, te cost to te vendor would be just F γ, were F is te cost per programmer-year. Our ualitative results seem robust wit respect to tese variations in te cost function, but oter variations may prove fodder for extensions of tis paper s analysis. 1

14 It is useful at tis stage to introduce Ψ(δ, t; m) = m[ V k( B( ] t FB(. We assume tat Ψ(δ, t; m) is concave in t and δ. Tis assumption is reuired for an interior maximum. Concavity is ard to sow for te general functional forms we used, but is readily satisfied for various specific common functions. (See Appendix.) Te first order conditions for t and δ are dπ ( t, dt 1 = [ V k( B'( t k( ] FB'( = 0 4 (4) dπ ( t, dδ 1 = k'( t FB( C'( = 0 4. (5) For future reference, note tat for δ satisfying te first order conditions d π ( t, dδ dt 1 = k'( ( B'( t + B( ) FB'( C'( > 0 4 (6) 1 * * * since k '( t = FC'( from e (5). Tus δ and t are strategic complements (Milgrom 4 and Roberts, 1990), implying tat factors tat increase te marginal payoff from δ will also result in a iger value of optimal t. In oter words wen investing more in patcing is desirable, te vendor releases te product earlier. Figure-1 below illustrates te intuition beind tis result and igligts te generality of te cost function we ave used. Note tat 1 V k( ] t 4 [, te total revenue of te monopolist, is a concave function of t for a given δ. Te costs are convex in t. A monopolist will coose an optimal pair (t, wic maximizes te vertical distance between te value and cost curves. As can be seen, te monopolist s optimal time in market is given by t 1 for δ = δ 1. A iger coice 13

15 of δ, δ > δ 1, implies tat te revenue curve sifts up since patcing is beneficial for users, for any given t. Cost = F B(δ ) Revenue, Cost R 1 R 1 Re venue = [ V k ( δ ) B( ] t 4 Re venue = 1 [ V k ( ] t 1 4 Cost=F B(δ 1 ) t 1 t Time in market ( Fig-1: Complementarity Between t and δ Te key to our results is tat a iger δ also increases te marginal payoff of t, so tat te revenue curve for δ as a iger slope (algebraically) for all values of t. Moreover, altoug increasing δ also increases cost, te increase in marginal cost is not as great as te increase in marginal revenue. As a result, te optimal t * increases wit δ. Tis figure also provides insigt into te implications of different forms of a cost function. For instance, if te cost function were additively separable in B( and, ten canges in δ would imply a parallel sift in te cost curve, leaving te marginal cost uncanged. It follows tat tis would leave te complementarity between t and δ intact, and our results 14

16 would be true a fortiori. Similarly, if costs were independent of B(, ten te marginal cost wit respect to t would be zero so tat te cost curve would be parallel to te orizontal axis. In tis case, te optimal t is simply given by te point at wic te value curve attains its maximum. Again, it is straigtforward to see tat t and δ are complementary, so tat increases in δ are accompanied by increases in t. Tus, te multiplicative cost function we ave used is uite general and our ualitative results robust to oter plausible formulations. 3.4 Bencmark: Te Socially Efficient Outcome It is interesting to compare te coices of te monopolist in time of entry and patcing support to teir socially efficient levels. Efficiency reuires tat te product is priced at marginal cost 11, wic we ave assumed ere to be zero. At tis price, te entire market will be covered. Terefore, te objective function to be maximized is S( t, = θ [ V k( ] t dθ FB( = [ V k( ] t FB( (7) 0 Note tat euation (7) and te monopolist s profit function ave te form Ψ(δ, t; m), were m = ½ for te social planner and m = ¼ for te monopolist. We prove in te Appendix tat te optimal values δ * and t * tat maximize Ψ(δ, t; m) are increasing in m. Tis implies tat te socially efficient outcome is to release te product earlier tan te monopolist (and ence wit more bugs) but patc more aggressively, as summarized in Proposition 1 below. Proposition 1: For a software product, te monopolist releases te product later wit fewer bugs but invests less in patcing and ex-post support tan te socially efficient levels. 11 Te result olds even if we impose a break even constraint, since, as we explain, te key is tat te monopolist serves fewer customers tan is socially efficient. 15

17 Te result is surprising in tat contrary to popular belief, monopolist actually releases less buggy products tan wat is socially optimal. But monopolist provides less value by bot entering te market later and by providing less ex-post support. 3.5 Role of Market Size In our analysis, te market size is normalized to one and m captures te proportion of market captured by te monopolist or social planner. Sould te market be larger, ten for te same m, tere will be more users buying te product. Tus, we can also interpret m as an indicator of market size. It follows from te proof of Proposition 1 tat a producer facing a larger market as a greater incentive to enter te market earlier wit a buggy product and provide extensive patcing support later. Te costs of patcing can be amortized over a larger sales volume, inducing greater investment in δ. Since δ and t are strategic complements, greater investment in δ is associated wit earlier (and ence buggier) product release (i.e., a larger. Tis result is also clear from Figure 1. An increase in market size sifts te value curve up and to te rigt for a given δ. Tus, all else constant, a monopolist wit a larger market would coose iger levels of δ and t. By parallel reasoning, te smaller te fixed cost per bug patced (smaller F), te greater are δ and t. Tus a corollary of Proposition 1 is: Corollary 1: A software producer facing a larger market or lower fixed costs per bug patced enters te market earlier wit more bugs but provides iger patcing support later. Tus, our model provides a clear and testable prediction, particularly if te number of patces is interpreted as te freuency wit wic newer versions are released by te vendor. 1 Our model implies tat, in a large market, one sould see freuent versions being released by te 1 Generally newer versions ave more features as well. In our model we ignore features. Similar to Jackson (00), in a non-traditional definition, lack of features can be interpreted as a bug. 16

18 vendor. Te costs of patcing also depend on te type of product and domain in wic firms operate, suc as systems software, and application software. Our results suggest tat wen te cost of patcing is small or wen te vendors ave better support infrastructure, it as an incentive to enter earlier wit buggier products. Microsoft typically as large market for its products, and also provides extensive patcing support. Many of its patces are released in service-packs tat make it easy for te user to download and install new patces. Te users (or network administrators) do not ave to worry about investing time and energy in looking for te rigt patces. Moreover, tese service packs can also scan te computer and networks, identify bugs and vulnerabilities and automatically download te rigt patces. Clearly, Microsoft is willing to incur tese large costs in after-sale support to reduce customer disutility. In oter words, Microsoft is investing more in δ and reducing k( for a user. 4. Quality in tangible products We now compare tis result wit a tangible product wose producer incurs a positive marginal cost of fixing a defect. We follow te same model structure except tat we replace te fixed cost per patc wit a cost tat varies wit te number of units fixed, as well as wit te number of bugs. Terefore, te profit function for a tangible product monopolist is p π ( p, t, δ [ V k( ] t = 1 [ p fb( ) ] Here fb( is te marginal cost of fixing a defect 13. Substituting for te profit maximizing price leads to te profit function: 13 We could add a marginal cost of production c as well, but witout loss of generality and to focus on te economics of patcing assume c = 0. 17

19 [( V k( ) t fb( ] π ( t, =. (8) 4[ V k( ] t As always, te socially efficient outcome would involve pricing te product at marginal cost. Terefore, p = fb(. Hence te social surplus can be written as [( V k( ) t fb( ] S(t, = (9) [ V k( ] t Since te monopolist s and te social planner s objective functions differ by only a multiplicative constant, tey will make te same coices wit respect to δ and t. Proposition : Wen te cost of fixing a failure ex post is a marginal cost (as in te case of a tangible good), te monopolist s coices of time to release te product and investment in patcing bugs later are socially efficient. Wy is tere a difference between tangible and intangible goods? Tere are tree important factors. First, te monopolist serves a smaller market tan te social planner. Second, wen patcing is a fixed cost, te smaller market leads to a smaller investment in patcing. Tird, patcing is a strategic complement to t. All else constant, te iger δ is, te iger is t. Tus, in software, te smaller market coverage of te monopolist leads to a smaller δ and terefore also a smaller t. Te importance of te strategic complementarity can also be seen by assuming tat te patcing tecnology is not available i.e., by setting δ = 0, in te previous section s model wit 1 fixed costs of patcing. In tis case, te monopolist s profit function π t,0) = [ V k(0) B( ] t, ( 4 differs by only a multiplicative constant from te objective function for social optimality 1 S t,0) = [ V k(0) B( ] t, wic leading to te following proposition. ( 18

20 Proposition 3: Wen ex post patcing of bugs is ruled out, a monopolist software vendor releases te product wit te same ex-ante uality even in markets of different sizes, and it is te socially optimal uality. Clearly, te fixed cost caracter of patcing software defects after product release can create a market failure (difference between te market euilibrium and te socially optimal outcome) tat does not exist for conventional pysical products, at least in te case of a monopolist. We next sow tat tis and related findings are not restricted to te case of monopolist providers. An alternative, but not mutually exclusive, interpretation of te difference between software and tangible goods as to do wit differences in te legal environment. In many tangible goods, a product defect exposes te manufacturer to liability claims, and in general, reuires restitution to te user. Typically, software vendors are not exposed to suc claims. Liability can be interpreted as anoter way of compensating users for defects (i.e., of reducing disutility from bugs) after product release. However, a key difference wit patcing is tat liability costs vary directly wit te number of users. On te oter and liability for software developers is still different from tangible goods producers because it can be reduced after product release via interventions tat ave a fixed but no variable cost, namely greater investment in patcing. A priory, it is ard to say weter exposing software developers to product liability would cause tem to improve te uality of products at te time of teir release. Te greater marginal cost per defect and per copy sold would create an incentive to delay product release in order to improve initial uality. It would also tend to reduce sales volume and, ence, te customer base over wic fixed investments in patcing could be amortized. On te oter and, 19

21 it would increase te incentive for patcing, and patcing is a strategic complement wit time in market. Wic effects would dominate could depend on te details of ow patcing was viewed as reducing liability exposure, wic in turn depends on te details of ow one models bugs as arming users. Tis topic could be an interesting uestion for furter researc. 5. Competition 5.1 Preliminaries Te monopoly case is not uninteresting for software (consider Microsoft s market sare in PC operating systems), but understanding te impact of competition on te timing of product release and investments in patcing tecnology is of obvious interest as well. Here we study te case of two firms offering functionally identical software products. It is clear tat te firms would like to differentiate temselves to avoid intense price competition tat would oterwise erode profits. In practice, firms can and do differentiate on multiple attributes. For example, tey can differentiate via different marketing efforts or by offering different product features. In our model, we allow firms to differentiate teir products only wit respect to te variables of interest ere: product release date (ex-ante uality) and investment in patcing. Te firm strategy space can be considered in two stages. In te first stage, firms announce te date of release of teir product and commit to a given level of investment in patcing. At te second stage, tey set prices and enter te market, wereupon consumers make te rational coices given teir preferences. We study te sub-game perfect euilibrium. Witout loss of generality we label te firm offering iger value to customers as H [Hig value firm] and denote its coices by t, δ, and p. We denote te firm offering lower value to 0

22 customers as L [Low Value firm], denoting its coices by t l, δ l, and p l, so tat te utility offered by te j type firm (j {H, L}) to users is j j j j p t t B k V U = )] ( ) ( [ δ θ To derive te demand function for bot firms, we presume customers cose optimally by buying from H, L, or neiter. For convenience, let denote te total value (gross of price) offered to te customer, so L H j t t B k V j j j j,, )] ( ) ( [ = = δ (10) Value, along wit price, is sufficient for determining customer s coices and so firm differentiation can be defined first in terms of tat single dimension. Te users will buy from H firm wen l l p p θ θ and buy from L as long as 0 > l l p θ. Simple algebra gives te optimal prices carged by bot firms (Tirole 1998; Ronnen 1991): 1 4 1) ( and 1 4 1) ( * * = = r r p r r p l l were l r =. Te revenue functions of bot firms can be written as ) (4 ) ( 4 l l H R = and ) (4 ) ( l l l L R =. Teir profit euations are ) ( ) ( ) (4 ) ( 4 l l H C t FB δ π = (11a) ) ( ) ( ) (4 ) ( l l l l l L C t FB δ π = (11b) 1

23 Wile solutions for optimal and l are well establised, solving for optimal t and δ is non-trivial. Terefore, before we establis te euilibrium, we first establis te relationsip between value offered and te optimal t and δ. In oter words, if a firm wants to increase te value, ow will it cange t and δ optimally to acieve tat? We sow tat t and δ are (weakly) increasing in te level of value cosen. First suppose tat te desired value is low enoug to be obtained wit δ = 0, i.e., [ V k(0) B(ˆ)] t tˆ = ˆ were t ˆ maximizes te value given δ = 0. (t ˆ is well-defined since is concave in t.) Assuming te cost of patcing is zero wen no patcing is done (i.e., 0) = 0), for some fixed ˆ, te objective function is independent of δ and minimized wit respect to t. Hence, if te vendor wises to offer a value attainable wit δ = 0, it will do so by manipulating t alone. In particular, since (t; δ = 0) is increasing in t for all t tˆ, te vendor can increase by increasing t. Any increase in uality beyond ˆ will be accompanied by investments in patcing. But investments in patcing will affect te time to release as well. Terefore, if te firm wants to offer some value =, ten it does so in te least costly way. In oter words, firms are effectively solving te following problem: Min F B( suc tat ( V k( ) t = Te Lagrangian function can be formulated as [( V k( t ) t ] Max L( t, δ, λ) = FB( λ ) Te first order conditions for an interior optimum are

24 L = FB'( λ( V k( ) + λk( B'( t = 0 t L = FB( C'( + λk'( t = 0 δ L = ( V k( ) t + = 0 λ We are particularly interested in te sign of δ t and. Lemma : At te optimal t and δ, δ t and > 0. Proof: See Appendix. In oter words, to increase value, te firm increases its investment δ in patcing wic, in turn, leads to early release of te product. Te firm offers a pair (t 1 *, δ 1 *) wen it wants to offer a value 1. If te firm needs to offer anoter value > 1, ten Lemma 1 sows tat it can do so most optimally by investing more in patcing but releasing te product earlier and terefore offering a pair (t *, δ *). 5. Euilibrium Analysis We sow tat te cost of patcing in competition plays a crucial role in te patcing support and release date decisions. We first caracterize te euilibrium of tis game. As is standard in te literature on vertically differentiated products (e.g., Saked and Sutton, 1984; * Ronnen, 1991) we focus on euilibria suc tat te two firms offer different uality levels: by te ig-value firm (H) and * l by te low-value firm (L) suc tat * > * l. From Ronnen (1991), we know tat tere exists an euilibrium suc tat l is non-decreasing in and viceversa Ronnen sows tat an euilibrium exists as long as cost of uality Φ () is convex, as is te case in our model. 3

25 First note tat bot l and are strictly positive. Oterwise, no customer would purcase tat firm s product. Next it is easy to sow tat firm H always enters te market before te monopolist. Substituting l = 0 in Euation 11(a) implies tat wen firm L offers zero uality, firms H s objective function is identical to te monopolist s objective function (Euation 3). Intuitively, wen a firm s only competitor is offering zero value, te firm is effectively a monopolist. Hence, wen l = 0, firm H offers value eual to tat provided by te monopolist. But we know tat in euilibrium te lower value firm offers l > 0 and tat increases in l. Hence firm H always offers a iger value tan te monopolist. From Lemma 1, we know tat δ and t are increasing in. It terefore follows tat firm H enters earlier tan it would absent competition from te low value firm and also invests more in patcing. Tus, we conclude tat, for te ig value firm, competition sortens time to market and increases te number of bugs, but it also results in greater post sales support and patcing, and overall, greater value to te consumer. We formalize tis intuition in te following proposition. Proposition 4: In te presence of competition, te ig value firm enters te market earlier tan would a monopolist. Figure displays ow uality and product release times of te various firms vary wit te cost of patcing, captured ere by variations in F. On te x-axis is te cost of patcing, F. On upper y-axis is te value provided. On te lower y-axis is te time t wen tese firms release teir products. It is easy to interpret te figure wen starting from te rigt most side of te x- axis. Wen te cost of patcing is greater tan some tresold level, Fˆ, no firm invests in patcing. Firm H (broken curve) enters at time tˆ, and invests δ = 0 in patcing support and offers te value ˆ. Following Proposition 3, te monopolist (solid curve) does te same. But 4

26 Bot Firms Patc Firm H Patces No Firm Patces m Value l F F c Fˆ ( t = tˆ, δ = 0, = ˆ ) Late Entry t m t l t t Early Entry Cost of Patcing F Fig : Patcing Cost (, Time in market (, and Value () firm L (ligt broken curve) differentiates by entering te market early and carging low prices. Clearly, firm L does not invest in patcing eiter. Note tat it offers a lower value tan ˆ. Moving to te left, as te cost of patcing decreases, firm H invests in patcing and enters te market earlier tan te monopolist, as noted in Proposition 4. In response, firm L increases its value by entering later (but initially not patcing). Eventually, wen cost of patcing falls to F c, it is possible te firm H enters earlier tan firm L but invests significantly more in after-sale support. For low enoug costs of patcing F, bot firms invest in after-sale 5

27 patcing support. As te figure also sows, regardless of ow t and δ are combined, as te cost of patcing falls, value provided increases. Te cost of patcing will depend on factors suc as te application domain and firm capabilities. For critical applications, patcing may not even be an option and te cost of patcing can be taken as proibitive. In suc cases, firm H will enter later tan firm L and bot firm H and a monopolist would produce efficient outcomes. Similarly, if te firm as te infrastructure and capabilities in ex-post support ten it will produce a lower uality product and enter te market earlier, but provide better after sale support. 5.3 Te Role of Market Size: A larger market leads to more revenues for bot firms witout any canges in teir cost structure. Terefore, in euilibrium bot firms want to provide iger value. In oter words, we can re-interpret Figure, by sifting te rigt vertical axis leftwards. Again, from Figure, it is clear tat, for all else constant, a larger market will invite an earlier product release in te presence of competition. Tis result provides support to wat is usually observed in te marketplace; i.e. competition forces a firm to enter te market earlier and patc more wen te market is more attractive. Interestingly, it is firm H tat as stronger incentives to enter early compared to firm L. But again, for low enoug cost of patcing, iger market size will lead to early entry by bot firms. (See Figure wen cost is less tan F.) Wile tere are many reasons firms tend to enter early, including first mover advantage and switcing costs, our results point to anoter dynamic in tis industry. Te ability to remediate defects after releasing te product and te fixed nature of tese costs provide incentives to enter te market earlier, and te larger te market, te stronger tese incentives. 6

28 6. Conclusions and Future Researc Te recent focus on software uality and vulnerability makes it important to understand te incentives of software vendors to provide uality. We provide a simple economic model of te penomenon of issuing patces and fixing software bugs by software vendors and its impact on ex-ante uality and total value provided. We analyze te firm s trade-off in time-to-release te product vs. investments in ex-post remediation of software failures. We first sow tat tere is strategic complementarity between time to enter te market and patcing investment. Terefore, as firms invest in patcing, tey tend to enter te market earlier wit buggier product. We also sow tat te fixed cost nature of investments in bug-remediation leads firms to enter a larger market early wit a buggier product but also invest more in after-sale remediation. Since a monopolist restricts output relative to socially efficient levels, tis leads to counter - intuitive result tat a monopolist actually provides a product wit fewer bugs but ten provides less after-sale support and terefore, reduces customer value. We compare tis result wit a traditional tangible good case and sow tat wen tere are significant marginal costs of remediation, market size does not play any role, and tus, a monopolist s time-of-release strategy is also socially efficient. Consistent wit our model, we note tat tat tere is widespread dissatisfaction wit, for instance, te many security vulnerabilities in Microsoft s products, even wen compared wit products from rivals wit smaller market sare. But few can matc te extensive system of updates and patces tat Microsoft provides. We extend our model to incorporate competition involving two firms tat endogenously differentiate by providing different levels of total uality or value. We sow tat sometimes bot firms enter te market earlier tan would a monopolist. But a ig uality firm compensates for early entry and a buggy product wit aggressive support for ex-post remediation. In oter 7

29 words, te firm provides iger after-sale support to its users. Tis result parallels te penetrate-and-patc strategy typically observed in tis industry were te firms want to enter te market early to capture market sare but te ig value firm is willing to invest in te infrastructure to provide after-sale support. Te time of entry also depends on ow expensive it is for a firm to fix te bug. Te easier it is to patc, te greater te incentives to release a buggier product early and patc it later. Inasmuc as te Internet reduces te cost of issuing patces, it migt in tis way be a contributor to te bugginess of software products. Tus competition elps, but it still does not lead to te socially optimal levels of time of product release, reliability or bugginess, or after-market support. Furtermore, suc a market failure in tis model is uniue to products, suc as software, for wic te marginal patcing costs are zero wit respect to number of users. Te same model parameterized for pure pysical products ad no suc market failure. Te uestion of weter and ow te government migt intervene to affect software uality is extremely delicate and certainly ougt to encompass many considerations besides te zero marginal cost of software defect remediation. However, tis paper is certainly germane to tat larger discussion. Even witin te domain of models focusing on software reliability and after market remediation, tere are opportunities for extensions and furter work. One could consider models of competition oter tan duopoly; and models in wic customers are eterogeneous not just on a single dimension (θ ) governing ow valuable te product is to tem, but also independently wit respect to ow costly bugs are. A furter extension would allow firms to release at multiple dates, wit later versions being more reliable and fewer bugs, and for customers to optimally decide on wic version to buy. 8

30 Anoter extension would consider patcing as a dynamic coice variable, not one to wic firms can commit. If outcomes are worse witout commitment, tat could be an argument for supporting organizations like CERT, wic acts as a security watc dog, and consumers consortia like te bugtra mailing list, tat are strong enoug to force even a firm like CISCO to respond to bug ueries promptly. Appendix Proof of Concavity of profit function in t and δ Consider te profit function π ( t, = m[ V k( ] t FB(. Te first order conditions for t and δ are dπ ( t, = m[ V k( B'( t k( ] FB'( = 0 dt dπ ( t, = mk'( t FB( C'( = 0 dδ Or, mk'( t = FC'(. Te second order partial derivatives are dπ dt ( t, = mk( [B' ( B'' ( t] FB'' ( < 0 dπ ( t, = mk' '( t FB( C''( < 0 dδ dπ ( t, = mk'( B'( t mk'( FB'( C'( = mk'( dtdδ To sow tat profit function is concave at any optimal t and δ, we need to sow tat H(t, = dπ ( t, dt dπ ( t, dtdδ > 0 dπ ( t, dtdδ dπ ( t, dδ 9

31 Te assumptions reuired to ensure H(t, > 0 do not ave direct economic interpretation. Te following example sows, owever, tat te second condition olds for some simple functional forms. Suppose B ( = t, K( = (1, t, = Ft δ. Ten te profit function is Π( t, = m[ V (1 t ] t Ft δ. Taking te first derivative π t = m[ V (1 t ] m(1 t Ftδ = 0 and π δ = m t Ft δ = 0 Similarly, te second derivatives are 3 π = 6m(1 δ t and tt ) = Ft π δδ Using te property m t Hessian is 3 Ft δ = 0 and simplifying leads to π t δ 4 = Ftδ. Terefore te 6m(1 t 4Ftδ 3 H ( t, = = 1 Fmt ( 1 16F t δ. 4Ftδ - Ft 3 Again using te property m t Ft δ = 0, H(t, = 4F t δ (1 16F t δ, so tat H(t, 0 if and only if 10δ 6. For F large enoug relative to m, we can ensure tat te optimal δ will always satisfy tis condition. Proof of Proposition 1 Consider te objective function Ψ( t, δ; m) = m[ V k( B( ] t FB( Te first order conditions for t and δ are Ψ = m[ V k( B'( t k( B( ] FB'( = 0 Ψ t = mk'( B( t FB( C'( = 0 mk'( t δ Or, = Taking te total derivative of bot euations [ mk ( ( B' ( B'' ( FB'' ( ] dt + [ mk ( ] dδ = [ mk '( ]dt + [ δ δ ] δ By Crammer Rule FC'( ' [ FB '( ]dm mk ''( ) B( t FB( C''( ) d = [ k '( t]dm. 30