Building the Invisible Palace: Capability gaps, traps and overlaps in complex software projects
Flowers, Stephen (2002) Building the Invisible Palace: Capability gaps, traps and overlaps in complex software projects In: EURAM 2002, 9-11 May 2002, Stockholm, Sweden.
The growth of the IT industry has been dramatic and information and communication technologies (ICTs) now play a fundamental part in many, if not all, of the critical infrastructures upon which modern society depends. The use of ICTs within organisations of all types is approaching ubiquity with many such systems being viewed as mission critical. This spread and the increasing criticality of such systems itself relies upon the ability to develop complex software systems in a reliable and predictable fashion. However, there is widespread evidence that the development of software in such a fashion continues to present a formidable challenge to many organisations with software typically being delivered late, over budget and with reduced functionality. Whilst large software projects have always posed huge challenges for suppliers of bespoke software, the nature and scale of that challenge has evolved over the years. At the level of the individual firm, modern information systems development takes place in the context of a series of interconnected complexities concerning alignment with organisational goals, primary and secondary sourcing, integration of new systems and backward integration with existing systems. At the level of the industry, these complexities are in turn managed in the context in which specialist capability has largely migrated to the IT services industry, necessitating the involvement of a range of intermediaries. Such work takes place within the wider context of a range of highly flexible technologies with high rates of technical and skill obsolescence. Within this turbulent situation data across a range of performance metrics may be difficult or impossible to obtain with the result that organisations will often operate in the absence of reliable information. This exploratory paper will provide a high-level examination of the major challenges associated with the development of large-scale complex software systems and trace the emergence of the current situation facing many organisations. In order to provide a context for the discussion, the paper will examine the recent growth and development of the software and computer services industry and draw on earlier work that assesses its maturity (Flowers and Brady, 2001). The development of complex sourcing for IT (Venkatraman, 1997) and the rapid growth and expanding role of intermediaries within the software industry will also be examined. The nature and implications of the migration of specialist capability within the IT and software industry will be explored and the paper will build on earlier work in the area (Feeny and Willcocks, 1998) to argue that there has been the emergence of a series of capability gaps, traps and overlaps that provide a further layer of sourcing complexity to what is already a hugely complex procurement and development context. The issues and implications of this situation will be examined and it will be argued that the emergence of significant capability imbalances against the background of poor data provide the context for the continued uneven performance of complex software projects. The paper will conclude with an examination of the implications of this situation to the development of complex software projects.
Repository Staff Only: item control page