Software and innovation

Software Innovation can be understood in two ways: 1. The creation of novel and useful software programs – product innovation 2. The introduction of novel and useful ways of developing software – process innovation Innovation should be distinguished from invention, and from creativity. Both are relevant to software innovation, but whereas creativity is the state of mind which leads to innovative thinking, and invention could describe a new algorithm or program (or software development technique), innovation implies that the creative act and invention are carried into wider use, leading to substantial kinds of change; thus the successful exploitation of new ideas (Department of Trade and Industry, UK). Software innovation therefore = invention + exploitation + diffusion, where invention refers to the creative act or process, exploitation refers to its enactment in practice, and diffusion refers to its adoption by a wider audience. Software innovation may refer to both radical (disruptive, discontinuous) and incremental changes to software products and processes. Typical the result of software innovation is experienced as change – in the way people work, business is carried out, in people’s choice of entertainment, in the way they choose to communicate how they govern their communities or interact with each other.

There are different kinds of underlying models for software innovation, which are derived from studies in technology innovation. In a linear model, software product innovation is understood as a sequence of stages or phases, rather like the waterfall model of software development. The initial creative idea (invention) is followed by phases of idea development, prototyping, full-scale development, production, release and sale. Here the focus is much on the software product, which is conceptualised at the beginning of the process and developed to be a marketable software system. In an iterative model, software innovation is considered in a more process–oriented way (reflecting agile system development methods). Innovation takes place through highly focused and creative bursts of development activity, which are repetitive but evolving. In this model it is not necessary to have a fully-formed software concept at the beginning of development – the creativity and energy of the process itself offers the conditions for innovative programming and development. In the network (or community) model the focus is upon the conjunction of people, ideas and expertise. In Silicon Valley, or in the Linux community, are found clusters of highly able software developers, partly competing with each other, and partly co-operating to be at the leading edge of technology development. Important in this model are teams, co-operative work, expertise sharing, timing (in relation to other innovations) and pushing existing boundaries.

Software Innovation relies on an understanding of the state-of the-art in a given software technology. Like other technologies, software technologies can be seen to have trajectories and convergence. The trajectory of a software technology shows the direction of its development, and can be used to anticipate the next feasible innovations. Thus database software evolves through flat-file, hierarchical, network models to relational and object-oriented models. Its latest development also illustrates software technology convergence, as modern XML databases combine features of databases with their web-implementations. A modern mobile phone is part phone, part camera, part video camera, part calculator, part calendar, part mp3 player, part games console and so on. Each of these kinds of software, together with their hardware platforms and supporting infrastructures have separate technology trajectories but converge in the mobile phone. Many software innovations exploit understandings of trajectories and convergence to provide an innovative software product precisely at the time that the market is ready, and the supporting infrastructure is well-enough developed. Skype, for instance, combines internet telephony (and other messaging services) with peer-to-peer computing to provide a software product which emerges as the infrastructure of the internet is sufficiently well developed to support it.

Software innovations can be market-led or technology-led. Market–led software innovation responds to demand from a particular user-community. Technology-led software innovation is pushed by creativity among software-developers – producing software for which a market has later to be developed. Commonly, software innovations are a mixture of both.

Innovative software products (algorithms, routines, programs, packages) share two characteristics: novelty and utility. Novelty means that they have not been developed previously, utility that they have some form of application which users value. Different kinds of utility can be distinguished. Software innovations can provide underlying improvements for the delivery of other computing services (operating systems, internet browser) Software innovations can provide new, improved, more efficient or cheaper services for communities of users (like the Skype application mentioned earlier). Embedded software can enable innovation in other technology products, such as cars and washing machines. Innovative software can be an enabler or driver for business change Innovative software can change the way people interact and communicate, as for instance with social networking software (Facebook, LinkedIn)

Software process innovation can take the form of the development of new techniques, tools or methods for software development, as for example with extreme programming (XP) or SCRUM. It can concentrate on one phase of a more traditional development process, such as requirements elicitation, introducing more creative or imaginative techniques or tools. Software process innovations can involve user leadership, where expert users collaborate in the writing of software which meets their own needs (for example the Linux community). Process innovation can also focus on market analysis – where the demand for new software products lies. Common to many software process innovations is a focus on productive work, and the avoidance of thrashing – unfocused work which is neither productive nor generating new ideas. A more modern pre-occupation is with ‘flow’ (Csíkszentmihályi’s description of a mental state characterised by high energy and focus) in a software team

The relationship between software process innovation and innovative software products is a complex one. At the moment there is no particular evidence that innovative software processes necessarily result in innovative software products. Some forms of innovative software products may be best developed using traditional or well-known methods.

Software innovation pre-supposes creative software developers. Creativity is a mental process involving the generation of new ideas or concepts, or new associations between existing ideas or concepts. From a scientific point of view, the products of creative thought are usually considered to have both originality and appropriateness. A creative software professional can cope with poorly-defined problems, exhibit novel and unconventional thinking, motivation and persistence, and think in a heuristic, rather than an algorithmic way. They may display different styles of creativity; whereas one prefers to adapt an existing concept, or code, another may prefer to explore an entirely new direction, or experiment with several such directions. Much creative problem solving is experience- and expertise-based – the chances of arriving at a creative or novel solution are greater if the developer knows the programming environment and application area extremely well, and has worked on comparable types of problems before. Creative system developers and programmers also display good overview - the ability to keep many aspects of a problem in play simultaneously, and well-developed mental modelling capacities. This gives them both the ability to understand the relationship between many complex facets of a development task, and to express them in appropriate modelling forms to help others. Creative developers can think ‘out-of the-box’ – that is they can go beyond conventional linear logical thinking and generally held assumptions about how development problems should be tackled to produce original solutions. They can exploit lateral thinking as described by Edward de Bono. In addition to thinking in an original or unconventional way, they can also challenge the mindsets of others, and provoke uncharacteristic reactions. Lateral thinking is best described as a creativity technique and there are many others which can be appropriate for software developers, for example: 5W’s + H, manipulative verb, metaphor, Crawford blue slip writing, problem reversal, wishful thinking, peaceful setting, nominal group technique, software games. However these techniques (with the exception of the last) are not designed specifically for use in software development, and can also sometimes be superfluous or unhelpful. It’s thought that creativity in a developer can be improved by focus on their personal innovation practice.

Creative software developers work in innovative teams. These teams are appropriate for the development of new and improved software. Innovative software teams display high levels of team learning, constructive software practice patterns, excellent communication, appropriate role division and good integration of expertise. Some work environments are thought to promote creativity and innovation, whereas others can hinder it. Innovation enablers include: freedom, challenging work, sufficient resources, supervisory encouragement, workgroup support, organisational encouragement. Other environments create creativity barriers through workload pressure, rigid work practices, stress and reward systems that penalise mistakes. Though software teams are associated with project work in conventional software development firms, the may also be geographically distributed and supported by co-operative work systems. They can also be self-organising, facilitated by the internet, and motivated by non-commercial goals, as in open source communities. There are several software tools which are designed to support the innovation process, including IDEA, Knowledge Breeder, and innovation task support (Schneider).

A final aspect of software innovation is its evaluation. The benefits of software innovation, and its effects often lie many years in the future, dependent on commercial exploitation and diffusion, whereas software developers need ways of understanding whether they are working innovatively at the time of working, or shortly afterwards. Bad projects need to be closed down and bad development practices improved. A commonly used measure for technology innovation is patents, but this is not always appropriate for software. The usual metrics for software development (such as lines of code, function points, bug counts and so on) are not very helpful indicators for innovation or creativity, and new ones need to be developed.