Where do technologies fit in to automate development and assist the developer?

Thursday 15 September 2022

Authors:

Where do technologies fit in to automate development and assist the developer?

How is software produced and operated in practice?

Low-Code and No-Code tools (e.g. Appinventor ‎[4] , Budibase ‎[5] , WebRatio ‎[3] , Figma ‎[6] , Debuild ‎[7], etc.) allow us to automate development following a user-centered design approach (they use user-friendly graphical interfaces and configurations to create solutions) [1]‎[2]. In addition, there are AI technology tools (e.g. GitHub copilot ‎[8] o OpenAI ‎[9] based on GPT3) that can be used as an assistant in design and code development tasks. In the following, to better understand the implications these technologies could have on software production and operations, we will see how software projects are carried out in practice and why it is becoming more and more powerful for us to follow a user-driven design approach in contrast to the classical development lifecycle we all know (regardless of whether we do it in a waterfall or iterative-incremental way).

Anyone who has faced the design and development of a software product, and even more so if the project is an R&D project, knows the initial uncertainty that these projects have. The design and development of prototypes allow us to acquire knowledge more quickly about the solution that is needed with little effort. In addition, any change in the exterior design of a product (its interfaces) will affect the interior design of the product, so it does not seem to make much sense for us to design its interior before we are sure that its exterior design will not change (or will change little). Once we are sure that the exterior is "more stable", we can systematically design and build it to be "useful", but always, keeping in mind that we will offer a service and that we must ensure certain "guarantees". The usefulness of the artifact is what would be mainly related to the functional requirements of the product, while the guarantee (security, capacity, availability, continuity, etc.) would be more related to what we call non-functional requirements. In the end, requirements are present throughout the product life cycle, as shown in Figure 1.

Figura 1. How requirements are related in the product lifecycle (source: own elaboration)

As we mentioned, regardless of the development cycle we follow (e.g. waterfall or iterative-incremental), traditionally the classic stages of software development, without going into operations, are: Requirements Elicitation, Requirements Analysis, Design, Build/Implementation, Test. In fact, the previous stages are the ones that appear in the Ministry's templates that we auditors use to evaluate the projects carried out in our country in order to qualify a project as "technological innovation" or "R&D". In these 3-4 years as an evaluator, I have been able to evaluate more than 50 projects of this type. Briefly, some of the issues I have observed are:

• The concept of "Requirements Elicitation" is mixed with the concept of "Requirements Analysis". Elicitation should only cover the understanding and definition of the problem (e.g. a business model describing the current reality, just to understand the needs), while in the analysis, the solution should be addressed independently of the technology (e.g. interfaces, class diagram, case diagram, activity diagram, sequence diagram, etc.).

• The concept of "Requirements Analysis" is mixed with that of "Design". Although analysis is generic, design is specific to the platform or technology used to build the solution. In most of the projects evaluated, Analysis diagrams are included in the Design (i.e. generic, with no technology specified).

• Technical documentation does not usually follow any reference model, standard, or regulation (e.g. UML or BPMN techniques). In this sense, it would be important to standardise a minimum of technical documentation for this type of project (something that I am sure all of us who evaluate this type of project would appreciate).

• When testing is included, it tends to be of low quality. Only verification activities (past, after designing and building, making sure that we have designed and built the right solution) are mentioned, with few validation activities (future, before designing and building, making sure that the solution is the right one). There is also no mention of continuous monitoring activities (present, making sure the software works in real time). ‎[10].

• In some cases, screenshots are provided so that you sometimes do not know whether they are prototypes or actual screenshots of the developed software. However, code extracts are often requested as evidence that the development was carried out.

What strategies can we infer from practical experience as a strategy for software production/operations?

If we make the effort to translate all these activities into a model like the one in the figure above, we would have something like the one shown in Figure 2. In the following figure, a complete life cycle of the software product has been considered, from its conception (Discovery), through its design and development (Development), until it is put into production as a service (Operations).

Figura 2. Product lifecycle (discovery, development, operations) (source: own elaboration)

In summary, the objectives and main activities are as follows:

• Discovery: The objective is to learn about the problem and solution as soon as possible, reducing technical and business uncertainty. It is achieved through the construction and validation of a disposable prototype (business) and small technical demos to help us make decisions about the technological components to be used in the construction. It is a research and knowledge generation activity.
• Development: The goal is to build as systematically as possible (it is not science; it is engineering). It is achieved by obtaining software that is "useful" for its users and that has been tested in a development environment. It is an activity where software is generated.
• Operations: The objective is to make the software available to users so that they can carry out their activities. This is achieved through the implementation of a service that allows users to use the software with "guarantees".

If we transfer the activities of a classical development cycle to this product lifecycle, we realise that it does not only cover Development. For example, elicitation of requirements would fit the understanding and definition of the problem. The requirements analysis activity would be separated into two activities: (Discovery) those that affect the external definition of the solution (interfaces or prototyping) and (Development) those related to the internal definition of the solution (class diagram, sequence diagrams, etc.).
Design and Build activities would similarly be separated into two activities: (Discovery) those activities that aim to design and build a small technical demo and (Development) those activities that aim to design and build the software.
Testing (and we can include continuous monitoring if we include operations), would be separated into three: (Discovery) system tests that can be defined using prototypes to "validate" the software by means of a disposable prototype, (Development) unit and integration tests that allow us to "verify" the software, and finally (Operations) continuous monitoring that allows us, in real time, to control and track the service we are offering.

Discovery/Research, Development and Operations (Learn2Build vs Build2Learn)

Depending on our context, there could be different situations that would imply that we can put the focus more on Discovery (e.g. applying a Design Thinking process), or on Development and Operations (e.g. applying the DevOps set of best practices).

Figure 3. Discovery, Development & Operations.

The Learn2Build approach allows learning to be enhanced prior to construction. It is useful when we find ourselves in a context with high business and technical uncertainty. In other words, we are not clear about what the problem is, what we have to build nor how we are going to build it. A strategy that would fit in this approach would be a Design Thinking process or the Design Sprint methodology. ‎[12]. The SAPIENS methods that we will see in the following would also fit in. The validation of the artefact (at least about its usefulness, not its guarantee) in very early stages allows us to reduce the efforts and the cost involved in finding possible competitive advantages (which we will see later and which is a concept related to innovation).

Figure 4. Learn2Build approach.

On the other hand, the Build2Learn approach allows for the enhancement of achieving functional products that bring real value to the user. ‎[11]‎[13]. In this regard, the definition of value is very important, and here it is necessary to differentiate the "Value" that is provided:

1. If we only deal with a "Development" where the deployment takes place only in a development environment and the user is not yet able to take advantage of the software provided.
2. If we also integrate “Operations" where deployment is carried out in both pre-production and real production environments, allowing users to use the software under guaranteed use.

In addition, this approach requires that we have a very technically mature team, as the cost of designing and building software solutions must be very similar to designing and building a disposable prototype. In addition, it may also require the application of minimal requirements discovery and management strategies (e.g. Lean Inception ‎[14] or Agile Inception), as it is very important that good planning is carried out to prioritise those functional (usability) and non-functional (guarantees) features that we foresee would bring more value to the business.

Figura 5. Build2Learn aproach.

With all of the above, the important thing is that we have a team that, at any given time, adapts to the circumstances according to the context as shown in Figure 6, where at a given time we could start by applying only a Learn2Build approach (Discovery), at another time we could combine both approaches but without deploying in production (Discovery & Development), or only apply a Build2Learn approach deploying in production environments (Development & Operations).

It is also important that a systemic execution of the process is carried out, as opposed to an improvised execution. The main difference is that we have clear our strategy (what processes and stages of the above we carry out) and our tactics (what methods, techniques, and software artefacts we use in those processes). Regarding the latter, I leave you with a phrase from Sun Tzu's book "The Art of War" that gives rise to reflection on strategy and tactics in software engineering: "Strategy without tactics is the slowest road to victory, Tactics without strategy is the noise before defeat".