A Framework of Human Systems Engineering

For truly effective HSE in system development, it is essential to measure alignment between stakeholders and their belief that the project will succeed along numerous axes to rapidly identify existing and emerging risks. In the previous work cited by the authors (Barry and Doskey, 2019; Doskey and Barry, 2018), the approach taken to ascertain this information was via survey or forensic assessment. While this can work effectively on small development efforts, this approach does not scale when the number of stakeholders begins to increase into the tens or more.

Using AI there is an opportunity to conduct real‐time and near‐real‐time analysis of sociotechnical measures using natural language processing and sentiment analysis. Sentiment analysis is well established in social media (Ortigosa et al., 2014), and APIs exist, such as Google Natural Language API (Putra, 2019), which allow individuals to quickly create sentiment analyzers. By employing agents that comb through communications like e‐mail, text messages, and information on collaboration sites (e.g. Kanban boards), discontinuities and misalignments can be identified across the sociotechnical ecosystem. This approach has the potential to recognize existing and emerging sociotechnical risk. By enabling communication between various risk tracking intelligent agents, an assessment can be made as to whether there is enterprise or localized risk.

Modern SE frameworks have recognized the importance of frequent communications between stakeholders. Further, the popularization of agile methods has provided a backdrop for these communications and the associated record of success. AI bots (Wünderlich and Paluch, 2017) and other applications can act as an unbiased risk assessor across the enterprise as the system is developed.

Figure 3.4provides a taxonomy of representative systemic risks that the AI can identify and assess:

Structural risk assessments are conducted to determine if there are inherent weaknesses in the sociotechnical ecosystems and relation risk looks at groups of stakeholders.

Relation risk is based on sociotechnical measures.

Evidentiary risk is an assessment of the progress of the system development (e.g. measurements to determine if value streams being delivered regularly and as scheduled).

Type risks aggregate groups of specific elements risks to identify systemic risk across the enterprise (e.g. widespread disagreement on technical approach).

Individual element risks focus on the specific aspects of the sociotechnical space (e.g. assessing if the objectives of the stakeholders aligned across the enterprise).

Figure 3.4 Risk taxonomy.

In Epoch 4, AI will bring persistent, holistic risk analysis across the enterprise. This will be combined with a historical assessment of progress using data from analogous systems and from the progress on the current development effort. AI will provide recommendations for addressing material weakness in the sociotechnical ecosystem and track progress along these lines.

It is important to realize that during system development activity, risks evolve. Thus, when constructing a holistic risk analyzer, various measures and interpretation mechanisms are necessary. There is a need to identify situations that are rapidly emerging and indicate an immediate risk (e.g. replacement of a key stakeholder). These immediate sociotechnical risks can be quantified via persistent analysis of sociotechnical measures and network analysis.

Perhaps more insidious than detectable risk is the scenario of a slowly growing risk, where cumulative deviations add up and/or there is a drift in beliefs from one perspective to another. Often, stakeholders have some idea that things are amiss but have no consistent methodology to identify the cause. A persistent AI that monitors, examines, and assesses the sociotemporal measures can infer the presence of these emerging risks, either from a structural perspective (e.g. lower than expected communication between stakeholders) or disparate indicators that when taken together can integrate into a full‐blown risk scenario.

There is a multifaceted opportunity for AI with respect to sociotechnical models. Consider the composition of the sociotechnical model. Typically, the development of the sociotechnical network is largely manual. The composition exercised is designed to identify the major components as well as their interactions. Defining the interactions consists of first describing and then modeling the interaction in a quantitative fashion. The interactions between disparate components require both an understanding of how the interactions occur and how the interaction affects the components. AI can assist in the definition and development of the networks by analyzing data in the ecosystem. As an example, it is well‐proven technology to develop and use link analysis (Oliveira and Gama, 2012) to create and explore networks. This process can also be complicated with the situation of an evolving social network as discussed by Cordeiro et al. (2018). AI can be used with evolving networks as well to continually assess flow relations, identify important relations, affinity relations, and alignment.

Typically, an SE ecosystem is rife with unstructured and semistructured data. Typically, large amounts of e‐mail are exchanged along with texts messages. Using natural language processing and uncertain reasoning, a base social network can be constructed that can then be adjusted as evidence is introduced. This network can then be employed to focus the measurement of sociotechnical measures. Without such analysis, the number of possible connections in a medium‐sized network can become very large, i.e. n ( n − 1)/2. Moreover, as some of the connections are likely not significant, without a network analysis, nonrepresentative emergent effects may be inadvertently modeled.

Perhaps the most intriguing use of AI in the development of sociotechnical systems is creating digital twins to represent the interests and proclivities of the various stakeholders. Representative stakeholder digital twins provide a mechanism to better understand how events in the development of the sociotechnical system will be interpreted. For example, if some stakeholders have significant interest in a rapid delivery of specific value, the effect of delays in that delivery can be modeled.

The mechanisms for building representative models can be simple. For example, a simple version of this would be the development of a Bayesian network that processes the introduction of evidence of successful or unsuccessful progress with an assessment of the likelihood of successful capability delivery (Misirli and Bener, 2014). One can posit a digital mapping of the stakeholders’ preferences that can describe influence propagation across the stakeholders that will either increase or decrease alignment and belief in the potential success of the capability development.

Digital twins that model stakeholder beliefs provide an opportunity to do exploratory experiments to examine how to best manage risks that may result from emergent effects of influence propagation across the sociotechnical network. There has been interesting work in this area, for example, agent‐based models have been used to model bidirectional influence propagation (Li et al., 2016).

Practitioners can apply the methods and tools, as described in the previous sections to create models of the sociotechnical environments where risks are actively modeled, monitored, and predicted; alternative courses of action can be simulated and assessed to provide practitioners with insight on how to lower risk and maximize value delivered. Using AI‐based approaches to develop explicit models of stakeholders, how they relate and how their beliefs directly affect the chances for successful capability development is foundational to the constructs employed in the fourth epoch. These “proxy stakeholders” provide a powerful mechanism to conduct explorations into possible futures and allows proactive risk mitigation as the scope and scale of development efforts increases. The opportunity for the development effort to potentially get off track due to the multitude of stakeholders and the rapid pace of development necessitates holistic active monitoring and continual risk assessment that is infeasible for manual human‐intensive approaches.