Category: agile

By and large, software testing has been in bad shape for a long time. And I feel that I should go very deep to clarify things. By the end of this article I hope that maybe you’ll understand better why: 

• Regression testing takes too long and finds too few bugs.
• Testing takes too long.
• Testing misses too many bugs.

With the advance of artificial intelligence and automation things are not getting easier. 

In the end I will speak about a thing that helped and can help regarding testing. I am talking here about the Context Driven Testing school of thought, and especially Rapid Software Testing (RST). And I will do this with the help of Cynefin® Framework.

But before that, since things are so problematic, I will examine some fundamentals like:

• Gödel’s Incompleteness Theorems to better understand the idea of formalization.
• Truth and its connection with Aletheia and veritas, because in the end we want to find the truth about our product.
• Phenomenology because in the end it is about our experiences with our things and how we see the reality regarding a product.

These fundamentals form the foundation for my next step in the article which is about Cynefin® Framework. I’m using Cynefin for understanding the nature of situations and revealing how testing can help in those various situations.

Then I will approach the: making sense of testing. I’ll examine how context, Gödel’s Incompleteness Theorems apply to formalization, finding product truth, phenomenology, and requisite variety specifically to inform testing. Finally, based on these I will connect RST with Cynefin® Framework. After this, hopefully, it will bring more clarity to what testing really is.

Then I will tell a story about testing that my team and I conducted together, and how we were aware of Cynefin throughout. This to demonstrate that these concepts can be real and useful in practice.

Keywords: Cynefin, RST, testing, diakrisis, sensemaking, continuum, context, truth (in connection with aletheia, veritas), formality (in connection with incompleteness theorems), experiencing (in connection with phenomenology)

1. How it started

The connection between Cynefin and Testing began while I was trying to make sense of Rapid Software Testing (RST). 

Cynefin was created by Dave Snowden. He is the Founder and Chief Scientific Officer of The Cynefin Co. and Director of the Cynefin Centre for Applied Complexity. He has pioneered a science-based approach, drawing on anthropology, the cognitive sciences, and complex adaptive systems theory. This approach, known as naturalising sense-making, has been acknowledged as one of five distinct schools of sensemaking.

RST was created by James Bach. He is the co-founder of the Context-Driven Testing. He has over three decades of experience in testing, developing, managing, consulting and writing about software. Michael Bolton is the co-creator of RST since 2006. He is a consulting software tester and teacher who helps people solve testing problems that they didn’t realize they could solve.

I was looking for things regarding complexity science in IT (inspired by Dave Snowden work) and wanted to see clear examples – hands on examples. This is how I found out about RST and convinced me to investigate deeply. There was a particular discussion I had with James Bach, in 2020, that made me realize that at that time that I cannot speak either about Cynefin or RST. But the big insight from that discussion was that I was in danger of becoming a blind follower of RST, Cynefin and their creators. And this scared me a lot, because I know for sure that they would not have wanted this. To be a blind follower, I believe, can do more bad than good for the exact things (Cynefin, RST) I cared about. 

Only now I dare to write about Cynefin. And it took me years to say that I am also a tester.

2. One word

If I would choose a key word for this article, it would be discernment (greek: diakrisis).  

Nicolae Steinhardt, in his book ‘The Journal of Joy’ (1), is stating an unexpected thing about the main quality of a person. He is stating that Christian orthodox monks do not consider the main quality of a person as goodness or intelligence or love or faith or piety or even holiness, but that of discernment. For him none of the virtues is absolute and only the skillful weighing/balancing of many can help us guard ourselves. Guard ourselves from evil which, he says, is quite easy. But also, to guard ourselves from scholarly roguery and sophisticated errors.

I feel that somehow this applies also to professional work. I think that in IT we were badly influenced by these scholarly roguery and sophisticated errors. Testing and management suffered and are still suffering a lot because of this.

The conceptual frameworks, on which I will present in this paper (Cynefin, RST), are, for me, also a way to help us with proper discerning.

3. Premises

I like how Dave Snowden, in almost each of his presentations, starts with some basic scientific facts on which he is relying on. For me they are like premises. By doing so, he is trying, I believe, also to guide the attendees with the proper framing (2) .

Michael Bolton said a thing about premises that influenced me deeply over the years which is that:

“most arguments seem to be about conclusions, when in fact, they’re about premises” (3)

I would like to mention three premises, but more could be added. The inspiration for them came from Adrian Lemeni and his book ‘Truth and Demonstration’ (4) – it was a surprise for me to see that this book would help me fill some important gaps in my understanding of things. The premises are:

• Gödel’s Incompleteness Theorems
• Truth and Aletheia
• Phenomenology

Wayne Roseberry summarized their need, while reviewing this article, so nicely:

“The incompleteness theorem tells us not all things can be understood via formalization and automated means. Aletheia and phenomenology tell us testing is about truth-seeking via experience.”

3.1 Gödel’s Incompleteness Theorems

Are there limits regarding formalization? 

“GT1 Any consistent formal theory of mathematics must contain undecidable propositions. 

GT2 No theorem-proving computer (or program) can prove all and only the true propositions of mathematics.” (5)

There are two things that really surprised me about Gödel and his theorems. 

The first was that he was a Platonist. Being a Platonist implies a certain positioning of the way of seeing things. This way of seeing things allowed him to prove these theorems because it was about the fact that the truth precedes knowledge and it does not represent the result of some axiomatic construction. That the truth is above the senses, and the knowledge is beyond what can be observed and provided through the senses.

The second is about important implications from the perspective of philosophy of mathematics referenced from these theorems. Adrian Lemeni devotes dozens of pages to these implications (marked with italic):

• truth is not exhausted by its formulation (whether written or spoken) – so there is more about a product in the wild and in our head, then there is described in the requirements, tickets, emails, wiki’s, automated deterministic checks.
• the truth of reality transcends any description of it – the most unusual ways on how an application can be used can easily surprise any description of it.
• truth transcends proof / axiomatic reasoning; there is no equivalence between truth and demonstrability; thought cannot be exhausted by formal logic – programmers are accustomed with deterministic checks as being the only testing. 

There are limits with formalization. And this is happening because, as Max Boisot said, the incompleteness theorems apply to the conversion of data into information. Moreover, they apply to the conversion of information into knowledge. The whole narrative form of our knowledge about a product and its context cannot be captured by formal codification or abstraction. (6)

3.2 Truth and Aletheia

How can truth help us in making sense of things? When thinking about truth is all about correctness and nothing else?

 Martin Heidegger is a German philosopher. He made an analysis about logic, and it states that for ancients, presocratics, logic is related to logos and the logos to the fýsi (english:one’s nature; romanian: fire). So, logic aims at the being/essence of reality. 

Martin Heidegger speaks about truth as Aletheia which means un-concealment or un-hiddenness. This truth-openness is discovered, not constructed, it is revealed by the un-hiddenness of the thing.

In Aletheia the person engages in the process of un-hiddenness. Aletheia is reality ‘seen’ via un-hiddenness. From this perspective, truth has its own ontology.

This is a very different perspective from what we are accustomed to nowadays. By logic, nowadays, we only refer to the correctness of propositions or the correctness of deductive reasoning or the correspondence between statement and reality (7) . So, it is about truth-correctness(veritas), because truth depends on a statement, on the correctness of a sentence. Don’t get me wrong. This is important and needed, but not enough.

Martin Heidegger is showing that veritas (truth-correctness) has as a prior foundation Aletheia (truth-opening).

So, the truth is not the result of some correctly executed algorithm for which reality can be reduced only to some statements. One loses the essence of truth of a thing, when reducing the truth of the thing to the truth of the sentence about that thing.

3.3 Phenomenology

Phenomenology focuses on being rather than knowledge. The phenomenological perspective is to assume that everything that someone experiences is real.

Adrian Lemeni is saying that it is about the knowledge of unmediated reality, linked directly to the experience of things.

I was discussing with a manager the other day about the experience I had in trying some things on which he was very interested. And I told him, among other things: “It is not about me being subjective or objective, it is about my experiences which were very real”. He was smiling, acknowledging what I was saying.

 4. Cynefin

Cynefin has five domains: Clear, Complicated, Complex, Chaotic and Aporia/Confused. And between these domains there are various dynamics. 

Before writing about Cynefin with its domains and dynamics, I deeply believe that I should write about some background information about Cynefin. These things play an important role in understanding Cynefin in a more in-depth manner. I speak about: 

• praxis – because Cynefin has a solid theoretical foundation.
• sensemaking – to help us understand more deeply what making sense means because Cynefin is about sensemaking.
• context – sounds so simple but is a deep concept.
• continuum – to understand that situations are very dynamic and not static.
• kairos time – there is a time and moment for everything.
• biases – there are two main biases that have a central place for Cynefin.

---

I have mentioned before about Aletheia. Cynefin, I believe, is so generous in helping the person in the process of understanding Aletheia but also veritas. For me it helps in the un-hiding process by offering various ontologies, ontologies which will help in how the person will know about the situation/world/context/system.

I also must say that in February 2025 I took the Cynefin Basecamp class. This class was full of insights that will be further discussed.(8)

4.1 Background information about Cynefin 

Praxis

Praxis is about theory-informed work (9). For me it was key to be sure that I make sense of what Cynefin really is (10). It was key because I was able to read part of the theory that influenced the creation of Cynefin and Dave Snowden‘s work.

With this occasion I was able to discover the work of: Max Boisot, Alicia Juarrero, Gary Klein, Paul Thagard, Paul Ciliers, Gabriele Lakomski, Michael Cohen, Gilles Fauconnier….

I was surprised to find out that problems with which we are struggling in IT are not something new in other disciplines and sciences have begun to offer some interesting answers.

Sensemaking

Dave Snowden defines sense-making as:

” How do we make sense of the world so we can act in it” (11).

I wrote ‘sense-making’ and not ‘sensemaking’ because Dave Snowden is making a difference between them.

This definition of Dave Snowden’s regarding sense-making is generous because it allows one to go on multiple levels depending on when and where needed. In my case this is about how we make sense of the world of software testing so that we can act in it. 

Nigel Thurlow in his beautiful book ‘The Flow System Playbook’ treats this subject so nicely. He is saying that sensemaking is for understanding complex problems, conditions, dynamics and environments. But also, that sensemaking is about thinking, but also doing, an interaction between the two – so agency (the capacity to act; the ability to make choices) is very important. Then it says a thing that connects so directly with the real testing:

” Sensemaking is a continuous activity that can involve a number of cyclical steps. Sensemaking requires constant surveillance of one’s environment, where data and information are continuously gathered, analyzed, synthesized, and interpreted.”

Continuum

In his work Dave Snowden constantly invites people to be careful about dichotomous thinking and to move from dichotomous (contrasting parts) thinking into a dialectical (discovering what is true by considering opposite theories; critical thinking) one. (12)

Also, David Levy, in his book ‘Tools for Critical Thinking’, says that there is a tendency to dichotomize variables that should be conceptualized as continuous. 

It seems that we have done this dichotomy exaggeration as an industry both for management and for testing.

At this Cynefin BaseCamp class an interesting slide appeared. For me it was the sign that what I felt since 2020 was right and it was time to write about it. It was a continuum line (Figure 1 below). On the left was ‘unpredictable’ and on the right ‘predictable’ and along the line some boxes to make it clearer what was on that line:

Figure 1

Notes: 

• Ignore the weight of the bars for the moment, it will make more sense when this continuum is put near the Cynefin diagram. Just treat them as borders between types of systems (ontologies).
• Anne Caspari told me that they “will no longer use the linear Cynefin exercise in the Basecamp, but move into what is called Three Points (https://cynefin.io/wiki/3-points)”.

This continuum felt so simple and intuitive. And since it is about humans that try to make sense of their situations to act, suddenly I was imagining a cursor on that line that kept moving.

Context

Context is so important, in the Cynefin Basecamp class they said that “is one of the big words within Cynefin”.

We live in a world in which knowledge is forced to go through successive layers (13). Then maybe this applies also to this important word context. Alicia Juarrero is showing this in a brilliant way regarding the meaning of the word context in the book ‘Context Changes Everything: How Constraints Create Coherence’. She is using the concept of constraints to better make sense of the importance of the context and how it influences us. She is saying that constraints are processes, events, relations, conditions that are influencing the energy flow. She is giving the example of roundabouts to illustrate the idea of constraints. Roundabouts will slow down the traffic while entering the circle. And this does not happen by force, and yet they make people change their behaviors while in them.

She is identifying many types of constraints and connects them with the idea of energy flow. Energy flow that can be influenced, eased, accelerated, canceled, standardized, possible.

A fixed constraint is about only one way to do something.

A governing constraint is about specific rules or policies that have been enforced/prescribed on the system to guide what is allowed to be done or not.

An enabling constraint allows emergent system behavior that would otherwise not be possible.

Kairos time

There is the Chronos time or chronological time, the one that we know about. This is independent of the context

Then there is Kairos time, which is about those right, critical, opportune moments.

Cynefin tries to engage with it, because it is about time entangled with context. Sonja Blignaut says it so nicely that:

“for each unique context, there will be unique Kairos moments” (14).

I am working on a special project. We still need to figure out a lot of things. Is not yet the moment to align with other projects on which we might help. Helping by the technologies we use, or the data we are retrieving, or the testing we do. We are still digging in order to make sense of the things we have to do. Several times already upper management tried to force some collaboration. These collaborations might make sense in some months from now but not now.  So, we have problems regarding Kairo’s time in rapport with the others. This is very important because enforcing a premature convergence in some directions can have some nasty side effects.

Biases

Eleanor Snowden, in the Cynefin Basecamp class, listed two biases in relation with Cynefin:

●       Perceptual bias – errors that distort the perception process.

●       Inattentional blindness – failing to perceive something because your attention is focused on something else.

I was dazzled while Eleanor Snowden was talking about this aspect. She said that this is the core science on which Cynefin is based. And that when making a situation assessment it is about minimizing the impact of these biases. 

4.2 What is Cynefin

Cynefin is a conceptual framework that does something interesting, namely that it shows us that ontology (type/nature of system) precedes epistemology (nature of the way we know things) (15) – reality exists independent of our knowledge of it. So, based on the type of situation various things can be done – this means that we will not always have the same answer, no matter the type of situation. And this means that we need a requisite level of variety in how we interpret the situation, but also how we will act in it. (16) This diversity/variety is key to adaptability (17), which will help us to act and make sense no matter what the situation, otherwise things will get very bad.

Cynefin is a sense-making framework – it is about decisions and what to do in various types of situations (ontologies/domains/contexts) and how to make sense of them.

4.3 The Domains

Cynefin deals with five domains: Clear, Complicated, Complex, Chaotic and Aporia/Confused. These domains are based on three ontological states: order (where in Cynefin we have the Clear and Complicated domains), complexity and chaotic.

If we border Cynefin with that continuum depicted before we have this (Figure 2):  

  Figure 2

There are so many things to be said regarding the domains, but I will list what seems important to me. I will say again that the Base Cynefin class helped me so much, therefore credit to this class.  

So:

• Chaotic: 

Word: The word chaos is not used in the same way as it is being used in mathematics and physics.  Dave Snowden acknowledges that over the years there have been some debates regarding its usage and he does not exclude a change of this word in the future. In Cynefin it means “accidental loss of constraints, the sudden collapse into randomness”. (18)

Constraints: It is not constrained. It is important to create some kind of constraint, to create a structure from which a solution will emerge.

Cause & Effect: No relationship whatsoever between cause and effect. Nothing seems to be connected; Everything seems to be random.

Learning: Learning here might be focused on metacognition – a reflection made after things become more stable. (19)

Humans: Panic will happen due to completely novel situations which we have no prior experience. Humans must act first, in a decisive manner. (20) Hopefully that person will know what to do, so he/she will not act in a foolish way or have malevolent intentions.

• Complex

Word: “That word plex” means braided or tangled” (21). “The root word plex pertains to this entwining, twisting together, or “weaving” of threads so that they become purposefully entangled.” (22)

Constraints: There are enabling constraints. Everything is connected to everything else. The nature of the connections can be visible, conditional, and elastic. 

Dave Snowden also uses the notion of dark constraints. Dark constraint means that you can see the evidence of the constraint, but you can’t see what is causing it.

Cause & Effect: The relation between case and effect is not known, it will be known only in retrospect.

Complex adaptive systems are dispositional not causal. We can’t make definitive statements about cause and effect. They are messy by nature; they have emergent properties.

We can only understand the context by acting in it.

Here we can have multiple contradictory hypotheses which can’t be resolved in the time frame available for a decision to be acted on.

Also, here we are looking at the relationships between things instead of focusing on one way of achieving a desirable change.

Everything we will do here changes the situation in unexpected ways. The only certainty is that whatever we do, will produce unintended consequences.

Important here is to make small, parallel, safe-to-fail experiments. In this way risk might be reduced and our ability to use unexpected opportunities will increase, therefore saving time and resources.

Heuristics are important here because they are helpful to make decisions under uncertainty. A heuristic is a fallible method for solving a problem or deciding. 

Learning: Here learning is open ended, guided by heuristics. The agency of the learner is very high. Learners control their learning. Learning and sense-making is emerging. There are no limits regarding the variations in activities and experiences. Learning will help in the development of capabilities. (19)

Learning in the complex domain is different, more authentic. Is here where exaptation might happen. 

Humans: Humans are essential. Machines can’t create meaning. (20)

• Complicated

Word: “that word plic it means folded…if you think of an origami folded, I can still unfold it ” (21)

Constraints: Here are governing constraints.

Cause & Effect: It is fully knowable, but unfamiliar. It does not have emergent properties. (21)

It is not an open system, so we know very well where the boundaries are.

Multiple linear causes and effects are knowable. It is about deciding which one to pursue further.

The problems to solve are so delicate that we have to see which experts can help and what are the good approaches to solving our problems. This is different from complex, because in complex we must be aware where we are and what patterns we are observing and the constraints that guide those patterns.

Learning: Learning is about discerning extensive and even contradictory knowledge sources. (19) The experts must choose between multiple tools or methods to address a specific problem. Here expertise is very important.

Humans: Humans lead; machines support. (20)

• Clear

Word: Initially this was called the Simple domain, but since this term can be applied across all domains there are simple actions and heuristics that can apply in other domains.

Constraints: Here are fixed constraints.

Cause & Effect: Here is about what is known and familiar. The relation between cause and effect is obvious.

Learning: Learning in this context is predetermined, linear, unambiguous, structured and can be controlled. We can have standards that do not have much need for interpretation. (19)

Humans: Best for machines. (20)

• A/C (Aporia/Confused)

Word: 

Initially it was called Disorder. Then, because of liminality, it was split in two: Aporia and Confused.

Aporia means deliberate puzzlement.

Constraints: Here, it seems to me, are not about constraints as is with the other domains. There is confusion and it’s here where we need to identify where actually we are.

Cause & Effect: We have no idea here about how to look and realize about causes and effects because we are not aware in which domain we are.

When in Confused it is about inertia. It is also about frustration of being unable to agree on a way forward. It is also when we feel that we are stuck. 

Very often I have observed this when someone tried to impose the Scrum rituals for a development team. When it was not yet the moment. In Scrum there is another dynamic or line of activity that can precede or is parallel with the development activity. That is the Product Owner, and its team, work. It is not about working in sprints because it is hard to innovate in sprints, to do research, and so on. And this is a moment when I realize that as a team, we are in the Confusion domain. 

Regarding Aporia, I recalled an experience where I tried to trigger an important manager from the client side to help in moving things in the right direction. 

I applied that technique which Dave Snowden is using regarding worst practices that never should happen, like red lines that should not be crossed – the don’ts. I articulated in that email those points that should not happen no matter what. I did this intentionally to try to help generate new ideas/approaches. 

Humans: Humans are very important, they are foundational. (20)

4.4 Liminality

Liminality (“on the threshold”) comes from anthropology and expresses a sort of state of suspension between conditions.

Liminality is about transitions, about being ‘in-between’. This means that there is that space between what was and what will come next.

• Complex to Complicated Liminal

The parallel experiments were done, in a complex domain, then we can start to see the way to standardize the things. This means it will be a move to sequential experiments. It is important to constantly look for things if they are still stable.

This boundary is very important because on one side, the complicated one, a person with the right training/qualifications can be trusted. On the other hand, on the complex side, we need to enlarge our understanding of who or what constitutes expertise.

• Complex to Chaos Liminal

There are useful moments when an intentional shift into chaos can bring important insights.

• Liminal Chaos to Complex

In this liminal zone it is about the fact that some patterns are starting to emerge (23).

4.5 Dynamics between the Domains

Things are not static in Cynefin. There are movements between the domains – dynamics. There can be lots of dynamics, but Cynefin identifies 3 common ones: grazing, stable, moving through chaos. (24) (Figure 3 below)

Figure 3

In the grazing dynamic – purple line – individuals or teams constantly evolve and change the hypothesis space and experiment space regarding things. This means that the definitions of things’ and how we might approach them will change over time.

While working on proof of concepts I found myself very often in this grazing dynamic. We noticed things just to realize that we were wrong and that we must start again. Those patterns that we have observed did not stay for long. 

When it enters Aporia, the grazing dynamic , it is like we start from the beginning and see how we might recombine hypotheses/techniques that we did before. If we are clueless and we enter panic, if I may say so, then we also are in Chaos. 

The stable dynamic – blue line – starts in the Complex domain.  Once we have settled on the things we want, in the Complex domain, then it is time to make sure that we will handle them in a stable way. This means we will enter the Complex to Complicated liminal zone. In it, maybe in an iterative manner, we will see what approach is stable and knowable. After that they will switch to the Complicated domain where they will work on that approach for some time. If the approach destabilizes, or the feedback changes, or problems with the environment or stakeholders then it will go back to the Complex domain. If the approach is so stable that it can be automated, then a switch to Clear domain.

The moving through chaos dynamic – red line – is happening when we thought that things were so settled with the current approach just to experience a sudden failure – that might happen because of complacency. This complacency aspect I am observing very often, almost daily, and is becoming very disturbing. 

 But this moving through chaos dynamic might also happen when the system/context is over constrained. 

5. Making sense of Testing

I wrote about the importance of context, Aletheia, Gödel’s Incompleteness Theorems, phenomenology, requisite variety. Now I will write about these in relation to testing. 

I deeply feel that they are important.  

Then I’ll make the connection between Cynefin and RST and provide further details regarding this.

5.1 Context and its importance to testing

We know that context is key in Cynefin and all its related frameworks. But context is a key word also for the testing I am talking about.

There are various schools of thought regarding testing out there. The one that I talk about is the Context Driven Testing school of thought:

“The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework. Ultimately, context-driven testing is about doing the best we can with what we get…We accept that very different practices (even different definitions of common testing terms) will work best under different circumstances”. 

It is in this testing school of thought where we have the RST (Rapid Software Testing – created by James Bach, Michael Bolton), JIT (Just-in-Time Testing – created by Robert Sabourin) and BBST (Black Box Software Testing – created by Cem Kaner). 

I appreciate the framework Robert Sabourin applies when interpreting the specific circumstances in which testing needs to take place., in his book ‘Charting the Course: Coming Up with Great Test Ideas’. He is saying that:

“Turbulence is not limited to changing requirements. Turbulence can involve changes to Business, Technological and Organizational context drivers”.

I like that he used the word ‘turbulence’ because it implies, for me, dynamics, not something static. Cynefin is also about dynamics, this is very important.

In RST we make it very explicit about the importance of people. Because desires, perceptions, and situations are centered around people and context. (25) Once we begin to draw the RST framework, here is how it starts:

Figure 4

I love The Relative Rule coined by Michael Bolton: 

For any abstract X, X is X to some person, at some time. (26)

This rule applies to problems, bugs, definition of done, quality, etc. So, it is about looking at the relationship between things, not just focusing on one way of achieving a desirable change.

By being aware of the importance of people and social aspects the complexity aspect becomes an important aspect to consider in testing (RST) as it is in Cynefin.

5.2 Gödel’s Theorems implications to testing

In testing we have the ‘belief’ that there are problems with the product, before and during testing.

With Cynefin we know that there can be multiple types of systems having various dynamics between them and this, it seems to me, helps us to try to uncover in a more faithful manner part of the truth

Do we really think that automated verifications (like unit tests or integration tests) are enough to discover problems?

James Bach and Michael Bolton in their testing class say: Something exists; then some of what exists can be known; then some of that can be described in words; then some of that can be expressed as propositions (statements with truth values) which are either true or false.(27)  Is not enough to have some automated verifications in some pipeline to think that the product is ok. Things get very difficult when we try to realize the reality of the product in the various contexts where it is being used.

Is formalization enough to know and test things?  Dave Snowden has a famous statement adapted from Michael Polany’s statement which is:

“We will always know more than we can say…we can always say more than we can write down”.

Testing is inherently human, this means that all testing is, to some degree, tacit, as opposed to being merely explicit. Tacit knowledge will always exist and will influence us each day. 

The dominant thinking in testing is to have some test scripts and then follow them rigorously and then try to automate them fully. These scripts might have values in certain contexts, but to be the answer to everything in testing is a little bit too much. 

In Cynefin terms, these test scripts [test cases] are part of the Clear domain.

5.3 Truth regarding the product

James Bach and Michael Bolton say that:

“The direct intent of testing is to discover the truth about the product” (28).

For a long time, I was not at peace with this statement. But once I had the perspective of truth as Aletheia, but also as veritas, it started to make sense to me.

And Cynefin can help deeply to make sense of this aspect of testing. 

5.4 Phenomenology and testing

In Cynefin it is very important how we perceive things (phenomenology) and this is directly linked with how the things are (ontology) and how we know things (epistemology) (29).

And I think that James Bach, in relation to testing, might also add what matters about those things (axiology).

In testing it is not enough to take the requirements as truth. We need to see what happens with it. We need to engage with the product by evaluating, learning, exploring, experiencing and experimenting with it.

Here is the Definition of Testing (in RST):

“ ‘evaluating a product…

            by learning about it through…

                  experiencing,

                  exploring,

                  and experimenting‘

(which includes questioning, reading, modeling, observation, etc…)”

And one of the RST premises is that:

“A test is an activity; it is a performance, not an artifact.”

And all this, for me, connects with an important concept, both for Cynefin and testing (RST), which is ‘agency’. It is important because it affects how people make decisions. 

5.5 Requisite Variety

Ashby’s Law says that “only variety can absorb variety”.

When James Bach talks about this, he is saying that: “he exploits Asby’s Law of Requisite variety to explore a phase space with unknown degrees of freedom”. As James and I stated in Taking Testing Seriously book (30), this phase space of a software product is the totality of all states that it can have, with all dimensions. The degrees of freedom are variables. Testing is the process of exploring all that a program can do or be (it’s phase space), yet we will not know all it can do or be – its variables are not fully known.

When talking about variables, Robert Sabourin has a nice take on them (31). He says that to vary is to change; anything which can change is a variable; Variables influence software; software influences variables. 

In RST variables are the factors, and there are a bunch of them related to structure, function, data, interfaces, platform, operations, time – and these are some general ones – please see Product Factors section from the RST appendices (32). I like that each time James Bach and Michael Bolton invite students/collaborators to make that list their own and modify it as it fits for their purpose.

So, Asby’s Law of Requisite Variety is an invitation to action, and apart from knowledge, decisions are also a key aspect for effective action.

5.6 RST Connection with Cynefin

When I saw the continuum related to Cynefin (see Figure 1), I began to connect it with software testing: 

• Clear to most people: some things, for the system under test, are based on intersubjective demonstrated coherence and correspondence with accepted facts. 

• Expertise is necessary: Testing is a discipline. This means that expertise is not something to be ignored or trivialized. We need experts and expertise.

• Mostly predictable: there are things that are predictable while doing the testing. That’s why we have, for example, automated deterministic checks, at unit or integration or system level.

• Mostly unpredictable: This made me think about boundaries. Paul Cilliers has an interesting take on boundaries in the context of complex systems (33). Paul Cilliers is saying that at a certain moment “different systems interpenetrate each other” then that “we are never far away from a boundary” and that we should not think of a boundary as a dividing thing, but as “something that constitutes that which is bounded”.

In my case, I was thinking in more simple terms, as components in the products and how they interact.

And I began to imagine visually how boundaries of one or more components began to overlap for a  specific need in the same time: boundary regarding limits of the  data types, boundaries regarding business data limits, boundaries regarding the structure of the code,  boundaries regarding the [misuse]/use case, boundaries regarding  how code is compiled/interpreted, boundaries regarding runtime  execution of that specific part of code/component. Imagine a Venn Diagram with multiple circles intersecting each other at the same time.

I like how Wayne Roseberry says that we try to reduce infinite possibilities to finite requirements. Afterwards to build those finite requirements or capabilities. And then going back to explore infinite possible problems. (34)

• Semi-constrained – patterns emerge: It is hard to know from the beginning the reality of the product without experiencing, exploring and experimenting with it.

When receiving a new product to test it we explore the product to discover various  capabilities. In JIT (Just-In-Time Testing), Robert Sabourin talks about: Reconnaissance charters, Competitive Products, Older versions, Capability Gap Analysis. (35)

In RST we might use survey testing or sympathetic testing, and if possible, to be done in parallel with other testers involved. Then to go more deep via the product coverage outline, in order to be better aware of the new patterns of usage regarding the application.

• Volatile and random – no predictability: Sudden things will happen with the applications and people will react in various ways

---

The trigger that made me realize that I can explain better testing, and more specifically RST, was because of a continuum that James Bach and Michael Bolton made – their versions differ a little bit, but for sure they are the same in spirit (36). They have called this continuum, the formality continuum. On one side it is perfect freedom, informal, and on the other side is the perfect order, formal (Figure 5): 

Figure 5

In RST the formality was mapped on a continuum because sometimes it’s important to do testing in a formal way, and sometimes it’s not.

I liked this continuum a lot because it provides indications regarding the activities that can happen while doing the testing – and there are many more available, but it was enough to get the main message regarding formal and informal activities.

Even though the intention is about formality regarding this continuum it unveils more general aspects regarding testing.

The left part of the continuum felt to me that it applied for unordered situations and where heuristics play an important role, while the items on the right matched more with ordered situations and where rules can help more. Then I draw – initially in my head – this RST formality continuum around Cynefin (Figure 6).

Figure 6

After I did this, it became easier for me to explain what testing meant for me.

Recently, in an RST: Testing, Automation, and AI class, James Bach was talking about this continuum. Here are some interesting things that he said:

“Systematic testing is the key focus of Rapid Software Testing methodology. Anyone can test unsystematically, but to test systematically means that you have a system for your testing, you have a strategy, you can talk about your strategy.

We place this on a continuum. All testing is on a continuum between: 

Informal –completely informal, which means do whatever you want. 

            vs. 

Formal – perfect formality is everything done in a very specific way with no choices that are left to be made.

Most real-life professional testing is in the middle somewhere. But we always begin here … [James Bach was showing the left side of the continuum]. Healthy testing, responsible testing begins here … [James Bach was showing the left side of the continuum] and moves in this direction… [James Bach indicating moving toward the right on the continuum from the left part]. Because you need to Play in order to do the Learning, to figure out what needs to be formalized. 

Lots of people want to jump into the deep end and just immediately formalize the testing. You have testers who are told to write test cases on day one on the project – I think that is a formula for terrible testing. 

You need to go through an investigation process; you need to go through a learning process. During that learning process you’ll find a lot of bugs and at the same time you’ll be developing a protocol for efficient re-testing of the product. So, these two things are happening simultaneously. 

If you write test cases prematurely, you’ll only have to delete those test cases later when you learn how to test that product. So, retrospectively you’ll either be wasting money or you’ll not delete those test cases, and you’ll be locked into bad testing. 

One of the key ideas in RST is that testing is never all scripted or all unscripted. There is always a combination of choices that you make in the moment and choices that you have made in the past, or that other people have made, that you are locked into.”

Important: The activities you see related to testing the image are just some of the many that are available in RST and Context Driven Testing. Please do not forget that this continuum is related to the Formal and Informal aspects of testing, but this is just one perspective. But, I hope, it is enough to understand the richness of testing that is available in RST.

5.7 Cynefin domains and RST

Clear domain

An important testing word for this domain is ‘checking’. 

Michael Bolton found a way to distinguish between testing and checking. Testing (all testing) requires a human, but checking does not. James Bach didn’t like this idea at first. It took Michael Bolton a couple hours to convince James Bach. This has become a tremendously helpful distinction for them, giving us a crisp and simple way to talk about the aspects of testing that need human judgement and those that don’t. (37)

Checking is the process of operating and observing a product; applying decision rules to those observations; and then reporting on the outcome of those rules; all mechanistically, algorithmically. (36)

In Figure 6 we can see various activities that pertain to this domain:

• oblivious ritual 

Is about not being aware of the relationships between the way the tester works and the many aspects of the context in which he/she is working (38)

• machine checking 

Is when the machines are doing exclusively the checks. It is about those automated deterministic checks.

• “Human transceiver” – 

Is someone doing things based only on the instructions of some other person, behaving as that person’s eyes, ears, and hands. (36)

• Human checking 

Procedurally scripted test cases. The tester is guided greatly by what the scripts say. (36)

---

It is here that a tester will follow those test scripts ad literam.

The unit/integration/acceptance ‘tests’ are also here. And now it becomes obvious that it is not the only kind of testing that needs or can be done when fully testing a product.

Complicated domain

This domain is about advanced topics regarding testing. It is here, it seems to me, the tester is doing:

• Vague/Generic test procedures

Generic scripts specify general test procedures and apply them to different parts of a test coverage outline. 

Vague scripts specify a test step-by-step but leave out any detail that does not absolutely need to be pre-specified. (39)

• Specific & Pre-specified Data

Particular, precise data to be used for a specific scenario.

• Matrices & Outlines of Test Conditions 

Are about any documents needed to track or record the progress of testing. (32)

---

I came to realize that if I think of advanced test design techniques that are knowable, but rather difficult, then it applies to this domain. Which somehow matches the three points mentioned above.

For example, it is in this domain that we will perform advanced memory leak testing if we have expertise in how to do it.

Another important aspect is that of tool implementation. For me it is here where we will implement some tools that can help further with the testing.

Complex domain

This is the most ignored domain, it seems to me, and yet so important. 

In this domain there are, among other things, about: 

• Exploratory surveys 

Any testing that has its primary goal learning about the design, purposes, testability, and possibilities of the product. It tends to be interactive, open and playful. It provides the foundation for effective, efficient testing, later. (39)

Exploratory process means self-controller seeking in a complex environment.

All testing is exploratory to some degree. Exploratory testing does not mean unplanned testing, but rather testing that is under the active control of a tester (as opposed to testing that blindly plods along without responsible control). Exploratory testing is normal testing. Just as exploratory management, exploratory medicine and exploratory legal work is ordinary. What is strange is to script your actions in advance and then fall asleep. It’s scripted testing that you should be warning against, if you must choose.

• Sympathetic testing 

It is about what is wonderful about a product, but the focus is not on bugs, but on learning about the product. (40)

• Interviews and discussions
• Play with the product

---

A new team beginning to test the product, that will start in a deliberate manner, will act here. And it can do this without any requirements. They need to play, to survey the product, to understand what is happening before doing more deep testing. 

New testing ideas will emerge here because the constraints are not so tight.

Chaos domain

It is in this domain that some form of action must be taken as soon as possible. 

Imagine the application crashes when it should not. Panic or urgency might settle in, but suddenly someone might think that looking into Event Viewer, from Windows, is the important thing to do. Once the tester acts on this, other colleagues might think of other ideas like looking quickly in the logs. And slowly, slowly some patterns of investigation will become more obvious and the shift from chaos domain to complex domain has already started.

5.8 Cynefin Liminality and RST

This aspect of liminality is ignored but is very important.

I asked James Bach about liminality in general. He did not know that I was asking in the context of Cynefin, but he knew about this anthropology concept. He explained to me that in RST it is all about liminality. 

Liminal Complex to Complicate

In the liminal zone from Complex to Complicate we began to have a clear sense of direction and of how to handle the testing in a more specific way. It is here that, for example, we will create, or to better have a sense of the testing scenarios and prioritize them. 

Also, here maybe a Product Coverage Outline should be created, because things are clearer of what is seen, but also of what is not seen regarding the product factors (variables). A Product Coverage Outline is about product factors that could matter. It does not include test procedures or detailed activities. 

Liminal Chaos to Complex

In this liminal zone it is about the fact that some patterns are starting to emerge (36). This means it might become clearer on what we should discuss and with whom (“Interviews and discussions”) also it might become clearer on the aspects of where we should concentrate the “Exploratory Surveys”. And then, once we are in the Complex domain, we can do lots of safe-to-fail experiments (survey testing) to see more coherent directions.

Liminal Aporia 

When I talked about Aporia I mentioned an experience where I tried to trigger an important manager from the client side to help in moving things in the right direction. And the way to trigger for action was to mention the don’ts. 

This confusion was intentional. The idea was to see what else they may find to do proper testing and not to have so many problems with it.  

The don’ts I wrote about testing are: 

• Counting things to manage testing (like test cases/scripts).
• Use pass or fail (in an absolute manner – a feature will never be pass or fail).
• Detailed scripts as a dominant test strategy.
• UI automation as a dominant test automation strategy

5.9 Cynefin Dynamics and RST

Figure 7

In real life things change, new insights appear and Cynefin will reflect this via the dynamics. 

Stable dynamic – blue line

As said, the stable dynamic starts in the Complex domain. It is here where, via multiple, parallel, experiments we begin to realize the things that need to be tested.

Once we have an idea of it, we will switch to the Complex to Complicated liminal zone to see which of the testing activities are stable/reliable enough in order to be approached, maybe even to prioritize them. This operation can be done in an iterative way. 

For me, it is in this Complex to Complicated liminal zone Product Coverage Outline (PCO) and the Release Coverage Outline (RCO) can start to be created. I say this because it is about an analysis that will open the path to see the stable things that need to be done for deeper reliable testing and to plan for it.

Once we have identified some stable approaches we switch to the Complicated domain.

Once in the Complicated domain it might happen that we realize that the corresponding part of the PCO or the scenarios we thought about the approach are not feasible at all. Then we go to the Complex domain to understand more about what is happening.

When things are getting clearer on how certain aspects of testing can be automated then a shift from Complicated domain to Clear domain happens.

I believe that new tool ideas will emerge in the Complex domain. This dynamic can help us to explain why. First, we got the hunches on what tools might help for testing. Then in the liminal zone we make sure which are feasible to do, once the tool identified that can help switch to the Complicated domain is made. If we realize that the tool intention is not ok, then we cycle back to the Complex domain to investigate more.

Moving through chaos dynamic – red line

We thought that things were well settled with the current approach just to experience a sudden failure with the current release because elusive bugs were found at the client side.  

Another example that crosses my head is when, for example, more than 80% of colleagues were fired and suddenly all the existing testing approaches are not sustainable anymore.

Another example is when it becomes clear that the automation strategy is so expensive and useless that sudden actions need to be taken as soon as possible.

Grazing dynamic – purple line

When we try to make sense of the product there are lots of hypotheses and experiments to be done. Sometimes we might get the feeling that it stabilizes our understanding only to return to the beginning and revise our hypotheses and start the experiments again.

When it enters Aporia it is like we start from the beginning and see how we might recombine hypotheses/techniques that we did before. If we are clueless and we panic, then we also are in Chaos. 

5.10 A Story

Sometime ago my team and I, full of programmers, were tasked to test a product. The client knew that we see testing in a different way – and it was ok for them, we had to find as many problems as we could. We knew that we would have to switch roles. And above all we were fully aware of the discernment aspect of our work and not being tricked by the best common practices applied in the industry no matter the context.

We knew that we did not have experience with it. Each of us began learning about it and constantly sync with the others. We were doing parallel experiments to make sense of what is happening with the application. We have intentionally ignored any existing test scripts that were present, because we wanted to discover the reality of it ourselves. 

We observed that when people say ‘exploratory testing’ it means some obscure clicks in the application and that’s it. Not for us, we had a structure, we had direction, we spotted some patterns, we had a strategy. Everything we did could have been audited.

We also knew that although this might seem slower, in the end it would make us go faster as time passed. Management wanted to test via the scripts, because this is the usual way, but also because they were thinking that it would have been enough. They did not understand that we needed to build that tacit knowledge.

The time for doing this, in this initial phase, was one week. We have discovered after this period that things ran more smoothly between us ( the team members ). 

The idea came from the RSTA classes where on the first day all students are doing survey testing. James Bach does a debriefing afterwards helping them connect the dots, but also to see the relevant patterns after connecting the dots. All the students are doing safe to fail parallel experiments to learn about the application.  While doing this they are in the Complex domain in the grazing dynamic. This experience of the first day of the RSTA class was before my eyes, and it took me some time to realize this.

The next week, after the initial phase, for us, was to make sure that we understood how to fill in bugs. 

In all this time we were in the complex domain, in the grazing dynamic. It was not easy but certainly made our lives better afterwards. 

 Then we were able to build a big image regarding the product, the ‘Product Coverage Outline’. We were not yet sure of how to deeply test all the factors discovered. But it was much more evident the direction and even how to manage it and how to prioritize it. We were in the liminal zone between Complex to Complicated.

I recall that a functionality, regarding some kind of import, was made some time ago. While we were doing exploratory surveys we noticed lots of commits regarding this functionality – it was not only the implementation, but also a lot of bug fixes. That functionality didn’t seem very robust.. We were still in the Complexity domain. And we had that strong belief that things will crash here deeply, especially with memory. In the liminal zone from Complex to Complicated we decided that we should follow our gut and stay for another two weeks because a possible memory problem will arise. Then in the Complicated domain we knew precisely the way to do so we could find that problem. And we found it. We were so happy.

Also, in the Complicated domain we knew how to build a tool, we identified this need before (liminal zone). This tool was created to help us to generate appropriate data to identify the possible memory leak problems. Initially we entered the Complicated domain, but with no tool. And we switched back to the Complexity domain because it did not feel right to struggle to constantly modify hundreds of lines of xml. We wanted to see the options. In the liminal zone we took the decision to implement a small tool that could help us. And then we switched to Complicated domain in order to do the tool.

Once we did this it was rather simple to just upload the file and see what is happening with the memory. We were taking some steps in a specific manner in the application to try to see for the memory problem, we were in the Clear domain while doing this operation.  We could have also built something to make this operation, but we decided not to do it.

We knew about Cynefin, we knew about its dynamics. We knew about the necessary confusion. 

Months after our testing, we found out that this import was tested, but only a test case was used. What was interesting is that by looking at that test case it gave the impression that it was tested thoroughly. When we tested, we used a tool to keep record of the activities we had done – those activities were sessions from Session Based Test Management (41). In short, we kept a journal about the activities done and their details. There were literally hundreds of examples compared with that one test case. It was so obvious why the test case approach induced error. Those examples were highly exploratory, not initially created and then forced upon us – yes, again, the stable dynamic.  

We have used Session Based Test Management because it was important to show with great determination what we did. It was for the first time we did this for our client, so we wanted to gain their trust with what we were doing, not just to say that we did ‘exploratory testing’. 

We had sessions that were deeply reviewed and we realized that it was not enough as a report. We wanted to show in a more transparent manner the status of what we had tested. Initially we tried to figure out what we could do, we had some ideas – yes, we were in the Complex domain. Then we entered the liminal zone from Complex to Complicated having some possible approaches to make a more transparent testing report that could help us in our context.  I went to James Bach for his expertise to help us – the Complicated domain. As you can imagine James Bach is very inventive and already has spotted patterns of kinds of reports and always, he is ready to adapt them based on the context. In our case he helped us to articulate the assessment of the quality of the project, and the quality of the feature area – because we wanted management to see the threats to the shipping date. He has also helped us to articulate the coverage in order to show how well we understand each area so far.

6. Conclusion

A friend of mine told me something interesting when he reviewed the latest version of this document: “I would also put a disclaimer that it’s an exercise in making sense of things, but like anything, ‘it’s a work in progress.’ I would dare say that it will always be “in progress” because I’m constantly learning and there’s no better exercise than expressing your thoughts in writing.” He is so right. As I said in the beginning all the details mentioned in the article were made with the hope to help with proper discerning regarding testing. And this discerning, it seems to me, is always “in progress”.

Testing is in bad shape—I said that at the start. Cynefin helps explain why. I hope you’ve now glimpsed the richness of testing across different systems, a richness we deeply ignore.

And this ignorance explains why, among other things:

●       Regression testing takes too long and finds too few bugs.

●       Testing takes too long.

●       Testing misses too many bugs.

One last thing: recently, I watched a video featuring Romanian poet Ana Blandiana (42). She was discussing Costică Brădățean’s book ‘Dying for Ideas: The Dangerous Lives of the Philosophers’—an analysis of philosophers who backed their ideas with their own sacrifice. She then said:

“I think that in a way we all lack this: the power to guarantee our ideas, to live them, not just to say them. Because the moment we actually live them, we would realize whether they are true or not, absurd or not.”

I deeply think that Dave Snowden, James Bach, and Michael Bolton live by their ideas; they realize what is true or not, what is absurd or not.

References

(1)  ‘The Journal of Joy’ , Nicolae Steinhardt, https://svspress.com/the-journal-of-joy/?srsltid=AfmBOoqD_ZEqeHpOU_gr9bGTiJ5J_dIIc-RHV1kcwY7Ylz68ciOvECV5  

(2)  Framing metaphor used by Gary Klein in ‘Making Sense of Sensemaking 2: A Macrocognitive Model’, https://www.researchgate.net/publication/3454376_Making_Sense_of_Sensemaking_2_A_Macrocognitive_Model  

(3) ‘How do you count an idea?’ , Michael Bolton, http://www.developsense.com/blog/2006/07/how-do-you-count-idea/  

(4)  This is a Romanian book; the original title is ‘ Adevăr și Demonstrație – De la Incompletitudinea lui Gödel la vederea mai presus de orice înțelegere a Sfântului Grigorie Palama’. There is no English translation, yet, of the book, but I would translate the title as follows: ‘Truth and Demonstration – From Gödel’s Incompleteness to the sight beyond all understanding of Saint Gregory Palamas’. 

I like his work a lot.

https://basilica.ro/adevar-si-demonstratie-de-la-incompletitudinea-lui-godel-la-vederea-mai-presus-de-orice-intelegere-a-sfantului-grigore-palama/

(5)  ‘A logical Journey – From Gödel to Philosophy’, Hao Wang, https://direct.mit.edu/books/monograph/4280/A-Logical-JourneyFrom-Godel-to-Philosophy 

(6) ‘Explorations in Information Space’, Max Boisot, https://www.amazon.com/Explorations-Information-Space-Knowledge-Organization/dp/0199250871  

(7) ‘Lumina celui Nevăzut. Logică și credință’, Adrian Lemeni, https://youtu.be/b6fBQXJmwF4?si=0HXBPRXc0PlZnxv5  

(8) Cynefin BaseCamp class was held in Feb 2025, held by Anne Caspari and Friso Gosliga. Where also Eleanor Snowden participated in and offered nice insights; https://thecynefin.co/training-events/ 

(9) ‘Praxis’,  https://cynefin.io/wiki/Praxis ; ‘Shorts and Not-So-Simples’, Dave Snowden, https://thecynefin.co/shorts-and-not-so-simples/ 

(10) Marius Francu, https://sensing-ontologies.com/tag/praxis/  ; 

My chapter from the book Taking Testing Seriously – https://www.amazon.com/Taking-Testing-Seriously-Software-Approach-ebook/dp/B0FXC3CXNV  

(11) ‘What is Sense-making?’, Dave Snowden, https://thecynefin.co/what-is-sense-making/  

(12) ’Clusters in the tail’, Dave Snowden, https://thecynefin.co/clusters-in-the-tail/; ’The enemy of my enemy is my friend’, Dave Snowden, https://thecynefin.co/the-enemy-of-my-enemy-is-my-friend/  

(13) ‘Un verset înțeles doar pe jumătate: Adevărul vă va face liberi’, pr. Nicolae Dima, https://youtu.be/wvvPoo-ZU0M    

(14) ‘Flowing through time: the need for a certain slowness’, Sonja Blignaut, https://sonjablignaut.medium.com/flowing-through-time-the-need-for-a-certain-slowness-76fa6321bb0b  

(15) ‘Good fences make good neighbors’, Dave Snowden, https://dl.acm.org/doi/abs/10.5555/2596465.2596475  

(16) ’Multi-ontology sense making a new simplicity in decision making’, Dave Snowden, https://www.researchgate.net/publication/7793823_Multi-ontology_sense_making_a_new_simplicity_in_decision_making 

(17) ’Complex acts of knowing: Paradox and Descriptive self-awareness’, Dave Snowden, https://www.researchgate.net/publication/241660493_Complex_Acts_of_Knowing_Paradox_and_Descriptive_Self-Awareness

(18) ‘The chaos word’, Dave Snowden, https://thecynefin.co/the-chaos-word/ 

(19) ‘Leadership by Learning Design: embrace complexity where it exists’ ,  Ray O’Brien; Samuel Mann; Richard Mitchell,  https://journal.aldinhe.ac.uk/index.php/jldhe/article/view/1372  

(20) A Cynefin Framework Lens: Where Machines Work Best and Why Humans Remain Indispensable,  from OurHaunt wiki – available to the Premium Membership users of  thecynefon.co site, https://thecynefin.co/memberships/ 

(21) ‘Complexity Management, Complexity fitness, & Wicked Problems – Agile to agility #85’, Sonja Blignaut, https://www.youtube.com/watch?v=GYxZpNGchQI  

(22) https://membean.com/roots/plex-weave  

(23) ‘’Liminal Cynefin & ‘control’’, Dave Snowden, https://thecynefin.co/liminal-cynefin-control/ 

(24) ‘Cynefin Dynamics’, https://cynefin.io/wiki/Cynefin_Dynamics 

(25) ‘Problems with Problems’, Michael Bolton, https://developsense.com/blog/2012/04/problems-with-problems 

(26) Michael Bolton, https://developsense.com/blog/2010/09/done-the-relative-rule-and-the-unsettling-rule  

(27) ‘The Logic of Verification’, James Bach and Michael Bolton, presentation from the RST class, https://rapid-software-testing.com/rst-courses/ 

(28) ‘Why I Am a Tester’, James Bach, https://www.satisfice.com/blog/archives/40351  

(29) ‘Spinning the Cynefin discs’, Dave Snowden, https://thecynefin.co/27822-2/ 

(30) ‘Taking testing Seriously’, James Bach; Michael Bolton, https://www.amazon.com/Taking-Testing-Seriously-Software-Approach/dp/1394253192  

(31) ‘Revealing Most Important Variables To Design Better Tests’, Robert Sabourin, https://www.thetesttribe.com/courses/variables-in-test-design/  

(32) ‘RST appendices’, James Bach; Michael Bolton, https://www.satisfice.com/download/rst-appendices  

(33) ‘Boundaries, Hierarchies and Networks in Complex Systems’, Paul Cilliers, https://art-sciencefactory.com/Cilliers2001.pdf  

(34) ’Drawn to testing’, Wayne Roseberry, https://www.amazon.com/Drawn-Testing-Wayne-Roseberry-ebook/dp/B0FHF9T28K/  

(35) ‘Charting the Course: Coming up with Great Ideas; Just in Time’, Robert Sabourin, https://www.amazon.com/Charting-Course-Coming-Great-Ideas/dp/B0CX8XPJQB  

(37) ‘Testing and Checking Refined’, James Bach, https://www.satisfice.com/blog/archives/856 

(36) ‘Breaking the Test Case Addiction (Part 3)’, Michael Bolton,  https://developsense.com/blog/2019/01/breaking-the-test-case-addiction-part-3 

I was not able to find online the James Bach version of the formality continuum.

(38) ‘Context-Driven Methodology’, James Bach,  https://www.satisfice.com/blog/archives/74 

(39) RST class materials, https://rapid-software-testing.com/rst-courses/ 

(40) ‘How I Invented Sympathetic Testing’, James Bach,  https://www.satisfice.com/blog/archives/748 

(41) ‘Session Based Test Management’, James Bach; Michael Bolton, https://www.satisfice.com/download/session-based-test-management . The tool is provided with the class materials.

(42) Ana Blandiana about ideas – is in Romanian,  https://www.youtube.com/shorts/rBn7Bb_YAzo