A team falls behind schedule. To catch up, it speeds up operations, cuts back on checks and postpones tasks considered secondary. For a few days, things seem to improve.
Then come errors, rework and misunderstandings. The extra work absorbs the time saved. The delay grows. The answer seems obvious: we need to move even faster.
The remedy is now contributing to the problem.
This mechanism appears in very different forms in businesses, ecosystems, public policy and our own habits. We intervene in a situation, obtain an immediate result, then discover that our intervention has changed the conditions that made that situation possible.
This is not necessarily a failure of individual reasoning. Sometimes it is a failure of perspective.
We have looked at an event where we needed to observe a system.
The world is not a collection of independent problems
In Thinking in Systems, Donella Meadows proposes a different way of looking at reality. [1]
A system is not simply a collection of elements. It is a set of interconnected elements, organised in a way that produces a certain behaviour.
A forest, an economy, a ship, an organisation or a city can all be studied as systems. What characterises them lies not only in their components, but in their relationships, their regulatory mechanisms and the way they function as a whole.
Consider a company with chronic delays.
We might blame understaffing, unsuitable software or poorly organised employees. These factors may be real. But they do not necessarily explain why the delays persist.
We may need to look elsewhere: at how commitments are made, how priorities change, how information circulates, or the incentives that lead everyone to accept more work than the organisation can reasonably handle.
Individuals can act rationally at their own level while contributing to a collectively harmful outcome.
This is one of the important insights of systems thinking: problematic behaviour can emerge from a perfectly ordinary structure, without any of its participants intending to produce it.
Changing the people is therefore not always enough. Sometimes we need to change the relationships between them, the rules they follow or the goals they pursue.
Invisible loops
Our intuition works reasonably well with simple relationships: one cause, one effect.
But systems are rarely linear.
An action produces a result, which in turn changes the conditions for the next action. The effect comes back to influence its own cause. This is called a feedback loop.
Some loops reinforce a trend.
The more widely a technology is adopted, the more attractive it may become to develop compatible services. These services make it more useful, encouraging further adoption.
Other loops seek to maintain a balance.
A thermostat measures the temperature, compares it with a set point, then switches the heating on or off. The system responds to the gap between its current state and the desired state.
Both mechanisms are indispensable. Yet their interaction can produce behaviour that is difficult to anticipate.
Take a renewable natural resource, such as a fish population.
When catches increase, fishers’ incomes can rise. That income allows investment in more effective equipment, further increasing fishing capacity.
If harvesting persistently exceeds the population’s capacity to replenish itself, the resource declines. Catches may then fall despite ever greater technical means.
Each participant may have sought to improve their situation. Yet the system as a whole has become poorer.
The mistake would be to look only at the amount caught this year, without observing changes in the fish population on which future catches depend.
Meadows distinguishes two fundamental realities here: stocks, which accumulate or become depleted, and flows, which increase or decrease them. [2]
This distinction reaches far beyond ecology.
Knowledge, trust, debt, skills, material resources and pending tasks can also be thought of as stocks. They have a history, an inertia and sometimes a threshold beyond which rebuilding them becomes difficult.
We tend to look at what comes in and what goes out. Less often at what accumulates between the two.
Why good intentions are not enough
A second difficulty concerns time.
We tend to reason from what happens immediately after a decision. Yet a system may respond with a delay of days, years or generations.
A policy can seem effective before its side effects appear. An organisation can improve its short-term results by gradually consuming its reserves of skill, trust or resilience.
The problem becomes more subtle when measures of success conceal this change.
Imagine a service whose official goal is to resolve users’ requests. To improve performance, management decides to measure the number of cases closed each day.
The figures rise.
But if teams are encouraged to favour easy cases, close some requests prematurely or defer complex problems, the measure can end up distorting the work it was meant to improve.
The indicator keeps rising while the actual quality of service deteriorates.
Meadows describes this kind of situation as one of the possible traps in systems: seeking the wrong goal. [1]
A system tends to produce the behaviour its rules and incentives favour, even when that behaviour contradicts its stated intentions.
There is therefore a fundamental difference between what we say we want and what our organisation actually encourages.
To understand that difference, reading the objectives is not enough. We need to observe behaviour over time.
Where to intervene
One of Meadows’ most fruitful contributions is her thinking about leverage points: places where an intervention can change how a system functions. [3]
We often try to act on what is immediately visible and measurable: a budget, a rate, a capacity, a deadline, a headcount.
These parameters matter. But they are not always the most powerful levers.
Changing the flow of information can sometimes transform behaviour without changing the available resources.
Changing a rule can have more effect than multiplying checks.
Reconsidering the goal being pursued can make years of optimisation unnecessary.
And changing our very conception of the problem can open up possibilities we could not previously imagine.
Imagine an organisation facing a chronic overload of work.
It could increase resources, recruit or speed up its processes. These solutions may be necessary.
But it could also make its actual available capacity visible, change the conditions under which new commitments are accepted, or stop treating maximum resource utilisation as the main sign of efficiency.
In the first case, the aim is to absorb more work.
In the second, the intervention addresses the mechanism that produces the overload.
This is not to claim that structural interventions are always simple or preferable. They can be costly, difficult to implement and produce unexpected effects of their own.
The value of systems thinking lies elsewhere: it broadens the range of possible interventions.
What if artificial intelligence amplified our systems?
This reflection takes on a particular significance with artificial intelligence.
We are developing tools capable of producing documents, analysing information, coordinating tasks and, increasingly, acting in digital environments.
The promise is often framed in terms of productivity: accomplishing more tasks, faster and with less effort.
But what happens when we introduce these capabilities into an organisation whose mechanisms remain unchanged?
A system that can produce reports ten times faster may simply lead to ten times as many reports being requested.
Automation that speeds up the handling of requests may increase the number of requests created.
An agent tasked with optimising a particular measure may improve it at the expense of dimensions missing from its objective.
These consequences are not inevitable. They are nevertheless hypotheses that should be examined from the design stage.
What matters, then, is not only asking what an AI system can do.
We also need to understand which behaviours its introduction might reinforce, which dependencies it could create and which human capabilities it might gradually weaken.
An artificial intelligence capable of optimising a process is not necessarily capable of deciding whether that process is worth optimising.
This distinction is essential.
Increasing our capacity to act does not guarantee a better understanding of what we act upon. It can even amplify the consequences of flaws in our conception of it.
The more powerful our tools become, the more the quality of goals, rules and feedback mechanisms matters.
Learning to see before trying to control
Systems thinking does not offer a universal method for solving problems.
That is precisely what gives it value.
It invites us to recognise that our representations are incomplete, that systems evolve and that their responses to our interventions can surprise us.
It recommends observing behaviour over time, making our assumptions explicit, seeking missing information and remaining attentive to unintended consequences.
This does not mean turning every situation into a complicated diagram.
It means developing a few habits of thought.
When a problem returns, ask what reproduces it.
When a solution seems obvious, examine what it might reinforce or weaken.
When an indicator improves, check what has actually improved.
When a system becomes more efficient, ask whether it is also becoming more resilient.
And when we think we have identified the cause of a phenomenon, consider the possibility that it is itself the consequence of something else.
There is a form of intellectual humility in this approach.
We are never entirely outside the systems we seek to understand. Our decisions, our measuring instruments and our attempts at correction change how they function.
The question is therefore not only how to act upon the world.
It is also how the world responds to our actions.
Before looking for a better solution, perhaps we should learn to recognise what makes the problem so easy to reproduce.
Further reading
Donella H. Meadows, Thinking in Systems: A Primer, edited by Diana Wright, Chelsea Green Publishing, 2008. [1]
The book is a major introduction to systems thinking: stocks and flows, feedback loops, delays, resilience, system traps and leverage points. The discussion of artificial intelligence is a contemporary extension of these concepts, not a subject Meadows addressed directly.