In this In A Nutshell series, I’m going to be covering Complexity, inspired by recently reading a fantastic book by Sean McClure titled, Discovered, Not Designed (Building Things in the Age of Complexity). The Complexity being discussed here is not the colloquial kind, but is referring to the modern definition which describes things that are more than the sum of their parts. This in some ways dovetails off of my last post about Chaos Theory, which is a related topic. My goal is to briefly touch on the main ideas of modern Complexity along with how what I learned from reading McClure’s book lends insights into building and managing complex systems. I’ll also look at how some of those ideas can be related to the practice of software engineering.
What is Complexity?
There is debate about how to define Complexity, but one way to think about it is that it’s an equal distribution of chaos and order. A completely random set of information could be thought of as “simple” in that it is uniformly unpredictable and chaotic – it is predictably unpredictable. A set of even numbers in sequence could be thought of being simple as well since it’s also predictable. But, a set of some random numbers interspersed with some sequences of even numbers – now we’re starting to see something that could be defined as “complex”.
For the purpose of this post, Complexity is meant to describe systems whose behavior cannot be explained by a causal chain of events or actions (or, as McClure puts it in English-styled humor, “things bumping into other things”). A car engine is an example of a deterministic system, i.e. a non-complex system that is predictable, where the parts not only make up the whole, but can be causally linked to explain why the engine runs and how it works.
In the modern age, as things progress more and more, we are starting to have to deal with and build truly complex systems such as A.I. which are not reductionist in the classic Newtonian sense. What this means is that you can’t understand the system by understanding the individual components as you can with a car engine. What the system is, is not the sum of its parts, but all of the inter-weaving interactions of its parts that in combination create an emergent outcome, structure and behavior which could be fundamentally different from any of its parts. This makes understanding such a system in terms of causal reasoning unachievable, as if it were a “black box” which we cannot look inside to understand, making it impossible to reverse engineer or analyze in the same way we would for deterministic systems.
This suggests a need to consider changing one’s paradigm for approaching how to deal with truly complex systems. Not only how to work with them, but how to build them.
Key Terms:
- Emergence: An unexpected behavior, attribute or structure that materializes in or from a complex system. The cause cannot be reverse engineered or determined, only witnessed and verified after the fact. This is a key property of Complex systems.
- Self-Organization: An interacting of parts in a system in an organized or seemingly intelligent way that is not designed, predictable or planned. A colony of ants self-organizing to accomplish a task is a well known example of this in nature. It’s also worth noting that the individuals in the group only have local knowledge and not that of the group or a high level view of “how they fit in the whole” to guide or direct their behavior.
- Nonlinearity: A result or outcome that is not arrived at “in a straight line”, or in steady, consistent steps (i.e. repeatedly adding or multiplying something by constants). Nonlinear results are the outcome of sudden jumps and can be surprising or hard to predict intuitively. For example, an exponential equation is nonlinear because as you exponentiate a higher and higher number, the result increases in larger and larger jumps. Complex systems are nonlinear in that they produce structures and behavior that are more than the “steady” addition of the parts of the system.
- Multiple Realizability: Outcomes or structures can be produced in many different ways via multiple paths or configurations. An argument could be made that the structures we see in nature are those with the most number of possible configurations to arrive at. In deterministic systems there is a right path to get the right output, where in Complex systems there is no “right” path, only a right outcome, and there are multiple potential ways to get there.
- Meta or Hyper-parameters: Going outside of a system and looking back into it. For example, “meta-data” is data about data, Hyper-parameters are parameters that affect internal parameters in a Neural Network.
The Primary Properties of Complex Systems
An important thing to understand about complex systems is that they cannot be understood in the same manner that deterministic systems are by using causally based reasoning. We know we are dealing with a complex system if it has certain properties that can only be observed and emerge after the fact. These properties include Self-Organization, Emergence, Group Selection, Nonlinearity and Multiple Realizability.
A key idea is that complex systems are not additive. The end result or behavior is not something you can get by adding one thing onto or with another in succession or in a specifically planned way. The structure or behavior of a complex system is the aggregate, unpredictable cumulative result and emergent properties that arise from all the overlapping interactions of its parts.
Group Selection
One of the most important points made in “Discovered, Not Designed” was that Nature uses the full distribution. What this means is that groups or the whole are the important “unit” of effectiveness, not individuals. You can’t think in terms of individual parts to the system in Complexity – only the group can be considered. For example, what makes an ecosystem is not “birds” or “rabbits”, but all of the components in aggregate working together as a group, from all corners of the distribution.
This is contrary to how we like to think about how a lot of things are distributed in terms of the Bell Curve, or the Gaussian distribution, where there is a peak which represents more important information compared to the smaller tails. In nature and complex systems, the tails are just as important as the peaks, so to speak, because the system is the emergent outcome of all of the parts interacting together – the full distribution.
How Do We Deal With Complexity?
Design Is the Enemy
Complex systems cannot be designed in the same way non-complex systems can. In trying to build truly complex systems such as those we find in nature, where the internals are too complicated to be comprehensible or reason about, McClure suggests we must go outside of the system. We must go “Meta” in order to build truly complex things which manifest emergent outcomes and behavior that align with what we want. And we only know that we’ve built what we want when we observe those emergent properties manifesting via intuition and heuristic driven trial and error.
For example, to manipulate the behavior of A.I. models such as Neural Networks, the hyper-parameters and meta properties that are outside of the system are designed and adjusted, but not the inner workings of the system which are far too complex and numerous to manage. How could you plan out or “design” a system with the right settings for billions of parameters? You can’t. The right settings emerge via iterative self organizing processes such as Reinforcement Learning, for example using multiple LLMs recursively (“Meta-LLMs”), or via Gradient Descent, which is an iterative automatic process that “finds” the right weights and settings and can be driven by hyper-parameters such as the Learning Rate.
Heuristics and Intuition
When dealing with true Complexity, what is more effective than pre-planned design is impassioned trial and error in combination with heuristics and intuition. Heuristics are built in “short-cuts” we use for making decisions in the midst of complexity. For example, if you see a panicked crowd running in one direction, you don’t stop to analyze what they are running from. You make the heuristic decision to run with them.
Though it’s worth considering how heuristics can sometimes lead us to make the wrong decision, McClure points out that they are probably hard-wired in us for a reason. They have stood the test of time, are built on deep-ingrained instinct, and therefore should be drawn upon in the context of working with Complex systems.
It is an interesting argument for favoring the use of what scholar Daniel Kahneman, in his book Thinking Fast and Slow, calls “System 1” as opposed to “System 2” (a more methodical, analytical and slow approach to problem solving). Truly complex problems found in things such as driving in heavy traffic or difficult terrain, negotiating with other people, or spontaneously improvising in a jazz trio are not solved via a slow, rigorous and analytical manner. They are solved “immediately” by using ingrained, time tested instincts whose purpose is to deal with such complexity in the first place.
Though it is considered more rigorous and scientific to use formal analytical methodology and Mathematical formulas or reasoning to solve problems, McClure makes the point that Mathematics may just be a residual of phenomena. Even famous scientists such as Einstein remarked at how much more intuition played a role in their problem solving and scientific discoveries.
Another example is James Clerk Maxwell, who formalized Electro-Magnetic Theory using Math, but was inspired by his precursor Michael Faraday, who was much less “rigorous” and instead used naive intuition driven experimentation and trial and error to first discover electro-magnetism.
One of my favorite phrases in the book was “fundamentals do not precede application”. Taking music theory as an example, the rules and fundamentals of voice leading which many great composers such as J.S. Bach used, were discovered, not designed, in that they were emergent properties that musicians noticed over time via trial and error as they let their ear and musical intuition lead them to recognize configurations of notes which sounded more harmonious than others.
Is Software Engineering Complex?
In my previous blog post on Chaos Theory, I speculated that software development unfolds in what could be compared to a Chaotic System. I’m not so sure about that anymore since there may be some technical nuances that exclude the practice from falling into that category, but I am convinced that it is Complex. We are dealing with teams of people (already that is complex enough) and changing requirements that emerge based on feedback from the network of customers, users and stakeholders. This is in addition to a dynamic technology space where tools, security patches and paradigms are constantly shifting. All of these parts are interacting and affecting each other in this system.
Ideally, unless we’re talking about an LLM or Neural Network, we want to eliminate stochastic processes or behavior in the software itself and make our software systems deterministic. The software itself should not be Complex, but software development sure as heck is.
Thoughts on how some of the ideas from McClure’s book could be related to everyday software engineering:
It already is considered and known by some to be an effective practice that stories or behavioral goals of a system should not be specified to great detail. Only the desired outcome and behavior at a higher level should be specified, and the inner details of the implementation will emerge, so to speak, as developers iterate and actually do the work.
Also, frequent quick feedback loops (communication with stakeholders and review of the software in its current/intermediate state) in an iterative fashion can allow us to continually react to emergent changes of user preference and new desires of the software’s behavior or changing customer goals as all the “parts” (i.e. people, products, environment, etc.) interact during the process of software development.
This is designing the system from the outside looking in, in that you allow a complex system to work – being a team of software developers along with changing dynamics and stakeholder preferences that interact and evolve – and react to emerging properties from those interactions to move towards a ‘meta-goal’. NOT trying to plan it all out, NOT trying to design every specific, but discovering the ideal emergence of the software and goals aligning.
Final Thoughts:
The key takeaway for me is to have humility. To realize that when we are dealing with truly complex systems, we can’t predict their behavior or control them as much as we’d like, and so must accept this and use it as a starting point.
The other key insight was the suggestion that truly complex systems should be approached “from the outside” at the Meta level. We need to let go of wanting full control of all the internal details and focus on facilitating the emergence of desired properties of the system instead.
A lot of real life and business problems are complex in this sense. While it may be somewhat useful to have a “plan”, or chart and try to control internal metrics (“Velocity”, etc.), I think that the reality is that these measurements only tell part of the story and may even be misleading in some cases.
An alternative approach is paying attention to the feedback from working with or building complex systems, or watching for evidence that signals to us that we are dealing with Complexity. Stepping outside of the system, making judgements based on properties we see emerging and using that to verify or adjust course instead of following a design or plan, may be how we need to deal with true Complexity.
Further Resources:
- Discovered, Not Designed (Building Things in the Age of Complexity) by Sean McClure – This is the book that in part inspired this blog post.
- Faraday, Maxwell, and the Electromagnetic Field: How Two Men Revolutionized Physics – Mentioned in this blog post, this recounts how Faraday and Maxwell discovered Electromagnetism. The book gives a great account of how Faraday used his curiosity and intuition to discover the phenomenon and how Maxwell formalized it into Mathematical formulas.
-
Einstein: His Life and Universe – A biography on Albert Einstein and touches on how he relied on intuition and imagination for complex problem solving.
- Thinking, Fast and Slow by Daniel Kahneman – This is a highly-regarded book on heuristics and decision making exploring how we think (by both using intuition and analytical approaches).
- Chaos Theory in a Nutshell – My previous blog post covering the main ideas of Chaos Theory, which is a sub-domain of Complexity, and how it can relate to software engineering.