What is a Chaotic System?

Chaotic systems in a nutshell are deterministic systems that you would think are predictable, but wind up producing wildly unexpected output. The weather is an example of a chaotic system and the famous colloquial example of chaos is The Butterfly Effect, for example: a butterfly in China can flap its wings which eventually causes a hurricane in New York.

Key Terms:

• Iteration: Doing the same thing again and again using the previous step’s output as the next step’s input. Note: Iteration is not repetition. In iteration, the output is used as the next input whereas in repetition, this does not occur.
• Seed: the initial condition or input to a dynamical system at the start of an iterative process.
• Orbit: Also known as an Itinerary, it is the various outputs in sequence starting from iterating from a seed.  You can plot and graph these outputs as they change over time and get insight into the behavior of various systems. Chaotic orbits become unpredictable and erratic on the plot.
• Period: A repeated cycle or pattern in an orbit. Periods can have a length or duration, for example,  an iterated function that produces 2 values that oscillate back and forth as iteration continues has a period of 2.
• Dynamical System: A system where some variable or set of variables changes over time. Iterated functions whose outputs change over time are one type of a dynamical system.

Chaotic systems have four main properties:

• Sensitive Dependence on Initial Conditions: This means that if you change the starting condition of a system by a little bit, it will produce a much different outcome.
  • Think of a product line conveyor belt, for example, where you set the speed to 2.1 feet per second vs. 2 feet per second. You would think increasing the speed by 0.1 ft./s would produce a proportional increase (i.e. 5%) in output from the line.  But, imagine that after doing this you get a surprising 37% increase in output.
  • So a small change in the starting condition produced an unpredictable result compared with that of another miniscule different starting condition.
  • To further illustrate the importance that iteration plays in these dynamical systems, imagine that as each product gets to the end of the line, it is fed back into the start of the same conveyor belt system, and at the end, that resulting product fed back in at the beginning again, and this cycle is repeated (iterated).
• Bounded: There are boundaries in the system output. The conveyor belt speed cannot be increased to infinity, for example.
• A-periodic: There is not a constant repeated pattern so that you can predict how the system behaves long into the future.  The orbit or itinerary of the system does not contain any periods or repeated patterns going forward in time.
• Deterministic: This means that if you plug in the same inputs to a system, you get the same outputs. So imagine that if we placed an unrefined product on the conveyor belt, we would always get the same refined product at the end (given the same unrefined product).

Behavior of Chaotic Dynamical Systems and Universality:

One of the most interesting things about Chaos Theory is that amidst the chaos in these systems there is also a strange order and universal behavior between systems. For example, something known as a bifurcation occurs as systems transition from ordered to chaotic states.

An example of this is a dripping faucet. As you slowly increase the flow speed of the stream, the drips will predictably start falling twice as fast, and then as you increase the flow more, twice as fast as that, and the rate continues to double until the rate of the drips are unpredictable and go into a chaotic state. This is measurable and found in many chaotic systems from unrelated domains as a consistent observed behavior.

Feigenbaum’s Constant:

What’s even more interesting and strange is that as these doubling periods increase, their proportional duration to each other will start to remain constant (a ratio known as Feigenbaum’s constant which is about 4.669). So the length of a doubling period will constantly wind up being 4.669 times the length of the following doubling period and this continues until the chaos state comes about.

Implications of Chaos Theory applied to Software Engineering:

Planning and Estimating, i.e. Waterfall vs. Agile:

One recurring property that exists in a lot of chaotic systems is that their trajectories (or orbits) can be predictable in the short term if compared with the trajectories of the same system with slightly different starting conditions.  But, these trajectories will diverge wildly as time progresses.  So, given this observation, the key in reliably predicting the behavior of these systems is to focus on short term timelines (consider that the weather, a chaotic system, can only be predicted reliably a relatively short time into the future).

Applying this idea to software development, this warrants caution against predicting or estimating outcomes in the distant future, considering the chaos and unpredictability that can unfold as time and development progresses.   Waterfall planning, for example, should be avoided and the pitfalls of putting ill-invested faith in these long-term estimations are well known.

Likewise, putting too much faith into estimations using a previous project’s trajectory to predict an upcoming similar project’s trajectory is also not advisable.  If the development of a software system is chaotic, as I suspect it is, then a small difference in the upcoming project compared to the previous similar project could result in wildly different trajectories.  It reinforces the importance of short iterations and frequent feedback loops to reset our trajectory constantly to look at a short time interval into the future, so that the results of our efforts are somewhat predictable.

Iteration:

As Tom DeMarco puts it in his book, Structured Analysis and System Specification (an oldie, but a goodie!), the brain operates naturally and most effectively as an iterative processor.  The fact that complex dynamical systems found all around us and in nature can be built from simple iteration is further proof its effectiveness.  One of the pieces of advice about how to build and design complex systems in the book is to simply start.  Put something down on paper, even if it’s bad, and then iterate on it over and over again to continuously improve it little by little until it takes shape into a well formed and robust design.

We shouldn’t underestimate the power of iteration.  The results of iterated chaos games and the resulting fractals from iteration like the Barnsley Fern, for example, show us the elegant and complex output that can be produced from simple iteration.

I can also see a parallel with Test Driven Development (TDD), in that a simple iterative feedback loop (Red, Green, Refactor) can produce an emergent design as the process continues.  This also mirrors at least in part what happens in chaos games, like the famous one that produces a Sierpinski Triangle; An elegant fractal emerges by iterating over simple rules.

More technically speaking, the idea of the recursion and iteration of deterministic functions in software and their similar counterparts and use in analyzing dynamical chaotic systems is noteworthy.  The idea that the iteration of simple functions can produce infinite complexity (see the Mandelbrot set) is food for thought and is a reminder that the best way to build complex systems elegantly should be to try to use simple building blocks and compose them.

Deterministic functions are also important in the functional programming paradigm.  Even though there are some random and stochastic elements found in the rules for Chaos games, you could still think of them as being input to a deterministic function.

An interesting thought experiment might be to ask how one might build complex programs or systems in a fractal or recursive manner.

Final Thoughts:

One of the things I find profound about the study of chaos is that these systems are observed occurring in nature and in mathematics and suggest that outcomes are not necessarily pre-determined, or at least you can’t assume outcomes or predict them in a lot of cases.  Like Keynes’ observation that true probabilities can only be calculated in theory due to inherent uncertainty, Chaos Theory gives me a scientific excuse to be pleasantly surprised and optimistic in life.