In this In A Nutshell series, I’m going to be covering Complexity, inspired by recently reading a fantastic book by Sean McClure titled, Discovered, Not Designed (Building Things in the Age of Complexity).  The Complexity being discussed here is not the colloquial kind, but is referring to the modern definition which describes things that are more than the sum of their parts.  This in some ways dovetails off of my last post about Chaos Theory, which is a related topic. My goal is to briefly touch on the main ideas of modern Complexity along with how what I learned from reading McClure’s book lends insights into building and managing complex systems. I’ll also look at how some of those ideas can be related to the practice of software engineering.

Group Selection

One of the most important points made in “Discovered, Not Designed” was that Nature uses the full distribution.  What this means is that groups or the whole are the important “unit” of effectiveness, not individuals. You can’t think in terms of individual parts to the system in Complexity – only the group can be considered. For example, what makes an ecosystem is not “birds” or “rabbits”, but all of the components in aggregate working together as a group, from all corners of the distribution.

This is contrary to how we like to think about how a lot of things are distributed in terms of the Bell Curve, or the Gaussian distribution, where there is a peak which represents more important information compared to the smaller tails. In nature and complex systems, the tails are just as important as the peaks, so to speak, because the system is the emergent outcome of all of the parts interacting together – the full distribution.

How Do We Deal With Complexity?

Is Software Engineering Complex?

In my previous blog post on Chaos Theory, I speculated that software development unfolds in what could be compared in some ways to a Chaotic System. I am also convinced that it is Complex. We are dealing with teams of people (already that is complex enough) and changing requirements that emerge based on feedback from the network of customers, users and stakeholders. This is in addition to a dynamic technology space where tools, security patches and paradigms are constantly shifting. All of these parts are interacting and affecting each other in this system.

Ideally, unless we’re talking about an LLM, a Neural Network, or to a lesser extent a highly distributed asynchronous system, we want to eliminate stochastic processes or behavior in the software itself and make our software systems deterministic. The software itself should not be Complex, but software development sure as heck is.

Thoughts on how some of the ideas from McClure’s book could be related to everyday software engineering:

It already is considered and known by some to be an effective practice that stories or behavioral goals of a system should not be specified to great detail. Only the desired outcome and behavior at a higher level should be specified, and the inner details of the implementation will emerge, so to speak, as developers iterate and actually do the work.

Also, frequent quick feedback loops (communication with stakeholders and review of the software in its current/intermediate state) in an iterative fashion can allow us to continually react to emergent changes of user preference and new desires of the software’s behavior or changing customer goals as all the “parts” (i.e. people, products, environment, etc.) interact during the process of software development.

This is designing the system from the outside looking in, in that you allow a complex system to work – being a team of software developers along with changing dynamics and stakeholder preferences that interact and evolve – and react to emerging properties from those interactions to move towards a ‘meta-goal’. NOT trying to plan it all out, NOT trying to design every specific, but discovering the ideal emergence of the software and goals aligning.

Further Resources:

• Discovered, Not Designed (Building Things in the Age of Complexity) by Sean McClure – This is the book that in part inspired this blog post.
• Faraday, Maxwell, and the Electromagnetic Field: How Two Men Revolutionized Physics – Mentioned in this blog post, this recounts how Faraday and Maxwell discovered Electromagnetism. The book gives a great account of how Faraday used his curiosity and intuition to discover the phenomenon and how Maxwell formalized it into Mathematical formulas.

• Einstein: His Life and Universe – A biography on Albert Einstein and touches on how he relied on intuition and imagination for complex problem solving.

• Thinking, Fast and Slow by Daniel Kahneman – This is a highly-regarded book on heuristics and decision making exploring how we think (by both using intuition and analytical approaches).
• Chaos Theory in a Nutshell – My previous blog post covering the main ideas of Chaos Theory, which is a sub-domain of Complexity, and how it can relate to software engineering.

In this In A Nutshell series, I briefly summarize and break down the main ideas in Chaos Theory, which is the study of how deterministic systems can produce unpredictable results and is also considered one of the “complex sciences”.  It’s a subject that I’ve become fascinated with in addition to fractal geometry, and thought it would be interesting to see how these ideas can be related to software engineering.

Building Complexity from Simple Components:

FURTHER RESOURCES:

• Chaos and Fractals: An Elementary Introduction: This blog post was based on concepts introduced to me in this great book on Chaos Theory and Fractals by David Feldman. Highly recommended as a good introduction to the domain of Chaos Theory and in addition some fascinating information on fractal geometry is provided as well.

• Check out my GitHub repo with demonstrations of things like the Chaos Game implemented in Python.

What is clean code?  How do you write clean code?  I will attempt to address these questions in this “In a Nutshell” series blog post.

Clean Code In a Nutshell:

• The crux of it is: Clean code is code that is written in a way that is easily understandable, readable and maintainable by other Humans.  It is based on empathy and consideration for other readers of your code.
• Key aspects of clean code:

1. It is DRY (does not repeat unnecessarily, more specifically: a piece of knowledge in the application is only in one place).
2. Concise (but not too concise).
3. Tells a story to the reader in a logical sequence about what is happening in the code.  The code expresses the author’s intent clearly and is easy to understand and read.
4. Incorporates the SRP principle, Separation of Concerns and is de-coupled.  This makes it easily maintainable (modular).  Whenever a change must be made, changes to one module will not unexpectedly break or change another module’s functionality.
5. It does not make the person who is reading and working on it want to pull their hair out.

That’s basically what it means to write clean code.  For further explanation and examples, continue reading…

Definitions:

• Code Smell: This refers to code that is not clean and “smells”.  There is something about the code that is unnecessarily confusing, unclear or something that could be improved upon to make it cleaner.
• DRY: The “Don’t Repeat Yourself” principle.  The goal is to eliminate as much repetition of code as possible and re-use as much code as possible, which keeps things less cluttered and clean.  Just keep in mind this doesn’t necessarily mean you should literally never duplicate any code – the most important principle is that specific knowledge is only in one place.
• SRP: The “Single Responsibility” principle.  A method, object or class should do one thing and do it very well.  This makes things easier to maintain and keeps things de-coupled.  Another way to put it is that a module should have only one reason to change (or alternatively, only one agent/person/department should be interested in asking it to change).
• Separation Of Concerns: Code should be logically grouped based on what operations it is performing or data it is related to.  Methods in classes and objects should be closely related to each other and separated.
• Spaghetti Code: This is code that is not separated logically and is not modular.  It is very difficult to maintain and understand and is usually heavily coupled (the code is connected to and affects many other parts and operations in the application) so that changing one line may have unforeseen side effects on all sorts of other parts of the code base.  It is a nightmare to deal with and the opposite of modular, de-coupled clean code.

METHODS FOR WRITING CLEAN CODE (and what to avoid doing):

NAMING VARIABLES:

• Names for variables should not be ambiguous, but be descriptive and meaningful.
• Magic Numbers and Magic Strings: Strings or Numbers that have some meaning that is not obvious to the reader of the code.  Always label these values with a descriptive variable name.  Name them so that it’s clear to the reader what those variables should be used for or what they represent.
• Make naming formats consistent.  If related variables are in camel case, then make sure all of them are in camel case, or if there is a word that is appended or prepended to related variables, make sure you keep using that format.
• Follow naming conventions in whatever language or framework you are using.  For example, in C# the names of classes, it’s properties and methods use Pascal Case while private fields, method parameters(arguments) and local variables use Camel Case.  In React, all component names and classes should have the first letter capitalized.

Examples:

// Avoid: Ambiguous variable names:
const x = "Please log in.";
// Cleaner:
const loginMessage = "Please log in.";

// Avoid: Magic Numbers and Strings:
if (userStatus === 1) {
  return true;
}
// Cleaner:
const LOGGED_IN_STATUS = 1; 
if (userStatus === LOGGED_IN_STATUS) {
  return true;
}

// Avoid: inconsistent naming formats:
const username = "Brent";
const user_profile = {...};
// Cleaner:
const userName = "Brent";
const userProfile = {...};

METHODS, OBJECTS, CLASSES (SRP and Separation of Concerns):

• Keep methods short and have them do only one thing!  Methods should do one thing and do it very well (have a single responsibility; They should only have one reason to change!  A method should not contain both view layer logic and persistence layer logic, for example.).  There are differing opinions on how long methods should be, but the goal is to make them as short and as contained as possible.  Some say they should not be more than 10 lines of code.   Just try to make them as short as possible without making the code hard to understand.
• If methods get too long, look for ways to extract parts of the code to a separate object or function that you use inside the method.
• Use Abstraction.  Abstract away the complexity of a complicated piece code by placing it in it’s own function with a descriptive variable name.  Create Black Boxes that can be used simply without needing to know their inner workings and implementation details.
• Cohesion Principle: All code that is related closely should be together and code that is not should be separated.  Objects and classes should contain data and methods that relate to each other and only have to do with the data or operation that the object/class is concerned with (Separation of Concerns).
• The Information Expert Principle (or similarly the Encapsulation principle) states that a method should be attached to the class or object that has the related data which it uses (similar to the Cohesion Principle).
• The number of parameters should not exceed three.   You can extract parameters and batch them in objects or encapsulate them to reduce the number of arguments.  If possible, also avoid using booleans as parameters.
• Variables should be declared as close as possible to where they are being used so the reader does not have to search around to find where they are defined.

Examples:

// Avoid: long methods and functions with too many parameters and that have more than one reason to change.
function fetchUserDataAndLoginUser(loggedIn, userId, db, userProfile, userPassword) {
  let userData = {};
  if (!user.loggedIn) {
     ... 
     lots of code to login user on the back end
     ...
     ...
     ...
     userData = db.find({ user: user.id });
     ... 
     lots of code to load userData into a profile view
     ...
     ...
  } else {
 window.location.replace("https://site.com/dashboard"
);
  }
}

// Cleaner (separate the operations into separate functions to use and combine the arguments into a single encapsulated object):

const user = {
   userId,
   userProfile,
   userPassword,
   loggedIn 
};

function authenticateUser(user) {
  ...code for checking passwords from input and authenticating the user.
}

function fetchUserData(userId) {
  ...code for fetching authenticated user data from the database
}

function renderUserData(userData) {
  ...code to render user data on the view.
}

authenticateUser(user)
  .then(user => {
    fetchUserData(user.id);
  })
  .then(userData => {
    renderUserData(userData);
  });

// Note how the code now tells a story in a logical succession to the reader and each method only has one responsibility.  The user is logged in and authenticated (auth layer), then their data is fetched from the database (persistence layer), and then the data is rendered to the view (view layer).  
// The code is also easier to maintain because if there is a point of failure along the way, the developer can look in the appropriate method dealing with the failing operation and pinpoint and debug quickly and easily which step is failing.

CONDITIONALS (IF/ELSE STATEMENTS):

• Keep conditional statements as flat and easy to understand as possible by using refactoring techniques and avoiding common pitfalls.
• A Nested Conditional is an if/else statement contained within another if/else statement.  Avoid them: they are difficult to read, understand and test.
• There are circumstances in which nested conditionals can be refactored by considering the relationship of the conditional parameters between the statements.  (See examples below).

Examples and ways of refactoring conditionals:

Use a Ternary Operator:

// Not nested, but easily refactored to make the code cleaner and more readable:
  if (user.id) {
    userLoggedIn = true;
  } else {
    userLoggedIn = false;
  }
// Cleaner:
userLoggedIn = user.id ? true : false;

// NOTE: Never combine ternary expressions and only use one - multiple ternary expressions are too difficult to understand and read.

If the statement sets a Boolean to a variable based on the condition, you can simplify it by setting the variable to the condition in a single line eliminating the if/else statement:

// Avoid: Unnecessary if/else statement:
if (userLoggedIn) {
  showProfile = true;
} else {
  showProfile = false;
}
// Cleaner: showProfile is simply dependent on the value in the condition...it can be refactored to:
showProfile = userLoggedIn; 

Watch out for nested if statements that a single block is based on. Use the && operator or early exit technique to refactor:

// Avoid: nested if statement inside a single block:
if (user) {
  if (user.id) {
    ...code
  }
}
// Cleaner: refactor using the && operator to eliminate the nested if statement:
if (user && user.id) { ...code } 

// Or use the early exit technique to return if a or b is not true to eliminate nesting:
function verifyUser(user) {
  if (!user || !user.id) {
    return;
  }
  ...code to run if user and user.id are true
}

Refactor nested if statements that have repeated conditions to combine them with logical operators:

// Avoid: Nested conditionals with identical conditions:
if (userIsGoldMember) {
  if(itemsInCart) {
    applyDiscount = true;
  }
}
if (totalCost > 100) {
  if (itemsInCart) {
    applyDiscount = true;
  }
} 
// Cleaner: combine the identical conditions and variant conditions with logical operators to eliminate the nested conditional:
applyDiscount = (itemsInCart && (totalCost > 100 || userIsGoldMember));

Make conditions more expressive by extracting the condition to return a boolean in a separate function:

// Avoid: conditions that are complicated and not easy to read quickly:
if (Date.now() >= (new Date(2018, 11, 25).getTime() + 24 * 60 * 60 * 1000)) {
  ...code
}

// Cleaner: separate the condition into a function that returns true or false and describes what the condition is clearly:
const isNowLaterThan24HoursAfter = (datetime) => {
  const twentyFourHours = 24 * 60 * 60 * 1000;
  return (Date.now() >= (datetime + twentyFourHours))
};

const deadline = new Date(2018, 11, 25).getTime();

if (isNowLaterThan24HoursAfter(deadline)) {
  ...code
}

Refactor Triplicate conditionals: A triplicate occurs when one of multiple conditions must be true.  Use the || operator instead of if statements:

// Avoid: separate if conditionals for a triplicate:
if (condition1) {
  return true;
}
if (condition2) {
  return true;
}
if (condition3) {
  return true;
}
return false;

// Cleaner: use the || operator:
return condition1 || condition2 || condition3;

COMMENTS:

• Comments should not be redundant or stating what is obvious in the code – it just makes more clutter in the code base.
• Ideally, you shouldn’t need comments since the code you are writing tells a story and it is clear and easy to understand what is going on.  Code should be self documenting for the most part. There are differing opinions on this and I think it is a good rule of thumb to follow most of the time.  In reality, there may be a situation where comments would be very helpful to the reader (i.e. when the code is written in an unintuitive way that is necessary for some reason or refactoring would cause breaking changes).
• If you need to write a comment, don’t write comments addressing “whats” (what the code is ‘saying’, so to speak), write comments that address “whys” and “hows” (why this code is the way it is or how it is doing some operation).
• Minimizing comments is also a good idea because developers may change logic in the code base, but not update the comments which can cause confusion in the future.
• The exception to leaving out comments is the case of making a `TODO` note;  These are good comments when a problem in the code is found and a note needs to be left for an issue that needs to be fixed and addressed.  Ideally, however, TODOs should be fixed on the spot.
• If you see comments for blocks of code describing operations in a method, that is a sign that it should be refactored and the operations should be extracted into separate methods.

FURTHER READING AND RESOURCES: 

• Clean Code: A Handbook of Agile Software Craftsmanship – a classic book on principles of writing clean code.
• C# Developers: Learn the Art of Writing Clean Code – Udemy course by Mosh Hamedani which goes over some of the principles from Clean Code.  Even if you don’t know C# I highly recommend this course as the principles are explained very clearly and they transfer to any programming language.
• Bob Martin’s lectures on YouTube – If you don’t want to invest in reading Clean Code, or you want to sample the ideas before diving in, this is an excellent lecture series that covers the main concepts of what makes code clean and easy to read.