JOURNAL: The Weaknesses of Highly Optimized Systems

I've written about the ways to attack or undermine systems with highly optimized tolerance in the past (systems that fit this description include everything from financial networks to electricity networks). Generally, my conclusion was that the best approach is to focus on design flaws. However, this analysis was less than complete. Systems that exhibit highly optimized tolerance are vulnerable to ALL of the following:

• Design flaws. These attacks come from outside the system (exogenous) variables. In warfare, the design of COIN (the US military's counter-insurgency doctrine) didn't account for open source insurgency. As a result, it was of little use in Iraq. Fortunately (or unfortunately, since it took four years of failure to grasp that the model was flawed) new methods/models of counter-insurgency were designed on the fly (based on local observations) that created the Anbar Awakening. In finance, we designed a system of national economic investment that relied nearly exclusively on capital markets (at the expense of individuals/incomes). The result was that capital markets became, without the decisions/input of hundreds of millions of Americans making daily investment decisions, completely divorced from reality.
  
• Exploitation. I touch on this in my recent brief on "Bow-Tie Control Systems." In short, its often possible for bad actors to access the core features/services of complex systems. This allows them to exploit these features for personal gain. If the exploitation becomes too intense, the entire system fails. In biology, cancer exploits the core machinery of the cell to propagate/function. In finance, investment banks accessed the mortgage market to exploit its cash flow for personal gain. In each case, too much success resulted in the death of the host system.
  
• Black Swans (events with small probabilities that yield exceptional or disastrous outcomes). Basically, this is what happens when a variable in your system's design (that you think you understand) blows up. Why does this happen? In most cases, we don't have nearly enough experience with almost all of our complex systems to truly understand the magnitude of potential variation in many of the variables we use (as in: hindsight doesn't work if the period of observation is too short). Let's use another example from Iraq (warfare). While we understood that terrorists could make things worse through targeted attacks , we completely under appreciated the magnitude of disruption possible. As a result, the attack on the Golden Mosque sent the country into civil war for a year. In finance, the historical probability distribution of defaults on home mortgages was completely wrong, by several orders of magnitude. The result was systemic financial failure as top rated assets quickly became worthless junk.

Comments

[moon]

You've laid out three peer categories of vulnerabilities, but the Black Swan vulnerability strikes me as second-order, derivative of the first-order design flaw and exploitation vulnerabilities. In the present crisis, wasn't the hard-to-compute attribute a direct result of the obfuscation and complexity of the exploitation? So, the failure to understand does not stem from a misreading of the historical probability distribution. Easy credit and lax lending standards had no precedent PDF.

I'm also pondering that the housing-bubble-cum-credit-crisis might be more of a grey swan. We'd to some extent seen it coming, but did not have the will / know how to go about mitigating it.

[atilla the pun]

i think the biggest problem we collectively face is scale - models/approaches that work at one size don't necessarily work at all multiples above/below - just look at lifeforms - an ant the size of an elephant is no more viable than an elephant the size of an ant - the current global wholesale finance crisis stems very much from this - i expect this theme to crop up repeatedly in my life, in fact, as long as human population on planet earth continues to grow.........we have have definitely "succeeded" WAY beyond the initial parameters for which we evolved - i do think we'll need serious aftermarket upgrades to help us (as a species/group) thrive from this point on, and those will come in forms of pharmaceuticals/genetic-engineering, but they will be pretty halting/lucky until we reverse-engineer the brain/nervous system.

so it goes

[Rogan]

The comparison of the actions of the collective decisionmakers at major banks to a cancer on the whole system is very accurate. What is unfortunate is that the whole sub-prime catastrophe was essentially created at the behest of the FED, which has, for all intents and purposes, abandoned the dollar. But one need look past the subprime issue - this will not be an isolated affair. There is the commercial real estate market, and then there is the consumer debt market. To contrast, banks held approximately 900BL in subprime mortgage backed paper; they hold about 915BL in consumer credit card debt. Once the failures begin affecting local communities - foreclosures, unemployment, implosion of the commercial real estate market, etc., it is altogether likely that consumers will begin to default on this credit card debt too. What we have seen in the past weeks is likely the tip of the iceberg; the beginning of a longer process of failures. On the bright side, there is no currency capable of becomming the next reserve currency; and Asia is so tied up with our debt that they have little choice but to continue the game - or suffer a fate worse than whatever we are about to endure. So the Dollar may well survive, though the definition of "survive" may need some tweaking... I don't agree that the subprime debacle qualifies as a black swan; this was a relative inevitability. The fact that the banks and the FED kept the game going so long - and continue to play - is not a testament to random events, but rather an indication of the wonton, craven desire to extract everything out of the market TODAY, with little thought for tomorrow. But the dollar may yet survive; as bad as we are gonna get it, the rest of the world will likely see worse.