Dangers of a Data Monoculture

For almost a year I have been trying to alert readers to the strong possibility that  organizations charged with estimating future energy usage are consistently underestimating totals. I think the source of their error lies in miscalculating the take-up of energy in the developing world, compounded by their ignoring current latent demand and, perhaps most importantly, their stubborn refusal  to acknowledge that energy demand is not very elastic. It’s the last bill to go unpaid, so to speak.

Because of these errors, I think they have underestimated short term energy demand (through 2030) by a third. And I’ve spent a lot of time here showing why and discussing the possible consequences.

How do organizations like the U.S. Department of Energy’s Energy Information Administration, the International Energy Administration, World Resources Institute, the IMF, the World Bank and companies like Exxon, BP and others who produce energy estimates get tied up in knots about things like this?

I think the answer, implied by the title of this post, is fairly easy to illustrate and explains a lot of problems with forecasting both within and outside the energy sector.

Take the DOE’s EIA. They spent a lot of time and energy developing a model of future growth of energy. I’m sure they were diligent and thought hard about it. (I think the defect is that it’s based on energy supplies, not demand, and that it thinks that people will quit using energy if it gets expensive.)

But the problem with their recent forecasts lies in their model-dependent analysis. After spending all that time and energy building their model, obviously they’re going to use it a lot. Sadly, it seems that they’ve tunneled their vision onto the model to the exclusion of a lot of data in the real world. (This is then exacerbated by the confirmation bias of looking at friendly analysis of the same sources by other organizations and feeling relieved when they arrive at similar results.)

Model dependency crops up in other areas as well. Economics, climate change–both are examples where tunnel vision is hurting analysis. We saw in my recent post that nobody had published a simple mash-up of CO2 emissions and recent temperature trends and my modest posting of the two together got quite a reaction. In finance, many of the great and the good seem determined to ignore Nassim Nicholas Taleb in charting paths to economic recovery, with one side making the (almost forgiveable) error of wanting to adopt one-half of the Keynesian prescription (deficit spending for investment in the face of a liquidity trap) without making a good faith commitment to practicing the other half (creating a budget surplus) when times are good.

The other side (the ECB and U.S. Republicans) are making the far more grievous error of looking at deficit numbers in isolation, thinking that the gross totals and their increase are reasons to abandon social programs and reduce debt at any cost. It makes me wonder if any of them have ever had a mortgage on a home. The cost to borrow money for governments with floating currencies has never been lower and the U.S. and the UK in particular should be spending their way to recovery. It will be tougher for the Eurozone countries, but they need to find a way.

In climate change discussions both activists and skeptics have found a comfort zone of data they are willing to use to advance their arguments. Activists like models, as real world observations are not exact enough to help them make their case. Skeptics like statistical rules and laws, which highlight the deficiencies and call arguments into question. Neither side has spent enough time examining the sources of data used by the other team. Activists still, in the waning days of 2012, show a surprising naivete and ignorance about statistics, while skeptics stubbornly refuse to acknowledge that models can be useful, if users keep their limitations in mind and a copy of The Black Swan on their desks.


The other fields that suffer from the same tunnel vision–healthcare, gun control, genetically modified organisms and their utility/safety or lack thereof–tend to comprise, with those examined above, most of the things we fight about.

I don’t consider that surprising. I don’t consider it unintentional. I do consider it as potentially fatal to the cause of solving any of the real problems that confront modern society.

5 responses to “Dangers of a Data Monoculture

  1. You misunderstand the nature of the climate debate, one side does actually spend most of their time examining output from their opponents. It is not a battle of competing theories but primarily a presentation of a single theory with reactive response by skeptics, As such skeptics pretty well spend all their efforts on examining the endless stream of presentational output and dodgy data from the opposing team.

    By contrast activists have such confidence in their theory and (especially) belief that they often do not ever glance at opposing arguments. This is not just confined to climate science, the same mission, looseness with the actuality and belief that the solution is regulatory control of others is apparent in other environmental, healthcare and political areas as well.

  2. “For almost a year I have been trying to alert readers to the strong possibility that organizations charged with estimating future energy usage are consistently underestimating totals”

    The 6th Northwest Power Plan is a useful read.

    Click to access SixthPowerPlan_Ch3.pdf

    There is ‘projected demand growth’
    Then ‘frozen efficiency improvements’. I.E. Improvements that will take place as a result of already existing economic incentives.

    In the Northwest a small portion of our electric bill is dedicated to ‘conservation’ efforts.(4/10ths of a cent/KWh). The utility can ‘target’ a lot more efficiently then the government.

    They can also ‘leverage’ various commercial needs to get ‘more bang for the buck’. I.E. Make a deal with the local Lowe’s to have a ‘free giveaway’ of CFL’s(limit 4). Lowe’s gets customers in the store and the utility only pays Lowes for the wholesale cost of the bulbs, which of course they got from the manufacturer at a knock down introductory offer price.

    Then ‘price effect’ improvements…I.E. As the price of electricity slowly edges higher then the ‘economic case’ for additional efficiency improvements increases.

    The resulting projection is pretty close to no growth in electricity consumption for the next 20 years. Some change in peak loads due to increased air conditioner use but western Washington only really has spring and fall.

    From where I’m sitting the US EIA projections are high…but then I live in a state with an energy conservation program that is both politically popular and effective.

  3. Hiya Harry–and in case I forget, happy holidays!

    The times I’ve looked at U.S. energy consumption in the same way I look at the developing world, I see medium sized growth due mostly to population increase and a rise in GDP. I haven’t broken it down by region–my crystal ball gets too cloudy at that level of resolution.

    I drove through Western Washington this summer–it’s beautiful. Lots of room for more people fleeing the torrid drought zones of the Southwest, if that ever should happen… just as an example of the problems of regional predictions. Would they count as climate refugees if they don’t cross a national border?

    • Happy Holidays to you too Thomas,

      We don’t have much of any develop-able land left in Western Washington.
      Lot’s and lot’s of open space but a lot of it floods on a regular basis.
      Since flooding is normal folks from outside of the area generally don’t hear it about it on the news. Then a lot of the land is national park, a lot of it is in documented volcano mud flow zones. When Mt Rainier next decides to spew all that snow up there is going to melt in a matter of minutes, everything potentially in the way has been designated ‘not buildable’.

      My local city council just lifted a 100 year old 3 story height restriction. It’s going to be 18 stories now. ‘New Urban’ is going to be about the only thing that can be built.

      As far as regional variation in electricity consumption I would think Texas will be the big wildcard. Anecdotal evidence is that ‘New Urban’ is doing quite well in Austin but I don’t know if that is enough to offset the endless growth of big houses with a big pool and 4 car garage around Dallas.

      If I look at how much is being spent on conservation in Texas($135 million) it’s absolute chump change considering the size of the population.


      The penetration of heat pumps in Texas is quite high however, so there is a potential for savings if incentives(market forces or otherwise) to replace with the ‘most efficient’ heat pump/air conditioners become available. Hoping that Texas would adopt ‘substantial’ energy conservation programs in the near term is probably borderline delusional. 😉

      But then if I look at their ‘grid reliability’ reports they are probably going to have some unexpected rolling black outs in the next few years that might nudge them a little bit to invest a bit more in ‘demand management’.

  4. Good reeading this post

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s