Putting the ‘estimate’ back in estimates

M. Granger Morgan wants to bring uncertainty back to energy forecasting.

Morgan, a professor at Carnegie Mellon, has spent his career investigating climate change and the energy system and understands how difficult it is to account for the many variables that can affect gas and oil prices even 10 years into the future.

Scientists routinely account for uncertainty in their estimates, but often the uncertainty is lost by the time reports and analyses make it to decision makers. There are a variety of reasons for this, Morgan said, among them the idea that precise forecasts make for more persuasive arguments and the belief that people can’t handle uncertainty.

“I start talking about this and, ‘Ah, well, people can’t deal with uncertainty,’” Morgan said. “Well, people don’t get it quite right, but they do basically understand there is uncertainty.”

People regularly manage weather forecasts that are accurate a few days into the future but grow rapidly fuzzy after that. They know about gambling odds and the uncertain predictions of sporting contests, Morgan pointed out.

But when it comes to talking about the future of the energy systems that underlie climate change, government agencies, economists, and computer modelers routinely discount the enormous uncertainty inherent in their predictions.

“My assertion is that decision-makers probably, if presented with probabilistic forecasts, could deal with them,” Morgan said. “So why is it that most government ministries, many government agencies and ministers, many economists, and a wide variety of modelers persist in single-value forecasts?”

Morgan offered a few examples of past forecasting that badly missed the mark, including coal prices on delivery to electric generating plants, which repeatedly predicted a rapid price increase that never materialized; primary U.S. energy production in 2000, predicted between 1960 and 1980, which also missed by a wide margin; and the future cost of U.S. gas and oil prices.

“Nobody in their right mind can predict U.S. gas and oil prices, plus or minus 50 percent, within 10 years, let alone 50 years, but government agencies, many policy modelers, and economists do it all the time,” Morgan said.

Morgan spoke Wednesday at Mallinckrodt Laboratory as part of the

Harvard University Center for the Environment’s (HUCE) Future of Energy lecture series. Morgan, the University and Lord Chair Professor of Engineering at Carnegie Mellon and head of its Department of Engineering and Public Policy, was introduced by HUCE Director Daniel Schrag, Sturgis Hooper Professor of Geology and professor of environmental science and engineering.

Uncertainty in forecasting, Morgan said, isn’t due to incompetence, but rather the difficulty in predicting which direction many variables will take. A forecast’s accuracy is affected by, among other factors, random physical processes, policymakers, new technology, and the aggregate effects of choices made by many smaller players.

Morgan looked at a variety of possible strategies that modelers could use to incorporate uncertainty, each of which has strengths and weaknesses. Boundary analysis, for example, examines high and low potential values, but they diverge so rapidly over time that they result in forecasts too imprecise to be useful. The use of specific scenarios that outline possible outcomes is potentially useful, but often forecasters do not make a statement about how likely a particular scenario might be. Another possible strategy, he said, is to make models that get progressively simpler over time. This would be counter to the current trend to make models more complex to mirror reality, but would reflect decreasing certainty as one looks further into the future.

Consumer behavior alone can make a big difference in energy consumption, Morgan said, outlining an experiment by colleagues that, in collaboration with a local utility, monitored household energy consumption using smart meters. One group served as a control and the other group received no new technology, educational programs, or admonishments to conserve, just postcards saying they were part of an energy consumption experiment, followed by reminders.

Even though the postcards included no instructions to save energy, consumers altered their behavior and did so, a change that highlighted the importance of understanding behavior when thinking about energy efficiency’s future. The effect vanished when the experiment ended.

“We believe behavior issues on the side of energy consumption are so important that … we recently put together a group of three Ph.D. psychologists in the department called our Energy Behavior Group,” Morgan said.