Errors of Prediction

We live in a world that is obsessed with prediction. Since the dawn of humanity, we have longed for predicting the future. We did this using the stars, bones of dead animals, tarot cards, and even visions and dreams we experienced. Yet, apart from pseudoscience, humanity first gained the powerful, reliable tool of prediction with Newton’s laws of motion and has ever since been applying lessons learned from Newton in order to uncover what the future might hold.

If we can predict future events, like the trajectory of a projectile (say a cannonball), with such precision, then why not that of a larger or more complex object like the Earth itself? What about comets or stars? If celestial bodies are in fact governed by universal laws, would we ever be able to find laws that govern our mundane lives—laws that govern weather, health, politics, wars, traffic, natural disasters, the stock market, etc.?

Somehow, we can land a probe on a comet after a ten-year journey in space, but we are still unable to predict a storm’s behavior beyond a few days. We are still unable to predict epidemics or financial crises regardless of the quantifiable underlying systems. Why is that?

Most scientists claim that these errors are merely results of technical problems — we need better knowledge of initial conditions and faster, more powerful computers to treat complicated models. Similarly, weathermen, financial advisers, and pharmaceutical researchers, who make a living out of their predictions based on the current models, have no interest in admitting errors that go beyond the technical aspect.

Recently, Irma and Harvey, two of the most destructive hurricanes in the history of the United States, resulted in damages that are expected to exceed $150 billion. Yet, with all the acclaimed capacities in weather forecasts for weeks ahead, experts could not predict the hurricanes’ formations or their accurate paths. One has to wonder why we cannot predict such disasters.

Since the 1960s, experiments in weather prediction have shown great sensitivity to initial conditions as described in chaotic systems. Errors in such systems grow exponentially but remain limited; this means that a slight measuring or rounding error, say of the third decimal, can result in a completely different prediction regarding the outcome of, for example, a hurricane’s path (look up Lorenz attractor for more information).

Such an error cannot be ignored in weather forecast since reports on models give an average error doubling time (the time needed for an error to double) of 1 to 5 days. Working with chaotic models, weather forecast is based on a probabilistic approach. Perturbations in the initial conditions are made resulting in different predictions that are averaged together; for instance, determining probability of rainfall on a given day in a particular location. With millions of variables and endless ways of perturbations, this becomes exceptionally hard to implement and harder to interpret. That is why experts who work on weather prediction aim at achieving more precision by employing more variables, only for the models to grow more chaotic.

Yet some experts argue otherwise. Mathematician and writer Dr. David Orrell claims that the error lies mostly in the model used and not in observation. He explains that observations are currently made on a grid of 40kms x 40kms x 1km, averaging and interpolating variables. However, in Orrell’s view, there are two problems with this approach: the first is that a cloud, for example, lying inside that grid will be averaged out in the model without taking into consideration that it may contain dust, smoke, sulfuric acid, chemicals, etc., all of which are major factors in the condensation that yields to rainfall. The second problem is that temperature (particles’ average motion) and pressure (air’s weight) are, by definition, averaging variables and cannot be applied except by averaging out the elements in clouds.

For Orrell, if weather forecasters want better predictions and finer grids to improve their observations, they have to take into account the cloud modeling–a nearly impossible task–adding parameters and variables to the model. This results in an explosion of variables added to the model, increasing its complexity and thus bringing down the level of accuracy.

Orrell’s experiments and models showed that the current modeling errors are greater in the first three days than the observational errors, suggesting that increasing the observation accuracy will only add more error to the weather prediction and that the solution lies in finding better models. Nonetheless, the current models are capable of giving general warnings, which gives them at least some value to scientists, governmental authorities, and affected citizens alike.

The second topic that Orrell treats is health. If we think about the fact that the combination of the two gene mutations BRCA1 and BRCA2 account for at least 20% of hereditary breast cancer cases and that the estimate of cumulative breast cancer risk by age 70 was reported with wide confidence intervals (BRCA1: 40–87%; BRCA2: 27–84%) [Mutations in context: implications of BRCA testing in diverse Populations. Gabriela E. S. Felix, Yonglan Zheng, Olufunmilayo I. Olopade], we can see the importance of making accurate predictions.

Take someone like Angelina Jolie, given her risk of developing both breast cancer and ovarian cancer (according to her doctors), she has undergone extensive treatment including a double mastectomy in order to help prevent any cancer from developing. So why not use such predictions all the time? This would make sense since biology, just like weather, seems to be a quantifiable system.

Recall that even before the discovery of genes and DNA, experiments by Mendel and Galton showed that an organism’s traits, such as height or pigment color, are inherited. This was followed by research on criminality and intelligence where people imagined a futuristic society where individuals would be allowed to breed based on how well they pass health and intelligence tests. Some fancied a deterministic basic process, thus dreamed of selecting the elite, which soon lead them to racist supremacist ideologies and fascism. Others believed in the total opposite and regarded human traits as a social construct that can be molded by the states; their fate was not much different.

In any case, away from politics, science aimed at deciphering the language of DNA in order to know what traits individuals would develop based on their genes and how that could help manage someone’s healthcare (as Angelina Jolie did). The complexity of such claims has developed over time, starting with trying to find a single gene responsible for heart disease (for instance), to finding a set of genes to ultimately adding other variables, environmental ones, such as stress levels, obesity, exposure to second-hand smoking, and many other examples including luck.

Moreover, not all genes get expressed as proteins; stress hormones or depression, for example, can control the expression of some of the genes; even radiation, if a person is a survivor of nuclear attacks, will play its role. All these findings suggested that there are endless factors that affect genes’ expression and that the deterministic approach to DNA was too simplistic.

 

 

 

 

 

In similar ways to weather forecast, many biologists aim at predicting health given a particular set of genes (initial conditions) and how the human body works (the model). But the problem resides in creating a model for living cells. As mentioned before, knowing the DNA sequence is not enough to determine cells’ structure or behavior (and that is obvious since all our cells share the same set of DNA but each cell is specialized; a brain cell and heart cell, for example, are very different even though contained within each cell is the full blue-print – all the set of genes that make up an individual’s DNA).

Thus, models must take into consideration the transcription from DNA into RNA, its translation into proteins, the combination of proteins, and other variables such as stress or smoking, the discrete random production of RNA, the dynamic process of metabolism, and the fact that “concentration” means nothing inside a cell (concentration, like temperature, is an averaging variable by definition). All this speaks only to the cellular level and does not take into consideration the complexity of the organism as a whole.

Again, like modeling a cloud, scientists who hope to know the initial conditions for better predictions will find themselves facing an explosion of variables and will need a new parameterization for the model. In other words, they might be better off simplifying the model. That does not mean the human genome project, among others, is a useless project. On the contrary, its findings will help us find correlations which can assist patients in making decisions in advance of any disease development for we all have a natural tendency to possibly be inflicted with.

I would venture to say that perhaps all human traits are as hard to model as those involved with cancer. Scientists have found genes for homosexuality, obesity, suicide, criminal behavior, aggressiveness, risk taking and plenty of others. But such claims were quickly debunked after finding that things are more complicated. Although some of us wish for certain traits to be totally determined by a set of DNA and others wish for them to be socially constructed (for reasons that can go from religion to social justice), science is beginning to tell us otherwise. It’s telling us that things are too complicated for the human mind and are almost always subject to both nature and nurture; and it never judges.

A third topic that Orrell treats is financial predictions. Lehman Brothers, one of the biggest investment banking companies in the United States, maintained a high rating of AA up until September 14th, 2008. It filed bankruptcy on September 15th giving way to the international banking crisis of 2008. How, with all the financial consultants and advisers, the Harvard MBA graduates, and the rating agencies, was such a crisis not predicted?

Some experts in economics assume that people are rational beings whose interest is to accumulate more capital, beings who know what they want and act rationally upon it. A Gaussian distribution for human behavior has been assumed in order to predict stocks. We often assume a perfect market, and that the forces of demand and supply will correct perturbations in the economic system.

Nowadays, we are completely aware of the falsehood of such assumptions. We know that human behavior is not always rational given that we are subject to endless biases and fallacious ways of thinking. We further know that distributions in this field are not Gaussian and have no reason to be, that the perfect market is an illusion, that instead of correcting the perturbation sometimes a positive feedback on either side of the couple supply/demand will result in a crash, and that humans change their behavior with new information they acquire, which makes observations nearly impossible.

In fact, plenty of theories are arising to mend the previous errors. Behavioral economics is a theory that has been proposed by psychologists and it tries to take into consideration the irrational aspect of human beings. Former trader and risk analyst Nassim Taleb tackles problems of uncertainty in financial predictions. Many financial experts admit that stock predictions work just as well as random guessing.

Of course, this should not come as a shock at this point when we are familiar with the staggering number of variables and parameters needed in such models to make accurate predictions. Again, in trying to make better observations, the variables will explode and we find ourselves better off averaging large systems, accepting general recommendations and warnings, and accepting a compromise in predictions needed.

We tried to look into each topic independently and found the complexity residing in it, but we neglected till now how intimately such topics are related and how they can affect other aspects of our lives. The cost of a hurricane can easily trump any terrorist attack, whether we measure losses in financial terms or human life. The 2008 crisis and its effects still echo through the global economy to this day. As some have argued, wars have been waged simply in order to restore the financial balance caused by the crisis. The effects of epidemics, such as Ebola, on the stock market are not to be ignored.

Finally, if you are wondering about the weather forecast, just ask Napoleon about the cost of a bad prediction of the severity of a Russian winter. This is not meant to belittle the power of prediction. On the contrary, this article is to show the importance and the complexity to predict future events. At the same time, we mean to take a skeptical look. In order to improve our predictions; it is mandatory that we don’t hold on to our current way of thinking and modeling simply because we have been using it for so long.




Disclaimer: The views and opinions expressed in this article are those of the author(s) and do not necessarily reflect the policy or position of the site administration and/or other contributors to this site.

Comments

comments