The Corner

On Data Samples, Con’td

From a reader:

Jonah

As an engineer in experimental R&D. To say the least, I am scientifically inclined. Your reader points out quite correctly the problem of small data sets. However, there are two other problems: the dirty little secrets of modeling: errors compound and validation is vital.

The sad truth is that every data manipulation you do makes the data worse. Instrument errors, sampling errors, environmental errors, these all build and increase the uncertainty. This doesn’t even take using the data. Every calculation has its own errors and these compound every time its used. This is why models become less and less accurate over time. The errors build up. Similarly, this is why the more complicated a model is the more error prone it is.

Which leads to the question of how do you know your model is right? To put it bluntly, it isn’t. All models are wrong. The important part is do determine how wrong, and under what conditions does the model give useful answers. This is called validation.

Consider something as complicated as the entire planet’s atmosphere, ocean, landmasses, and solar behavior. And consider that, unlike most science, there is no experimentation. One can’t buy a bunch of earths from a supply house and put various conditions on them and record the results.

This makes model validation challenging. In normal validation, experimental data is used to see how well the model predicts a multitude of input conditions. In climate modeling, there is only one set of data, since there is only one Earth.

This results in less validation, compared to models that can use experiments. It’s no wonder that climate models have proven themselves woefully inept at predicting current conditions when given past conditions. Give them data up until ten years ago and they can’t predict this year.

Remember that this is what they’re talking about when they say the “science is settled”.

Oh, and a reader sent me here, which is a very interesting read.

Most Popular

Film & TV

A Film for All Christians

‘The growing good of the world is partly dependent on unhistoric acts,” wrote George Eliot in Middlemarch, “and that things are not so ill with you and me as they might have been, is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs.” The passage provides the title ... Read More
Film & TV

A Film for All Christians

‘The growing good of the world is partly dependent on unhistoric acts,” wrote George Eliot in Middlemarch, “and that things are not so ill with you and me as they might have been, is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs.” The passage provides the title ... Read More
Law & the Courts

The FBI’s Corrupt Cops

White-collar criminals should hope for one thing this Christmas: that they get to live under the Horowitz rules. Michael Horowitz has testified that he found no evidence of political bias on the part of the decision makers who, under the Obama administration, relied on hilariously implausible “evidence” ... Read More
Law & the Courts

The FBI’s Corrupt Cops

White-collar criminals should hope for one thing this Christmas: that they get to live under the Horowitz rules. Michael Horowitz has testified that he found no evidence of political bias on the part of the decision makers who, under the Obama administration, relied on hilariously implausible “evidence” ... Read More
White House

The Horowitz Report and the Power of Inertia

The best thing I've read about the report is by Julian Sanchez. An excerpt: The heart of the Horowitz report deals with the Carter Page FISA application, and documents a progression that ought to sound familiar to anyone who’s studied the history of the intelligence community: An investigation begins with a ... Read More
White House

The Horowitz Report and the Power of Inertia

The best thing I've read about the report is by Julian Sanchez. An excerpt: The heart of the Horowitz report deals with the Carter Page FISA application, and documents a progression that ought to sound familiar to anyone who’s studied the history of the intelligence community: An investigation begins with a ... Read More