How to not make bad models

Levin (1980)* is a concise and insightful discussion of where mathematical modelling can go wrong. It is quite relevant to my investigation of The Art of Mathematical Modelling and does a nice job of addressing my ‘why make models?’ question.

Vito Volterra is referred to as the father of mathematical biology in Levin (1980).

This paper answered one of the questions that I had long been wondering about: who is considered to be the father of mathematical biology? Levin’s answer is Vito Volterra** – at least for mathematical biologists who come from a mathematical background. Levin then says that modern day mathematical biologists, as the descendents of Vito Volterra, lack his imagination; too often investigating special cases or making only small extensions of existing theory. It’s a fair point, but thinking takes time and time is often in short supply. My take on Levin’s comment is ‘aspire to be imaginative, but to remember to be productive too’. Furthermore, Levin identifies one of the ingredients that make great models great: imagination – I’m adding that to my notes.

A second piece of advice is that mathematical models that make qualitative predictions are more valuable than those that make quantitative predictions. Levin’s reasoning is that ‘mathematical ecology as a predictive science is weaker than as an explanatory science because ecological principles are usually empirical generalizations that sit uneasily as axioms.’ That is quite eloquent – but is it really quite that simple? For example, if you make a quantitative prediction with a stated level of confidence (i.e., error bars) is that really that much worse than making a qualitative prediction? The sentiment of the quote appears to be to not overstate the exactness of the conclusions, but to me this seems equally applicable to quantitative or qualitative models.

Levin coins the phrase ‘mathematics dressed up as biology’. I have my own version of that, as I like to say ‘that’s just math and a story’, in both cases, for use whenever there are weak links between any empirical observations and the model structure.

To conclude, this paper discusses why the different approaches of biologists and mathematicians to problem solving can result in mathematicians that are keen to analyze awkwardly derived models and in biologists who lack an appreciation for the mathematician’s take on a cleanly formulated problem. Rather than discussing what makes great models great, Levin’s paper reads like advice on how not to make bad models, and because it’s so hard to distill the essence of good models, looking at the art of mathematical modelling from that angle is a constructive line of inquiry.

References

Levin (1980), Mathematics, ecology and ornithology. Auk 74: 422-425

Footnotes

*Suggested by lowendtheory, see Crowdsourcing from Oikos blog.

**Do you agree? For me, if this is true then the timing is interesting: Vito Volterra (1926), Ronald Ross (1908), Michaelis-Menten (1913), P.F. Verhulst (1838), JBS Haldane (1924) and the Law of Mass Action dates to 1864.

Levin also hits on several items from my ‘why make models’ list and so I have updated that post.

Advertisements
This entry was posted in General by Amy Hurford. Bookmark the permalink.

About Amy Hurford

I am a theoretical biologist. I became aware of mathematical biology as an undergraduate when I conducted an internet search to learn about the topic. Now, twelve years later, I want to know, what is it that makes great models great? This blog is the chronology of my thoughts as I explore this topic.

7 thoughts on “How to not make bad models

  1. Perhaps if we released the pressure to publish and be productive, we would all have more time to think and be creative. As a young scientist I would gladly trade my time devoted to publishing or trying to keep up with the never ending mountain of research with more time for creative thinking….

  2. In my view, when Simon Levin suggested being more confident on models making qualitative predictions rather than on models making quantitative predictions, he made a rather sensible, inspiring and insightful point; as all scientific models, ecological models are always partially-specified in some respects: not all aspects of “reality” are taken into account, so it might not be that straightforward to recast a quantitative prediction with some “ontological” feature of the study system; as Levin quite correctly points out, ecological models are just empirical generalizations, not axioms. Richard Levins also dealt with this kind of stuff in the seventies, introducing loop analysis to derive qualitative insights from partially specified models (see, e.g., Levins (1974) The qualitative analysis of partially specified systems. Ann N Y Acad Sci, 231,123-138).

    The crucial point of all this, perhaps, is that we should be primarily concerned with making the right questions, rather than focusing our energies on deriving quantitative answers. This reminds me of a delicious quote from John W. Tukey, the brilliant statistician: “An approximate answer to the right problem is worth a good deal more than an exact answer to an approximate problem”. This bold statement is always guiding my personal scientific practice, and I take Simon Levin’s advice in the very same sense.

    • Hi Pablo,

      Thanks for the great quote, the reference and the insightful comment. I hope I didn’t make it sound too much like I didn’t appreciate that Levin quote: I chose it to discuss because I thought it was insightful and particularly the ‘ecological principles are usually empirical generalizations that sit uneasily as axioms’, I thought was very well put.

      I agree entirely with what you say but my point was to ask whether it’s really fair to equate ‘approximate answers’ with ‘qualitative answers’ and ‘exact answers’ with ‘quantitative answers’. Particularly, since the point of the quote is that the model assumptions and parameter values have uncertainty in them. This uncertainty could be estimated and explicitly considered as part of the model output/analysis (for example, using a sensitivity analysis). I do still think that if I predict a range – a numerical range that my model output will fall into with, say, 95% certainty – that this is still a quantitative rather than a qualitative prediction.

      On the other hand, the qualitative model seemingly is viewed as being better only because it is less precise. This, in of itself, I don’t think makes it necessarily any better at overcoming the limitation of the model assumptions having been based on empirical generalizations since being unnecessarily vague is also not a desirable property of a prediction.

      Having said all this, another characteristic of qualitative models is that frequently they delineate outcomes that are possible versus impossible. Could this be the characteristic of qualitative models that have made them so successful? And if so, I might be tempted to argue that this too does not necessary exclude quantitative models either.

      Anyway, I’d say that I agree with you on all your points, I just thought I’d be fun to delve a bit deeper into some of the reasons why qualitative models are often advocated for.

      • Amy, sorry for the delay! Fair enough, those are good points you made. I think these issues are rather more “philosophical” than we traditionally acknowledge, and I confess that I tend to be lost in the details. However, focusing only in the models we construct to understand reality, I think my view is qualitatively (not joking) different from yours. I mean, I agree with you that it is not fair to equate ‘approximate answers’ with ‘qualitative answers’ and ‘exact answers’ with ‘quantitative answers’. But I don’t think Simon Levin or any other would hold this as a general statement. Qualitative and quantitative are clearly not mutually exclusive characteristics of a model of enquiry.

        For example, I can build a set of N alternative models and fit them to a dataset Y. Then, “by assigning different model probabilities” (oops!) I can provide a set of N different qualitative scenarios that “fit” quantitatively different to the dataset Y. There’s your continuum: a quality (a set of models, or perspectives) matches a quantity (a set of data). The fact that you can somehow calculate a confidence interval for your “parameter” does not make it more “approximate”, in the same way as the fact that you build an arbitrarily large ensemble of “models” does not makes your enquiry more “precise”. The uncertainty of the parameter arises both from the quality (model structure) and the quantity (the available data); and the precision of the model ensemble arises from the limitations imposed by the quantity (the available data) and our ability to specify a process (quality). From reality to models, quantity and quality are two different perspectives of the same underlying process.

        But, as reminded by Richard Levins, even a number has qualitative properties (“period 3 implies chaos”); and once you estimate a parameter it begins a life of its own, and this life can have nothing to do with reality, and… I personally find these kinds of discussions exciting, which means that I could be writing ad infinitum, and so become pedantic ad absurdum :)!

  3. Pingback: Probabilistic models in statistical analysis – mechanism or phenomenology? « theoretical ecology

  4. Pingback: Ecology blog roundup « theoretical ecology

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s