When a forester fails to predict that a tree might fall and it kills someone, he is arrested. The same goes for a train mechanic who fails to repair a carriage, a cook who poisons a customer and a builder whose house collapses. They didn't mean to kill, but they failed to forecast what might ensue from their defective expertise.I am not sure I follow the argument. Workers and professionals of exercising their job as described above are not in the prediction business, neither are they giving advise to public bodies. They simply do a bad job which will land them in trouble. They could be guilty of negligence or manslaughter.
Why does the same not apply to the professional scientists, experts and pundits on whose predictive genius so much of our life depends?
The Aquila case is different. Scientific experts issued public reassurances which were arguably not warranted on the basis of their scientific expertise. We have seen many cases in the past, from assurances about the risk free nature of nuclear energy to the safety of British beef. And we see poor predictions in meteorology and economics, the two areas most visible in the public eye.
Here Jenkins makes a reasonable point when claiming that the Met Office has issued "persistently pessimistic" forecasts. The reason for doing so is blame avoidance but has the side effect of deterring tourists. The Met Office never seems to review their errors publicly. The same applies for economic forecasting.
As always: What are the costs (and consequences) of false positive, what are the consequences (and costs) of false negative.
ReplyDeleteA reader
Reiner,
ReplyDeleteperhaps the problem are not the predictions per se, but the raised expectations of accuracy. It is indeed very difficult to convey the inherent uncertainties to the general public, but some times an approximate measure of accuracy, as disclaimer, could be possible. I am thinking of something like: 'of N past earthquakes we correctly predicted x % and were wrong in y%. The rate of incorrect alarms was z%
Eduardo
ReplyDeletedecision makers will want to know what to do. Should they call for an evacuation or tell the people to stay put? If experts advising on this question don't know they should say so.
reader #1
the costs will be assessed differently by different people. Such considerations should play no role when making statements about earthquake risk.
If the public has the impression that a risk assessment is made with a view to broader consequences (including costs, or public reaction) such an assessment will not gain credibility.
If it turns out that the public is confronted with more than one view from technical or scientific experts, they will make up their own mind. A widespread cynical reaction seems to be: It is your own fault if you trust expert recommendations.
From what I hear from Italian friends, is the accusation in this case rather bizarre. But there may be something to the case beyond the question how the communication took place in Italy.
ReplyDeleteThe interesting NEW dimension is that societal actors have raised demands on what science is delivering, and that part of these demands are (in this case: unrealistic) levels of quality. In a nutshell: "We are paying for you, and for this effort we expect the following utility." We scientists need to think about what we tell the public, we need to talk about falsification and accountability.
With this statement I do not want to imply any wrong doing by the Italian scientist - only that a new type of question has been asked not only in the Italian court but also in mainstream newspaper of the US - see http://www.philly.com/philly/opinion/inquirer/20121026_The_world_s_scientists_feel_a_tremor.html (reprinted in International herald Tribune)
Reiner,
ReplyDeleteI though decision makers are the ones that make the decisions. They cannot expect scientist to tell them what to do. Scientist can tell them that there is a x% probability of earthquake in the next d days.
Form the "article in Science:
'The best way to avoid such problems in the future, Jordan says, is to clearly delineate the role of the scientists and that of authorities responsible for civil protection. Experts should provide “carefully
constructed probabilistic statements” regarding the risk, he says, which decision-makers would then use to choose the best course of action.'
Thomas Jordan is the chair of the earthquake forecasting commission ICEF in the US, and I think he is spot-on
Eduardo
ReplyDeletein this case the decision makers were the public themselves. What appears to have happened is that the scientists reassured the public without having the relevant knowledge. I quote from the Science article you linked:
"On trial are seven men—four scientists, two engineers, and a government official—who participated in a meeting of an expert panel of Italy's Civil Protection Department (DPC) known as the National Commission for the Forecast and Prevention of Major Risks, which met on 31 March 2009 in L'Aquila to assess the ongoing series of tremors.
After the group adjourned, two members gave a press conference, accompanied by local officials. On that occasion, prosecutors say, they gave L'Aquila's inhabitants the mistaken impression that they had nothing to fear, and as a result, some people who would otherwise have fled their homes during subsequent tremors stayed inside—and were killed on 6 April. Indeed, when the prosecutor asked Linda Giugno why her brother was so sure there would be no destructive earthquake, her answer was clear: Experts quoted on TV news reports had said that there would be no tremors stronger than those already experienced."
If this is true, I don't think this case is characterized by exaggerated expectations on the science.
Here is another snippet from the Science article which indicates that relying on lay knowledge would have been far better:
ReplyDelete"Traditionally, prosecutors argued, people in L'Aquila had been trained by their parents to leave their homes as soon as they felt the ground shake, to avoid the effects of any further, potentially larger, tremors. That was what happened on the day before the meeting, when the magnitude-4.1 event happened; many people gathered near the castle or in one of the town's squares until they felt confident enough to go home. But the meeting of the Major Risks Commission changed many minds, contends the prosecution. “It was as if we were anesthetized, like someone had removed our primitive fear of the earthquake,” the court was told by local surgeon Vincenzo Vittorini, whose family stayed inside the night of 5–6 April. “After that damned meeting, they instilled in us the idea that something terrible couldn't happen."
I assume all Italian clergymen and psychics will be held equally accountable for this disaster.
ReplyDeleteReiner's quotes in #6 and #7 clearly say that the "scientists" lied to the public ("no danger" instead of "we don't know more than everybody else") to the effect that people ditched their local knowledge, here a realization of the precautionary principle. Some died for this reason.
ReplyDeleteOf course someone needs to go to jail for this! What's wrong with you people? Others are doing time because they lied to the public about a company's stock prospects.
Reiner,
ReplyDeleteI think this cases raises quite a few interesting questions, independent of the particular legal issues which I obviously I am not qualified and informed to comment on. The article in Science does suggest that the story is not that simple. Also, the article is just one source and might not be reflecting the whole truth.
Who should be allowed to advice the public ? Some weeks before the earthquake another 'maverick' scientist alarmed the population predicting erroneously an earthquake that did not happen. Should there be limitations in this respect ? how these limitations interact with freedom of speech ?
Should researchers (I mean scientist that work in labs or universities, and do not hold public office) be allowed to advice the public ? There are more issues than only scientific analysis. In these cases, for instance, one should also consider evacuation infrastructure, security of the left property, etc.
Where should liability lie ? A researcher salary does not include liability for his/her predictions.
Should researchers collude with policy makers, so that the latter can hide behind the former when things go wrong, like in this case ?
All this question apply directly to climate or climate related issues, like hurricanes, droughts and may be on long-time scales as well. Think of the IPCC process, where policy makers hide behind 'what science tells us' although there are directly involved in the IPCC process.
My view is that the process should be much more strongly regulated, or alternatively, the advice to the public or the advice to policy-makers should be strictly separated from researchers, and, in the end include liability for the advising body. Liability not for the wrong advice, since predictions can only be probabilistic, but liability for a biased advise.
Addendum to #9.
ReplyDeleteOf course its not so simple; the excerpts I based my first impression on are a selection representing the prosecution/L'Aquila's residents view.
The best article about the issue I could find is this Nature feature. From that is seems the scientist are guilty of having let themselves be used, rather naively, by the politicians to calm the population. Not good, but not 6 years in jail. Except for the government official in the commission perhaps, who apparently blatantly lied in the press conference.
From the article:
Even Boschi [convict scientist] now says that "the point of the meeting was to calm the population. We [scientists] didn't understand that until later on."
De Bernardinis [government official in press conference] said that the seismic situation in L'Aquila was "certainly normal" and posed "no danger", adding that "the scientific community continues to assure me that, to the contrary, it's a favourable situation because of the continuous discharge of energy".
"That night, all the old people in L'Aquila, after the first shock, went outside and stayed outside for the rest of the night," Vittorini [father and husband who, based on the official statements, convinced his family to stay in the house, where wife and daughter were crushed to death] says. "Those of us who are used to using the Internet, television, science — we stayed inside."
Steve Curry has written a comment worth a read:
ReplyDeleteShaking with anger: why Simon Jenkins is wrong — and right
"the most annoying thing about Jenkins's piece is that he has a point. It might have been difficult to spot in the midst of his bilious rant but the scientific community needs to be careful in presenting commentary — and recommendations — on matters of serious public interest. The Royal Society and the US National Academy of Sciences did us no favours by issuing a curt statement that simply condemned the decision of the Italian court. There was no critical assessment of the particular actions of the scientists that had raised the concerns of the people of L'Aquila. The absence of nuance failed to reflect the realities of the case and shaded into an arrogance that served neither the public nor the scientific community."
eduardo
ReplyDeleteyou ask good questions, also big questions - I am drafting a paper which will address some of these issues. Will put it up in the next couple of days or so.
Some background is provided by the BBC Radio 4 program More or less
ReplyDeleteThis quote didn't get out of mind:
ReplyDelete"“It was as if we were anesthetized, like someone had removed our primitive fear of the earthquake,” the court was told by local surgeon Vincenzo Vittorini, whose family stayed inside the night of 5–6 April. “After that damned meeting, they instilled in us the idea that something terrible couldn't happen."
A surgeon who felt "anesthetized": what kind of magic do those scientists and experts in the commission have? This is what makes me really wonder. Who in the world would ever believe such a commission let's say in the case of nuclear accident? Nobody, of course. But in the case of an earth quake... really strange, doesn't make sense to me.
Werner,
ReplyDeletethe magic is called "expertise".
Reiner's recent conceptualization provides the answer to your question: There is no value conflict in earthquake prediction, or so one should assume. So you have no reason to doubt the expertise-informed official statements. That apparently there was a value conflict, between the politician's complacency and laziness and the tourism-impact or whatnot on the one hand side and the aim of protecting the population from death, this really is the ethically unexpected, which prompted and justifies the lawsuit.
Nobody believes official statements about nuclear safety because everybody knows that this is highly value laden: between the safety of the population here and really big money ("economic politics") there.
hvw
ReplyDeletethanks for your comment. You are perfectly right, the magic is called "expertise". I agree, but the story is not yet fully explored. Here some preliminary thoughts:
I was stunned because we talk here about a surgeon, an educated person, and simultaneously about something he calls his "primitive instincts". Primitive!
Science and the nation state are closely related; science counts, measures, weighs the nation - which would not exist without those numbers. Simultaneously, the nation state takes scientific issues such as nature, the body, health, risks and so on to conquer its territory, to include the population and so on.
I know studies about Italy how nature conservation is used to get access to "wild" territories like Sardinia and its population. In the course of this, local populations are labeled "backwards" and "primitive" and not yet "modern" (or ecological etc) - in any case, scientific experts help to make them subject to administrative measures; the population gets educated and finally governed. Science is "partner in crime" with the nation, even if the scientist is only interested in wildlife, ecosystems or earthquake.
This might explain why the surgeon got "anesthetized": he lives in a "backward" earthquake region but feels already modern: he listens to the nation's experts and no longer to his "primitive" side.
And it might explain why the scientists failed as experts: because they are subject to the same process - they forgot their instincts, too: the instinct to distrust power.
(This is a short version of a long thought, but I hope you get the idea).
Werner,
ReplyDeletethanks for your thoughts. You manage to pull some really deep stuff from the L'Aquila incidence.
I know studies about Italy how nature conservation is used to get access to "wild" territories like Sardinia and its population ...
I clearly hear an ethnologist who is very well aware of his own field's history:). I would never have guessed that this happens in today's Italy though.
I am afraid I don't have a good opinion on the issues you raise. The role of science on the personal scientist's level is complicated enough. On the one hand side, the experts, say a German tenured "verbeamteter" professor has probably more chances to speak his mind freely, in the public debate, in a topic related to her area of expertise, than anybody else could dream of. Also not difficult to place an op-ed in a newspaper. In private conversations they often have strong opinions. Yet - silence. Our expensive experts just don't bother talking much to the general public. I think they should. But once they do, political opponents and a good share of their colleagues are all over them for leaving the ivory tower, taking sides in the political debate, being "advocacy" instead of honestly brokerish reporting options and consequences to decision makers and bringing them sandwiches.
But that is what we would have expected from the L'Aquila experts, no? Not bending over for the politicians but grabbing the mic and reminding the people in what danger they are.
What if too many scientists become uncomfortable for the powers that be (don't forget the science funding industry here)? No money no research. What makes good scientists good scientists? That they care little for anything else than science. Still wonder why they bend over?
If society wants outspoken, independent and trustworthy experts, maybe structure has to change to allow more independence and security ?
Sometimes I see science as a nation-state transcending tradition, which uses societies and power structures for its own goals, instead of the other way round, occasionally dishing out some goodies in return for favors.