Learning From Data

[8/9/2012 update]: Got an automated email from Nature:

The following post you wrote on the Nature News website has been hidden by the moderator in accordance with our terms and conditions.

To editor Brian Owens:
I wonder the so called ‘performance profiling’, as mentioned in the title of this article, could be used to judge if the editor was drunk on duty or is lack of basic knowledge of Statistics.

If this is article used to highlight what so called science with crappy data analysis cannot tell us, congratulations! You passed with A+.

This comment contains abusive language and as such breaches our Terms of Service.
-Nature News editors

I thought I’m going to stop writing about numbers and enjoy the Olympic games for a while, but I cannot help writing this piece when I saw the Nature news article: “Why great Olympic feats raise suspicions, ‘Performance profiling’ could help to dispel doubts“.

It reminds me books like Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists and More Damned Lies and Statistics: How Numbers Confuse Public Issues.

NAUTRE, you are better than this! I’d rather not classify such a journal as a news outlet at this moment, but you might prove me wrong soon.

L. NEAL /AFP / GETTY IMAGES

Was Ye’s performance anomalous?
Yes. Her time in the 400 IM was more than 7 seconds faster than her time in the same event at a major meet in July. But what really raised eyebrows was her showing in the last 50 metres, which she swam faster than US swimmer Ryan Lochte did when he won gold in the men’s 400 IM on Saturday, with the second-fastest time ever for that event.

First of all, would it be fair to give a reference or link to Shiwen’s record in July (which July)? Comparing last 50-meter performance of Ryan Lochte and Shiwen Ye is cherry picking of evidence. It makes headlines, but anyone with basis statistical knowledge would think it harder before claiming anything purely based on it. As pointed out in the comment posted by Lai Jiang

First, to compare a player’s performance increase, the author used Ye’s 400m IM time and her performance at the World championship 2011, which are 4:28.43 and 4:35.15 respectively, and reached the conclusion that she has got an “anomalous” increase by ~7 sec (6.72 sec). In fact she’s previous personal best was 4:33.79 at Asian Games 2010¹. This leads to a 5.38 sec increase. In a sport event that 0.1 sec can be the difference between the gold and silver medal, I see no reason that 5.38 sec can be treated as 7 sec. ……

Third, to compare Ryan Lochte’s last 50m to Ye’s is a textbook example of what we call to cherry pick your data. Yes, Lochte is slower than Ye in the last 50m, but (as pointed out by Zhenxi) Lochte has a huge lead in the first 300m so that he chose to not push himself too hard to conserve energy for latter events (whether this conforms to the Olympic spirit and the “use one’s best efforts to win a match” requirement that the BWF has recently invoked to disqualify four badminton pairs is another topic worth discussing, probably not in Nature, though). On the contrary, Ye is trailing behind after the first 300m and relies on freestyle, which she has an edge, to win the game. Failing to mention this strategic difference, as well as the fact that Lochte is 23.25 sec faster (4:05.18) over all than Ye creates the illusion that a woman swam faster than the best man in the same sport, which sounds impossible. Put aside the gender argument, I believe this is still a leading question that implies the reader that something fishy is going on.

Fourth, another example of cherry picking. In the same event there are four male swimmers that swam faster than both Lochter (29.10 sec)3 and Ye (28.93 sec)4: Hagino (28.52 sec), Phelps (28.44 sec), Horihata (27.87 sec) and Fraser-Holmes (28.35 sec). As it turns out if we are just talking about the last 50m in a 400m IM, Lochter would not have been the example to use if I were the author. What kind of scientific rigorousness that author is trying to demonstrate here? Is it logical that if Lochter is the champion, we should assume he leads in every split? That would be a terrible way to teach the public how science works.

Again, I expect better performance from a prestigious journal like Nature. As the online news editor replied in the comment

We appreciate that the case of Ye Shiwen is a sensitive one for some readers. However, I would like to point out that this story was not intended to insinuate that Ye is guilty of anything. As we point out in the first paragraph, she has never failed a drug test and so is the rightful Olympic champion.

We wanted to use the controversy as a way to highlight what science can and can’t tell us with respect to athletes’ performance. We have done similar stories before, for example in the case of South African runner Caster Semenya

Congratulations to Ye Shiwen on her incredible win!

Brian Owens
Online news editor

I wonder the so called ‘performance profiling’, as mentioned in the title of this article, could be used to judge if the editor was drunk on duty or is lack of basic knowledge of Statistics. If this article is used to highlight what so called science with crappy data analysis cannot tell us, congratulations! You passed with A+.

Numbers are number, but there is a story behind each number, don’t just take its face value!