Meta-Analysis: Less Boring Than It Sounds?

“Meta-analysis” describes a statistical method that collates data from a number of sources and gives an answer more accurate than any of the single individual sources could alone. It doesn’t sound all that interesting, but it can give a pretty neat insight into virtually anything; as we shall see…

Recently I was put on to a web site called Chart Porn by someone with genius tastes. As I’ve mentioned here before I am a fan of graphs, in fact I don’t believe it is possible not to like graphs, unless you are an actual moron.

How can you not relish the wonder that is thousands of data points described in a colourful icon that you can understand without hardly moving your eyes or brain?

I like stats, I’m certainly no expert, I don’t know all the proper words and I certainly couldn’t work a stats software package. But, to me, there is something mind expanding about trends, flows and averages. Call me “Mr Sad Man” if you like, or even “Little Miss Boring” – I do not care. The truth hurts but I am bigger than that now.

Stats can be misused though. And that makes everyone, including Buddha and Moses cry. I did it myself once by mistake. I used to work for a large utililities company, doing basic stats about the reasons why people called in to their customer service lines and what happened when they did.

Riveting stuff, but it gave me a chance to fiddle around with spreadsheets and graphs; and that, I reveled in. One day I was fiddling about in a report that showed whether complaints were sorted out on the first phone call or whether they dragged on and generated further phone calls.

I was bored, as you might imagine, and decided to painstakingly go through all of the data and put in a Male/ Female column. That way I could find out whether boys were better than girls at handling complaints. Like I said, I had time on my hands.

Once I’d finished all the jiggery pokery, the results showed that blokes handled complaints on the first call about ten times as often as girls; the ladies, on average, took a few calls to sort things out. I found this amusing, so I e-mailed it to my boss (male) just for ships and wiggles. He found it very interesting and immediately e-mailed it to his superior (female). It leaped from their inbox to the head of department’s in-tray within 24 hours.

Suddenly, I am being discussed at a high level. It was only a week from that point until it reached Director level and was flaunted as a fascinating bit of work from our department. By now I was very uncomfortable, I had made the graph with little regard for checking, let alone double checking.

So, I looked back on the data I had used and it turns out I should have checked it before I sent it. There was one idiotic guy in the study who claimed to have sorted out the complaint on the call every single time. Whereas, everyone else had told the truth.

Once you removed him from the data, there was absolutely no difference between males and females. Whoops. But thankfully, no one ever came and asked for my workings.

There are probably Directors swilling Brandy at Christmas earnestly discussing that erroneous result to this very day. And that is just my humble experience of bad stats and the effect they can have.

The billy big balls in this business sphere do not have the time or inclination to check things. Other people should do that. But unfortunately, it’s hung over chimps like me that are making these basic errors and causing misinformation on a hideously global scale. Double whoops.

Always double check people. As Vic Reeves once said:

“87.3% of stats are made up on the spot.”

I’m sure that’s accurate, but remember the other 12.7% weren’t made up on the spot, they were worked out incorrectly by an “expert.” So where does that leave us?:

META-ANALYSIS:

In a nut shell a meta-analysis is where you get a whole bunch of experiments that test the same thing and bang all the results in to one big graph train. So you get more accurate results because the dip sticks are diluted.

I came across them recently on the Chart Porn site. Someone had compiled 120 of Top 10 charts for the best album of 2011. And you know what, there was good stuff in there, stuff I didn’t know, stuff I’d forgotten. And it gave me an incline of hope that humans are not doomed after all.

If you were to believe MTV, Kerrang or The Chart Show, we would be listening to Mariah Carey and The XX across the globe and that’s that. Here are the albums that won in the meta-analysis.

Have a listen if you don’t know them, I did and found some stuff that I am still listening to. You are probs totes cooler than me though and already own them all in triplicate. Kuodos my friend:

It’s not all awesome obviously, but it’s not necessarily expected either. One day we will beat Pop for ever.

So I thought I’d see if I could find any other interesting meta-analyses to wet my number whistle. To be honest, I didn’t find any. Some dude did a Meta-analysis of the Top Bluegrass Albums of 2011 and the winner there was The Gibson Brothers – “Help My Brother.” I listened to it and it was rubbish, well not the whole thing, just about 10 seconds on Youtube, but it still counts.

I found a meta-analysis for the Miss Universe 2011 contest, here’s the blurb:

“Miss Universe 2011 is just less than 48 hours and a new queen will be crowned. However, before the coveted crown is given, let us make a meta-analysis of all the candidates based on the lists that were posted in the Miss Universe Forum of this pageant site.  There’s 89 contestants in total”

The list below is their predictions and, in bold, are the predictions that were correct once the real life competition had been completed:

• 1. CHINA
• 2. VENEZUELA
• 3. MALAYSIA
  4. PHILIPPINES
  5. PUERTO RICO
  6. AUSTRALIA
  7. GREECE
  8. FRANCE
  9. USA
  10. UKRAINE
  11. PERU
  12. COSTA RICA
  13. PANAMA
  14. MEXICO
  15. KOSOVO