The Hierarchy of Research

Ahh, research. Every debater — nay, every student’s fondest memory and dreamiest fantasy.

Somewhere in its depths, Google has an article or two to support any view, no matter how crazy: Obama is a lizard, milk before cereal, and Season 8 of Game of Thrones actually happened.

And the fact is, this wayward evidence can be used in debate rounds.

Should it be? Probably not. Will it be? Almost certainly. Will it win? Well, that’s for your opponents to try and you to prevent.

But only by understanding the hierarchy of research, from good to bad, can we convince the judge that our evidence is indeed better than our opponents’. Kind of a tall order.

That is, unless you, my unnamed and likely nonexistent readers, follow this guide.

We start at level 0 (‘dear Lord’).

This is, you might say, before the starting point. Not only have we not started climbing the mountain, we’ve actually gotten lost and fallen into a ditch on the way to the trailhead.

Level 0 includes straight-up lies, conspiracy theories (the bad kind), tacky flash-animation blogs the should have been deleted from the internet decades ago, tabloids, and the like.

Examples of these sources include Brietbart, Infowars, the National Enquirer, and a huge number of less-famous and therefore less-obvious sources assembled purely for the sake of propaganda, as portals to anti-truth.

Avoid these at all costs, and take pity on your opponents if they break these out. Poor, misguided fools.

Next, we graduate to level 1 (sh*t sources).

We’ve climbed out of the ditch, found the path, and are firmly on the way to the mountain. Level 1 sources provide a biased narrative of events. This bias can be toward the left, the right, lobbying for an industry or policy, or perhaps journalism that represents or is associated with a government (’state media’) rather than trying to find the truth.

Examples of level 1 sources are most blogs (although, hey, there’s a lot of blogs), MSNBC, and Fox. Of course, there’s good journalists writing in these publications, and the information you’ll find there is likely largely true. But as a matter of course, their allegiance isn’t to the truth — their allegiance is to their narrative.

Now we reach level 2 (maybe true)

We’ve reached the gentle rolling foothills — citing information from these sources would convince be 90% of the time — although, to be sure, I’d double-check it against a better source.

These sources are generally credible, with no obvious bias (although non-obvious biases abound), and without a clear motive to deceive anyone. Here I’d include Wikipedia (no, you can’t ‘just change it’; go try) and other encyclopedias and reference materials and opinion articles or blogs written by experts in the field.

That said, Wikpeida’s reputation has been tarnished from its early days, so, although I’m a big advocate for it — don’t cite it in debate rounds.

Finally, level 3: the news.

Here, the climb truly starts.

News sources that are generally considered, well actual news, include CNN, Forbes, Drudge, Biz Insider. Some bias is still left here, but the reporting of the facts takes precedence. Some caution is still required here, because different stories might bring out different spins, and different writers for the same paper may have different education, motives and thoughts.

Still — getting current-event information from CNN or financial information from Forbes will sound extremely authoritative to most people as it should: it’s usually true. Hell, even Buzzfeed has great journalism sometimes.

But there’s better than news: there’s level 4, or good news.

We’ve mastered many of the smaller peaks. The mountain is firmly being conquered.

Good news are the world’s best and most-trusted newspapers, with a reputation for uncovering previously-unknown stories and truths through the tough journalism of, well, journalisiming. These include the New York Times, AP, The Economist, Politico, and Washington Post.

Once again — these publications do make mistakes, and their word isn’t gospel. But it’s pretty damn close.

There’s few commoners (read: non-debaters) who ever reach this level of enlightenment, and even fewer who get past it. But that’s where we’re going next.

Next up is level 5: actual research.

This is the peak of the mountain.

Peer-reviewed research, whether scientific studies or even better, actual experiments (ask your science teacher about the difference), is often dense, hard to read, harder to understand, and entirely, completely boring. But it’s /true/ in as far as we can say things are true as simple fallible human beings.

Go to Google Scholar, JSTOR, or other research databases to search through this treasure trove of humanity’s knowledge. I sincerely hope we’ll have better knowledge in the future — and I think we will. But for now, this is as good as it gets.

Finally, la creme de la creme, level 6.

If level 5 was the peak of the mountain, level 6 is the whole range.

This level is known as the ‘gold standard’ of research across the sciences, the adherents of science, and for our purposes, debate. At level 6 we reach metastudies, analyses of several — often dozens — of level 5 research papers. If level 5 is science, level 6 is uberscience, using the tools of statistical analysis multiplied by the power of actual scientific studies and often actual experiments.

If you find a meta study that confirms a fact in your case, you’ve found the arbiter of truth in that round, as far as their conclusions go.

Pay attention to evidence quality: in your research, in your reading, in your card-cutting, and especially in your debate. If two sources are in conflict, see if one source is obviously ‘higher up’ the hierarchy than the other. This can guide your thinking, and if practiced consistently, even win you some debates.