Who Can You Trust?: Source Evaluation

Thanks to some conversations recently, I’ve been thinking about evaluating sources. Obviously, this is something I’ve taught for years in the classroom setting, but I think it is even more important to consider in a wider context given the growing propensity for people to link articles in online discussions (arguments, rants, whatever) as evidence (or “evidence”).

So, why should we evaluate sources? Why not just go grab the first online articles that seem to agree with our positions?

Several reasons, really, but the most important is: just because a source agrees with your position, does not mean that it is a good source.

Again, so what?

To best convince an open minded audience, we need to present the best possible evidence. Part of doing so is presenting evidence from solid, reputable, credible sources.

So, how do we determine a source’s credibility?

Just because it’s written (or on the interwebs), this does not mean that it is credible or reputable. Honestly, anyone can write anything. And these days it’s easier than ever before for anyone to publish whatever the heck they want to, whether it’s true or a drug induced mental rambling. This is one reason that scholars and other professional researchers are often skittish about internet and self-published materials.

Here are some useful criteria, gleaned and boiled down from a variety of collections of criteria found in dozens of writing and research handbooks (this is by no means a complete discussion):

1) Authorial Reputation and Associations

As we know, most writers and speakers have reputations for various things and everyone is associated with something. For example, we know that Mike Huckabee is an evangelical conservative and is associated with Fox News while Al Gore is an outspoken environmentalist. These reputations create certain expectations. They can also help us determine whether the individual is typically trustworthy, approaches multiple perspectives fairly, or not. A recent example I saw was an article written about problems with the Canadian health care system, written by a person who works for a public policy group. One of that ppg’s primary goals is to dismantle the Canadian health care system, so the author’s reliability is suspect.

2) Publisher/Host Reputation

Just as authorial reputations influence whether we can call a source reliable, so does the reputation of the publisher (or web-host). And this varies from topic to topic. For instance, we can consider the Harlequin publishing group an excellent source for the romance genre of novels; however, they would certainly not be a reputable publisher for sources on nuclear physics. On the other hand, the Oxford University Press has spent over 600 years building a strong reputation for solid publications in a range of fields.

3) Obvious Bias

Does the source make clear its bias without attempting objectivity? Obviously, a purely objective position is impossible to achieve, however any good piece of journalism, scholarship, policy, etc. should at least attempt to be as objective in tone as possible. For example, Ludwig Von Mises’s Socialism: An Economic and Sociological Analysis is a fairly obvious propaganda piece that does nothing to appear objective—one of the first lines is even a logical fallacy (either/or fallacy).

4) Whom Do They Reference?

This is important with all sources, really, although it is more obvious with print. Most print non-fiction will include a list of source material consulted by the author. A read through that list should bring up some recognizable names within the field the source is in. For example, in a source of medieval magic, I would expect to see Carlo Ginzburg’s name come up; for medieval childhood, Barbara Hanawalt should be in there somewhere; etc. Likewise, virtually every article online references other articles, interviews, or other sources. These sources of information for the writer are important to look at, both for veracity of data and credibility of the original source.

5) Do They Look at Multiple Angles or Perspectives?

One way to present objectivity, and build the author’s credibility, is to look, honestly, at multiple perspectives on an issue or topic. By honestly, I mean by clearly and truthfully presenting the other positions without resorting to stereotypes, ad hominem, or misrepresentation of the arguments. This lends both credibility and objectivity in that it shows the author looked at the topic from a number of directions and chose one in particular as best, or synthesized one of out many, rather than simply starting with a single premise, putting on blinders, and ignoring any other ways of looking at the topic.

6) What’s the Context?

No material is created in a vacuum. Everything ever written is produced within a cultural-historical moment that influences it, or even acts as a catalyst for its creation. For instance, MLK’s historic “I Have a Dream” speech was written in 1963. If the year had been 1943, the speech would have been very different, likewise if it had been 1983. But, that doesn’t tell us about credibility. Because culture influences thought and source creation, cultural biases and issues come into play as well. For example, it is difficult to trust an anti-socialism source from the early-1950s in the U.S., because of the effect of McCarthyism (e.g. a pro-socialist book would never get published, the author would be blacklisted, and the author would potentially be deported or face imprisonment as a Soviet spy). So, even if the author thought socialism was the greatest thing since sliced bread, (s)he would write that it is evil as a means of self-preservation.

7) Can the Information Be Confirmed by Unrelated Sources?

One of the most important, and potentially difficult, measures of source reliability is confirmation. That is, can the claims be confirmed with reference to other sources. The difficulty is that the other sources must be unrelated to the first and must reference different primary sources (studies, interviews, etc). For example, recently someone in a discussion posted links to three articles—one from a conservative website, one from a left leaning site, and one from CBS—that all said the same thing. On the surface, this is good. However, all three cited the exact same study, from the exact same conservative public policy group (in fact, the second one, on the left leaning site, was written by a senior fellow of said group). So, there was no credibility in terms of data confirmation, since they all used the same primary data source.

8) Has the Work Been Peer Reviewed?

For print, and eprint, works, particularly those from scholarly writers, the question of peer review arises. Peer review (double blind) basically means that two people, experts in the same field, read the document and decide: print as is, print with modification, or do not print. The author gets feedback from the reviewers, modifies as needed, and the piece gets printed. This is essentially quality control for the written word. It keeps publishers and journals from printing materials written by individuals who have no knowledge or expertise in the appropriate field(s), usually. Versus non-reviewed sources that literally anyone can publish.

9) For Online Sources, Who’s the Sponsor?

With online sources, it can be difficult to determine who produced the material or what the goal of the organization hosting it is. So, we can also look for the sponsor of the website, e.g. who pays the bills. Then we apply the same criteria to the sponsor as to the publisher or author.

10) For Online Sources, What’s the URL Suffix?

I know it has been said before as well, but the URL suffixes can be helpful in determining web source reliability. As we know, the most common is .com (commercial site; e.g. they’re selling something), probably not the best option. After that, .org (non-profit organization), which will likely have obvious biases that need to be accounted for. Perhaps the most reliable are .gov (government), .edu (American academic), and .ac (academic network; non-U.S.), as they generally undergo some standards for posting and review of materials.

Three Elements of Argumentation: Logos, Ethos, Pathos

Many, many years ago, three elements of argumentation were formulated by ancient Greek orators. They were discussed in depth by Aristotle in his book On Rhetoric. Isocrates, Plato, Cicero, and others continued his discussion, mostly in terms of application and best usage. Those last two are important points because which element(s) is/are the best choice or most effective depends on the situation, medium, and audience.

Logos is basically logic and reasoning. It deals with verifiable data and syllogisms (premise-conclusion constructions). It is generally preferred in academic argumentation because the arguer needs to possess knowledge and understanding of the subject in order to make logos effective. That said, logos is not always the best choice for every situation, particularly because it takes time to construct a fully formed logos argument.

Logos is also prone to logical fallacies. Among the most common are the false premise (one element of the syllogism is demonstrably untrue), non sequitur (conclusion doesn’t follow from premises), straw man (false representation of the opposition’s argument), and the slippery slope (if A, then B, C, and D bad things follow). There are others, but these are among the most common.

Many people confuse the other two elements of argumentation with logical fallacies, erroneously. I think ethos gets mistaken because of the false authority fallacy (Person A is held as an authority when Person A has no expertise on the subject; commonly seen in advertising with celebrity endorsement). Logical fallacies are inherently things that pretend to be logical but are not (thus being false logic). Neither ethos nor pathos ever claim to be logical, therefore they are not logical fallacies.

Ethos is a false cognate. It looks like the English word ethics, which we do derive from the Greek ethos. But, there is more to the word. Ethos is a mix of the character of the speaker and the authority of the speaker. We use ethos regularly, often in conjunction with logos. Any time that we quote someone else on a subject to support our position, we’re using ethos (borrowing the other person’s authority).  At the same time, by citing research, we establish our own authority on the subject.

Pathos is the source of the English words empathy, empathetic, and pathetic. Pathos is emotional appeals, which we see most often in advertising and politics. Generally, pathos is avoided in formal argumentation and academic argumentation because it demonstrates no knowledge of the subject. What pathos demonstrates is knowledge of the audience and what buzz words will get knee jerk, gut level reactions. It is still useful and effective in the right context, such as advertising. Pathos is also commonly used in speeches and other public speaking because it operates much faster than logos and requires less build up. Therefore, it is easier to follow—consider the average listening audience: will they follow an hour long monologue in which a complex logical argument is constructed or will they react better to a five minute speech that reaches the same conclusion through emotional appeals?

Sources: Aristotle On Rhetoric, Isocrates I, various handbooks (Allyn & Bacon, Prentice Hall, Everyday Writer, Norton Field Guide), Purdue OWL (Fallacies, Elements)

Bill O’Buffoon, or Equality and Straw Men

A few years ago, an older (and rather politically & socially conservative) student referenced an absent student with a learning disability. The first student asked, “You don’t really think she’s equal to you, do you?”

Fairly recently, a notable TV personality, referenced in the title, claimed equality is impossible because he’ll never be a famous basketball player.

My answer to the first inquiry was, “Yes, I do.” My answer to the second is, “Nice straw man fallacy.”

When we talk about equality in society, we don’t mean that everyone can do everything equally well. What we mean is socio-political equality. That is, what we mean by equality is that everyone gets treated the same, legally, socially, and politically.

For example: everyone gets paid the same for doing the same job at the same level of experience, regardless of race, religion, gender, or orientation. Likewise everyone has the same opportunity to nurture their talent, whatever those may be, instead of being held back by the accident of birth into a given socio-economic level or prejudices about race, gender, religion, or orientation (or whatnot).

This does not mean that everyone gets to play professional basketball. But, it does mean that anyone who has a talent for basketball should have an equal chance to potentially play professionally. Likewise, my own talents are in the teaching and writing realms, therefore equality means a fair chance for me to develop and make a living from those talents (despite my total lack of basketball ability), regardless of unchangable factors (e.g. race, gender, orientation, or even religion). The referenced personality’s talents, from what I can see, are conning, bullying people, and fearmongering . . . but are clearly not in formulating logical argumentation.

In the example I started this post with, the absent student was/is a proficient (maybe even talented) computer programmer, something I’ve tried and found that I have no talent for. On the other hand, said student’s writing needed a fair bit of work and did not come easily to her. We’re equal, nonetheless, even though our talents are different and we’ll never be identical.

In short, equality means equal opportunity, not everyone being identical.

P.S. The first student mentioned above was also the inspiration for my morality post as (s)he stridently claimed that religion is an absolute necessity for morality.

Atheism

I came across this piece a few weeks back and finally got around to writing about it. This will tie back to writing and argumentation, eventually.

http://m.theweek.com/article.php?id=241108

First, I’d like to take a look at Linker’s claims. As I read them, they are: atheism is inherently tragic and terrible, “honest” atheists suffer unto nihilism, and no god is akin to no meaning in life. These are all common arguments against atheists and atheism, as such they’re likely familiar to anyone who is open about his/her atheism. They’ve also been argued ad nauseum (I can present the counterarguments, if desired, just ask).

More important, I think, are Linker’s underlying assumptions. Although he doesn’t directly state his assumptions, they are key to his argument. First, he has the assumption of absolute forgiveness, the idea that no matter what a person does, no matter how horrible, everything can be forgiven. Second, he has the assumption of absolute justice. Third, he assumes that all atheists must inherently be despondent, a common assumption on the part of many, particularly conservative, religious people. Fourth, he assumes that atheists are just people who have had a crisis of faith and are looking for a way back in, also a common assumption amongst conservative religious people. Finally, he assumes there is, must be, an intrinsic fear of death.

Each of his assumptions has some notable flaws. Absolute forgiveness has always been a sticking point for me, particularly in regards to Catholicism and Baptists (and I spent eight years in Catholic school). In theory, a person could commit genocide and still get into the good afterlife because they repent on their death bed. This doesn’t sit well with me. It also causes a problem with absolute justice: absolute forgiveness inherently undermines absolute justice (assuming that we don’t accept the later Catholic invention of Purgatory). I think Robert Heinlein said it best, noting, “TANJ” (There Ain’t No Justice). I’ll hit the other three assumptions in my own take on atheism next.

I find atheism to be freeing, rather than depressing. There is a definite freedom in not having a reward-punishment cycle and in the lack of empty ritual. And I have yet to meet an atheist who is looking for a way back into any religious faith. I think one of the interesting aspects of atheism is the acceptance of death, the acknowledgement that there is nothing after. This frees people to live in the present, to live for this life not for some imagined reward or to avoid some imagined punishment in another metaphysical place.

I think atheism, the lack of reward-punishment, also frees people to true morality. Admittedly, morality is in many ways a social construct (e.g. without society, there’s really no need for morality, since morality is basically a code for getting along with others). Without a deity, I argue, people achieve true morality: they aren’t performing, or not performing, acts because of an expectation of reward or fear of punishment (which is conditioning, or self-interest, not morality), they perform acts because they are the right thing to do or don’t perform them because they are the wrong thing to do. This also means that the atheist bears personal responsibility for his/her actions, (s)he can’t claim “god (or Satan or whoever) made me do it.” Perhaps that is one thing that scares a lot of people.

On that note, the promised tie to writing and argumentation. Linker’s piece is an excellent example of the importance of ethos (authority), understanding counterarguments, and understanding the positions of others in debates. His piece fails on all three. It fails because of Linker’s initial assertion that all of his opponents in the debate are dishonest if they do not fit perfectly into the mold he wants them to fit. Obviously, this assertion causes problems for true discussion or debate, since one party comes into the debate assuming the other is lying.

For a more reasoned discussion of the subject see:
http://www.psychologytoday.com/blog/the-human-beast/201402/why-are-educated-people-more-likely-be-atheists