2.4 Thinking Critically About Sources

a puzzle piece
Evaluating sources often involves piecing together clues.

[1]This section teaches you how to identify relevant and credible sources that you have most likely turned up on the Web and on your results pages of the library catalog, Google Scholar, and specialized databases. Relevant, credible sources will meet the information needs of your research project.

In order to evaluate a source, you have to answer two questions about it:

  1. Is this source relevant to my research question?
  2. Is this a credible source– a source my audience[2] and I should be able to believe?
It’s important to determine relevance before credibility because no matter how credible a source is, if it’s not relevant to your research question it’s useless to you for this project.

Tip: Other Criteria from Your Professor

Don’t forget that you also have to make sure your sources meet any other criteria that your professor may have given you for this assignment. For instance, professors often stipulate that some of your sources have to be scholarly sources or articles from a particular database, so make sure you have identified enough of the kind of sources your professor has requested.

You might already be worrying about how long evaluating sources is going to take. When you begin your preliminary research, you won’t have to read all of a source to decide whether it is relevant and credible[3]. However, before you use the source as evidence (direct quotes, paraphrases, and summaries), you will have to read the entire source in order to be able to ethically use the information.

Nonetheless, our advice is to not begrudge the time you spend evaluating sources. It’s one of the most important things to learn at college—the opportunity to evaluate sources is one of the key reasons your professors assign research projects. For the rest of your professional and personal life, you will be using the critical thinking skills that make choosing the right sources possible, so learning those skills is a good investment.

Happily, you’ll also get faster the more you do it.

Making Inferences: Good Enough for Your Purpose?

Sources should always be evaluated relative to your purpose – why you’re looking for information. But because there often aren’t clear-cut answers when you evaluate sources, most of the time it is inferences – educated guesses from available clues – that you have to make about whether to use information from particular sources.

Your information needs will dictate:

  • What kind of information will help.
  • How serious you consider the consequences of making a mistake by using information that turns out to be inaccurate. When the consequences aren’t very serious, it’s easier to decide a source and its information are good enough for your purpose; though, there’s a lot to be said for always having accurate information, regardless of the consequences of using inaccurate information.
  • How hard you’re willing to work to get the credible, timely information that suits your purpose (what you’re learning here will make it easier).

Thus, your standards for relevance and credibility may vary, depending on whether you need, say

  • Information about a personal health problem.
  • An image you can use on a poster.
  • Evidence to win a bet with a rival in the dorm.
  • Dates and times a movie is showing locally.
  • A game to have fun with.
  • Evidence for your argument in a term paper.

For your research assignments or a health problem, the consequences may be great if you use information that is not relevant or not credible.

Activity: Quick Check[4]

  1. What’s an inference?
    •  a. An educated guess
    •  b. An assumption
    •  c. The obvious answer
  2. One mark of an educated person is
    • a. Being argumentative
    • b. A gold star on the forehead
    • c. A habit of thinking about whether a source of information is relevant and credible
    • d. Having more friends
  3. You will always have the same standards for accuracy and reliability when you look for information
    • a. False
    • b. True

See footnote for answers.

Evaluating for Relevancy

Relevant sources are those that pertain to your research question. You’ll be able to figure that out fairly quickly by reading or skimming particular parts of sources. We’ll show you how below, including where to look in specific kinds of sources and what questions to ask yourself as you do.

One thing to consider early on as you make inferences about relevancy is the effect that timeliness, or a source’s currency, should have on deciding whether a source is relevant.

Your research question will determine that.

For instance, if your research question is about the life sciences, you probably should consider only the most recent sources relevant because the life sciences are changing so quickly. There is a good chance that anything but the most recent sources may be out of date. So aim for sources no more than 5 years old. (An example discipline that calls for even newer sources is computer security.)

But suppose your research question is about the Edo Period in Japan (1603-1868) or about Robert Falcon Scott, who explored the Antarctic from 1901-1913. In these cases, an item from 1918 might be just as useful as an item from 2018 (although new information may have been found in the 100 year gap). But something from 1899 about Antarctica or from 1597 about Japan would NOT be current enough for these research questions.

These example research questions also give you two more clues about how to treat the timeliness or currency of sources as you consider relevance:

  • Because of how long ago they lived or occurred, it would be unusual for many sources on Robert Scott or the Edo Period to have been published very recently. So, unlike sources for the life sciences, whether a source is very recent should probably not determine its relevancy to those research questions.
  • Primary sources might be considered especially relevant to all three research questions. Life science journal articles that provide research findings for the first time count as primary sources. And primary sources (such as Scott’s diaries and expedition photographs, as well as paintings, literature, clothing, and household items from the Edo Period) go a long way in explaining faraway people and times.

For your sources for which timeliness matters, see the section below titled “Where to Look,” which includes where to look in websites, articles, and books for information about a source’s currency.

Time-Saving Tips

Instead of thinking you have to read all of every source in order to figure out whether it is relevant, read or skim only parts of each source. If you’re looking at the right parts, that should give you enough information to make an educated guess about relevancy.

But what should you be looking for as you do that reading and skimming? One way to figure that out is to first parse your research question so that you can figure out its main concepts. (This is like identifying the main concepts in your research question in order to search precisely.)

For instance, suppose your research question is: How does having diverse members in a group increase the critical thinking of the group?

What are this question’s main concepts? Our answer is: group diversity and critical thinking.

So when trying to judge which sources are relevant to these main concepts, you would assess whether each source you’ve found pertains to at least one of these concepts. We recommend you use a table like the one in the example below to keep track of which sources address each main concept.

To be considered relevant to your research question, a source wouldn’t necessarily have to cover all of your main concepts, but finding sources that do is ideal. Don’t forget that each source would have to pass the currency test, too, if currency is important to your research question. So it’s wise to record your decisions about the sources’ currency on your tables, too.

Example: Sources’ Main Concepts and Currency

Research question: How does having diverse members in a group increase the critical thinking of the group?

Source Currency Okay Group Diversity Critical Thinking
Source A title X X
Source B title X
Source C title X X X

The table in this hypothetical example indicates that both Sources A and C are relevant because each pertains to at least one main concept from the research question. Currency doesn’t seem to matter much to our research question, so all three sources were marked current. But since currency is all that Source B has to offer, it is not relevant for this project.

 

If you do make little tables for relevance, it’s probably a good idea to hang on to them. You might find them helpful later in your research process.

Where to Look in Websites, Articles, and Books

There are certain places to look and questions to ask yourself to assess three kinds of sources’ relevancy to your research question. Whatever you do, don’t stop evaluating a source after looking only at a website’s name or the title of another source.

Save time by looking in particular places in sources for information that will help you figure out whether the source is relevant to your research project.

Tips and Tricks[5]

On a website, check the name of the website and its articles for clues that they contain material relevant to your research question. Consider whether time should have an impact on what information can be considered relevant. If so, skim any dates, datelines, “What’s New pages,” and press releases to see whether any website content works with the time considerations you need. Page creation or revision dates can also help.

Skim any site map and index on the website for keywords related to your research question. Try the keywords of your research question in the search box. Do you see enough content about your keywords to make you think parts of the website could be helpful?

For an article, think about the title. Does it have anything to do with your research question? Consider whether time should have an impact on what sources can be considered relevant. If so, is the publication date within your parameters? For instance, if there is a time period in your research question, does the article address the same time period, or was it created in that time period?

Look at the intro and section headings in the article to locate the problem or question that the article addresses, its solution, and the outline of the article’s argument for its main claim. Can those help answer your research question? Do they make it seem like the article will give you information about what others have written about your research question? Do they offer a description of the situation surrounding your research question?

Do the article’s introduction and conclusion sections help you answer your research question and/or offer a description of the situation surrounding your question so you can explain in your final product why the question is important? Check whether the bibliography contains keywords related to your research question. Do the sources cited in the bibliography pertain to your research question?

For a book, check whether the title indicates the book could be about your research question. Consider whether time should have an impact on what sources can be considered relevant. If so, is the publication date or copyright date (usually listed in the library catalog or a few pages after the book’s title page) too early or late for any time constraints in your research question? Also, skim some of the preface and introduction to see whether the book works with the time considerations you need.

For help answering your research question, skim the book’s table of contents and any summary chapters to locate the problem or question that the book addresses, its solution, and the broad outline of the book’s argument for its main claim. Do they also give you information about what others have written about your research question? Do they offer a description of the situation surrounding your research question? Look for your keywords in the bibliography. Do the sources cited pertain to your research question? Skim the index for topics with the most page references. Do the topics with the most page references pertain to your research question?

Activity: Follow a Title’s Clues for Relevance[6]

Instructions: This activity asks you to use logic, the titles of sources, and their publication dates, and identify the source most likely to be relevant to each research question. (Outside of this activity, sources are not actually in competition with one another to be relevant. But this seemed a good way to have you practice your skills at assessing relevance.) Many titles below are imaginary, but that doesn’t affect their relevance in the activity. Note: Book, journal, and newspaper titles are italicized; chapter and article titles are in quotes.
  1. Research question: How would American journalism change if the federal government started sponsoring at least one newspaper in every state?
    1. a. Source: “The Revolution Years, 1775-1783” in History of American Journalism, published in 2016.
    2. b. Source: The Death and Life of American Journalism: The Media Revolution that Will Begin the World Again, published in 2009.
    3. c. Source: The New York Times: Media Master, published in 2014
  2. Research question: How does “prospect theory” in economics help explain medical doctors’ decisions to favor surgery or radiation to cure cancer in patients?
    1. a. Source: “Medical Decisions and Reasons for Variance,” in Physicians’ Notebook, 2017.
    2. b. Source: Cancer Treatment in Older Americans, published in 1999.
    3. c. Source: “Cancer Treatment Prescription–Advancing Prospect Theory beyond Economics,” in Journal of The American Medical Association Oncology, June, 2016.
  3. Research question: Why have some Big Data mathematical models and predictive algorithms been criticized as being unfair?
    1. a. Source: Weapons of Math Destruction, published in 2016.
    2. b. Source: What is Big Data? published in 2006.
    3. c. Source: “Who Buys Google’s Ads?” in The Wall Street Journal, November, 2017.

See footnote for answers.

 

Evaluating for Credibility

Next, you’ll be evaluating each of the sources that you deemed relevant.

What are the clues for inferring a source’s credibility? Let’s start with evaluating websites since we all do so much of our research online. But we’ll also include where to find clues relevant to sources in other formats when they differ from what’s good to use with websites. Looking at specific places in the sources will mean you don’t have to read all of every resource to determine its worth to you.

And remember, the more you take these steps, the faster it goes because always examining your sources becomes second nature.

What Used to Help

It used to be easier to draw conclusions about a source’s credibility, depending on whether it was a print source or a web source. We knew we had to be more careful about information on the web simply because all the filters that promoted accuracy involved in the print publishing process were absent from most web publishing. After all, it takes very little money, skill, and responsible intent to put content on the web, compared with what has to be done to convince print publishers that your content is accurate and that they will make money by printing it.

However, many publishers who once provided only print materials have now turned to the web and have brought along their rigorous standards for accuracy. Among them are the publishers of government, university, and scholarly (peer-reviewed) journal websites. Sites for U.S. mainline news organizations also strive for accuracy rather than persuasion because they know their readers have traditionally expected it. All in all, more websites now take appropriate care for accuracy than what used to be true on the web.

Nonetheless, it still remains very easy and inexpensive to publish on the web without any of the filters associated with print. So, we all still need the critical thinking skills you’ll learn here to determine whether websites’ information is credible enough to suit your purpose.

5 Factors to Consider

Evaluating a website for credibility means considering the five factors below in relation to your purpose for the information. These factors are what you should gather clues about and use to decide whether a site is right for your purpose.

  • The source’s neighborhood on the web
  • Author and/or publisher’s background
  • The degree of bias
  • Recognition from others
  • The thoroughness of the content

How many factors you consider at any one time depends on your purpose when seeking information. In other words, you’ll consider all five factors when you’re looking for information for a research project or other high-stakes situation where making mistakes have serious consequences. But you might consider only the first three factors at other times.

A Source’s Neighborhood

To understand this concept and begin to use it, imagine that all the sites on the web constitute a community. Just like in a geographical community, there are neighborhoods in which individual sites hang out.

Thinking about what neighborhood a source is in on the web can help you decide whether the site is credible and suits your purpose.

The Outernet is a comic that depicts common web sites as parts of a city - eBay is the mall, Google is the library, YouTube is the movie theatre, Facebook is a park, and Twitter is the town square. Transportation modes like buses and bikes are internet service providers.
Visualize the web as a community. (Image source: John Atkinson,
Wrong Hands)

Tip: Author’s Purpose for Print

Rather than examine print sources for their web neighborhood, examine them for their author’s purpose. Read the introduction and conclusion and look at the table of contents to discern the author’s purpose.

For instance, did the author intend to use the book or magazine article to inform/educate, persuade, sell, or entertain?

And is the author’s purpose suitable for your purpose? For instance, does the fact that a resource was intended to persuade mean it can’t help you answer your research question?

Clues About a Website’s Neighborhood

On a website, check pages labeled About Us, About This Site, Mission, Site Index, and Site Map, if available. (If such pages or similarly labeled ones don’t exist, it may be a sign that the site may be less trustworthy.)

Ask yourself these questions to gather clues that will help you decide what neighborhood you’re in:

  • Is the site selling products and/or services (even if there are articles and other useful information, too)? Perhaps it’s a retail, service center, or corporate site.
  • Are there membership applications and requests for contributions of money or time anywhere on the site? They’re usually a sign that you’re on a site that promotes particular ideas or behavior – in other words, they’re in the advocacy neighborhood.
  • Do postings, articles, reports, and/or policy papers give a one-sided view or multiple views on issues, people, and events? If they’re one-sided, the site is probably a commercial site or in the advocacy group neighborhood. If the information is even-handed and includes different sides of an issue, the site is more likely to be on the library/museum, school, or mainline U.S. news side of town. Sites there usually provide information designed to educate rather than persuade. Newspapers, online or in print, usually do have editorial pages, however. But labeling opinions as such helps keep mainline U.S. news sources in the newsstand neighborhood and out of the advocacy neighborhood.

Activity: Neighborhoods on the Web

Which Neighborhood?[7]

Instructions: Consider each website and choose its neighborhood on the Web.
  1. In which online neighborhood is OWL?
    1. a. Service sites
    2. b. Schools
    3. c. Entertainment centers
    4. d. Newsstands
  2. In which online neighborhood is Sequoia and Kings Canyon National Parks?
    1. a. Government
    2. b. Advocacy group
    3. c. Retail store
    4. d. Service center
  3. In which online neighborhood does Project Vote Smart best fit?
    1. a. Government
    2. b. Advocacy group
    3. c. Libraries (museums, etc.)
    4. d. Retail

See footnote for answers.

Examples

Think we’re making a mountain out of a molehill about being careful about web sources? Please click the links below to look at three websites. Is there an inference(s) you can make that applies to all three? Perhaps whether a website looks professionally done is not enough to insure that it is credible.

Making the Inference

Consider the clues. Then decide the extent that the site’s neighborhood is acceptable for your purpose. It might help to grade the extent that this factor contributes to the site being suitable on a scale like this one:

  • A – Very Acceptable
  • B – Good, but could be better
  • C – OK in a pinch
  • D – Marginal
  • F – Unacceptable

You’ll want to make a note of the resource’s grade for the neighborhood so you can combine it later with the grades you give the other factors.

Author and Publisher

A seal of approval
The reputation of the author and publisher influences your confidence in a source.

You’ll always want to know who’s providing the information for a website or other source. Do they have the education, training, or other experience that makes you think they are authorities on the subject covered? Or do they just have opinions?

The more you know about the author and/or publisher, the more confidence you can have in your decision for or against using content from that source.

Authors and publishers can be individuals or organizations, including companies. (Webmasters put things on the site but do not usually decide on everything, content-wise, except for the smallest websites. They often just carry out others’ decisions.)

Sites that do not identify an author or publisher are generally considered less credible for many purposes, including for term papers and other high-stakes projects. The same is true for sources in other formats.

Clues About an Author and/or Publisher’s Background

If they’re available, first take a look at pages called things such as About This Site, About Us, or Our Team first.

  • But you may need to browse around a site further to determine its author. Look for a link labeled with anything that seems like it would lead you to the author. Other sources, like books, usually have a few sentences about the author on the back cover or on the flap inside the back cover.
  • You may find the publisher’s name next to the copyright symbol, ©, at the bottom of at least some pages on a site. In books, the identity of the publisher is traditionally on the back of the title page.

Sometimes it helps to look for whether a site belongs to a single person or to a reputable organization.

  • Because many colleges and universities offer blog space to their faculty, staff, and students who use the university’s web domain, this evaluation can require deeper analysis than just looking at the address. Personal blogs may not reflect the official views of an organization or meet the standards of formal publication.
  • In a similar manner, a tilde symbol (~) preceding a directory name in the site address indicates that the page is in a “personal” directory on the server and is not an official publication of that organization.
    For example, you could tell that Jones’ web page was not an official publication of XYZ University if his site’s address was: http://www.XYZuniversity.edu/~jones/page.html. The tilde indicates it’s just a personal web page—in the Residences, not Schools, neighborhood of the web.
  • Unless you find information about the author to the contrary, such blogs and sites should not automatically be considered to have as much authority as content that is officially part of the university’s site. Or you may find that the author has a good academic reputation and is using their blog or website to share resources he or she authored and even published elsewhere. That would nudge him or her toward the Schools neighborhood.
  • Learning what they have published before can also help you decide whether that organization or individual should be considered credible on the topic.
Tip: Find Out What the Author (Person or Organization) Has Published

Library Catalogs – Search in a large library catalog to find books/sources written by the author.

For example:

Web Article Database – Use a free web article database to search for articles and books by the author that the college may not have a subscription to. Note: While you can search for free, you may not be able to retrieve articles unless searching through a library.

For example:

Specialized Database – Locate articles written by the author by using a specialized database that covers the same topical area as information on the website. Check your library’s website to find databases that you can use for this purpose. (Such databases are also called periodical indexes.)

For example:

Extra Tip: Find Out What Has Been Written About The Author

Web Search Engine – Use a search engine to find web pages where the author’s name is mentioned. (Be sure to search for the name as a phrase, as in “Jane Doe”)

For example:

 

Making the Inference

Consider the clues. Then decide the extent that the source’s author and/or publisher is acceptable for your purpose. It might help to grade the extent that this factor contributes to the site being suitable on a scale like this one:

  • A – Very Acceptable
  • B – Good, but could be better
  • C – OK in a pinch
  • D – Marginal
  • F – Unacceptable

You’ll want to make a note of the source’s grade for the author and/or publisher so you can combine it later with the grades you give the other factors.

Degree of Bias

Probably all sources exhibit some bias, simply because it’s impossible for their authors to avoid letting their life experience and education have an effect on their decisions about what is relevant to put on the site and what to say about it.

But that kind of unavoidable bias is very different from a wholesale effort to shape the message so that the site (or other source) amounts to a persuasive advertisement for something important to the author.

A balance scale
Look for evidence of bias in your sources.

Even if the effort is not as strong as a wholesale effort, authors can find many—sometimes subtle—ways to shape communication until it loses its integrity. Such communication is too persuasive, meaning the author has sacrificed its value as information in order to persuade.

While sifting through all the web messages for the ones that suit your purpose, you’ll have to pay attention to both what’s on the sites and in your own mind.

That’s because one of the things that gets in the way of identifying evidence of bias on websites is our own biases. Sometimes the things that look most correct to us are the ones that play to our own biases.

Clues About Bias

Review the website or other source and look for evidence that the site exhibits more or less bias. The factors below provide some clues.

Coverage
Unbiased: This source’s information is not drastically different from coverage of the topic elsewhere. Information and opinion about the topic don’t seem to come out of nowhere. It doesn’t seem as though the information has been shaped to fit. Biased: Compared to what you’ve found in other sources covering the same topic, this content seems to omit a lot of information about the topic, emphasize vastly different aspects of it, and/or contain stereotypes or overly simplified information. Everything seems to fit the site’s theme, even though you know there are various ways to look at the issue(s).
Citing Sources
Unbiased: The source links to any earlier news or documents it refers to. Biased: The source refers to earlier news or documents, but does not link to the news report or document itself.
Evidence
Unbiased: Statements are supported by evidence and documentation. Biased: There is little evidence and documentation presented, just assertions that seem intended to persuade by themselves.
Vested Interest
Unbiased: There is no overt evidence that the author will benefit from whichever way the topic is decided. Biased: The author seems to have a “vested interest” in the topic. For instance, if the site asks for contributions, the author probably will benefit if contributions are made. Or, perhaps the author may get to continue his or her job if the topic that the website promotes gets decided in a particular way.
Imperative Language
Unbiased: Statements are made without strong emphasis and without provocative twists. There aren’t many exclamation points. Biased: There are many strongly worded assertions. There are a lot of exclamation points.
Multiple Viewpoints
Unbiased: Both pro and con viewpoints are provided about controversial issues. Biased: Only one version of the truth is presented about controversial issues.
Examples: Bias

Look at the following cites to see some of the above traits in action.

Making the Inference

Consider the clues. Then decide the extent that the bias you detected in the source is acceptable for your purpose. It might help to grade the extent that this factor contributes to the site being suitable on a scale like this one:

  • A – Very Acceptable
  • B – Good, but could be better
  • C – OK in a pinch
  • D – Marginal
  • F – Unacceptable

You’ll want to make a note of the source’s grade for bias so you can combine it later with the grades you give the other factors.

Recognition from Others

Checking to see whether others have linked to a website, tagged, or cited it lets you know who else on the web recognizes the value of the site’s content. Reader comments and ratings can also be informative about some sites you may be evaluating, such as blogs.

If your source is a print book, the blurbs on the front or back cover give you information from authors, experts, or other well-known people who were willing to praise the book and/or author. The same kind of “mini-reviews” may be available on the publisher’s website. You can also look for reviews of the book or other sources by using Google and Google Scholar.

Those links, tags, bookmarks, citations, and positive reader comments and ratings are evidence that other authors consider the site exemplary. Book reviews, of course, may be either positive or negative.

a rating of 2 1/2 stars
Ratings and positive comments and reviews are evidence that others find a source valuable.

Exactly which individuals and organizations are doing the linking, tagging, citing, rating, and commenting may also be important to you. There may be some company you’d rather your site not keep! Or, maybe the sites that link to the one you’re evaluating may help solidify your positive feelings about the site.

Don’t let an absence of links, tags, citations, ratings, and comments damn the site in your evaluation. Perhaps it’s just not well-known to other authors. The lack of them should just mean this factor can’t add any positive or negative weight to your eventual decision to use the site—it’s neutral.

Tip: Peer Review and Citation as Recognition

The peer review most articles undergo before publication in a scholarly journal lets you know they’re considered by other scholars to be worth publishing. You might also be interested to see to what extent other researchers have used an article after it was published. (That use is what necessitates their citation.) But keep in mind that there may not be any citations for relatively new popular magazines, blogs, or scholarly journal articles.

Activity: Influence You?
Would the blurb on the front cover of Redirect by psychologist Timothy Wilson influence you positively or negatively in your evaluation of the book?
The front cover of Redirect by psychologist Timothy Wilson.
The blurb says: “There are few academics who write with as much grace and wisdom as Timothy Wilson. Redirect is a masterpiece. – Malcolm Gladwell”

 

Clues about Recognition

Use Bing to find sites that link to a particular URL.

Google does not adequately support this.

Enter inbody:[URL of known site] in the Bing search box.

As you look over the results page, always pay attention to more than the number of sites linking to the site about which you’re seeking information. Also, consider the kind of sites that do that linking. Are they the kind of sites you are pleased to see associated with the site you’re interested in? If you are doing academic writing, you should want them to be professional, scholarly, and/or mainline news sources—not, for instance, something from the entertainment neighborhood.

If you prefer recognition from a particular neighborhood, you can ask Bing to look specifically for links from that neighborhood.
Suppose, for instance, you wonder whether scholarly sources link to the site above but don’t want to look for them among all that you turned up before. You should type in the Bing search box inbody:www.deathpenaltyinfo.org site:edu in order to see whether any scholarly sources link to that site.

Find citations of an article. Although there is no simple way to find every source that cites an article in a popular magazine, a blog, or a scholarly journal, there are some ways to look for these connections.

  • For articles published in popular magazines or blogs, enter the title of the article in quotes in the search box of a search engine like Google. The resulting list should show you the original article you’re evaluating, plus other sites that have mentioned it in some way. Click on those that you want to know more about.
Activity: Inferences

Use Bing to determine how many scholarly sites have made links to these sites. Copy each search statement below and put it in the Bing search box to make your searches.

  • inbody:www.DOAJ.org site:edu
  • inbody:www.nvic.org site:edu

Sometimes you’ll even have to take a look at the linking organization’s document in order to figure out why it’s linking. For instance, is it linking to a bad example that it wants to show? As with all source evaluations,  you’ll have to make inferences.

  • For articles published in scholarly journals, use Google Scholar to enter the title of the article in quotes. In the results list, find the article you’re evaluating. (Many articles have similar titles.) Look for the number of citations in the lower left of the listing for your article. If you want more information on the authors who have done the citing, click on the citation number for a clickable list of articles or papers and get the names of authors to look up at the end of the articles or with a search engine. (This is a good way to discover more articles about your topic, too.)
A Google Scholar search for “Massillion Diplomacy before the Second Punic War” shows over 30 other publications have cited the work.
Google Scholar shows how many articles have cited a given article.
View the live example.
Making the Inference

Consider the clues. Then decide the extent that the source’s recognition from others is acceptable for your purpose. It might help to grade the extent that this factor contributes to the site being suitable on a scale like this one:

  • A – Very Acceptable
  • B – Good, but could be better
  • C – OK in a pinch
  • D – Marginal
  • F – Unacceptable

You’ll want to make a note of the source’s grade for recognition so you can combine it later with the grades you give the other factors.

Thoroughness

A checklist on a clipboard
Consider how well a source covers your topic.

Figuring out whether a website or other source is suitable for your purpose also means looking at how thoroughly it covers your topic

You can evaluate thoroughness in relation to other sources on the same topic. Compare your source to how other sources cover the material, checking for missing topics or perspectives.

Clues About Thoroughness

Click around a site to get some idea of how thoroughly it covers the topic. If the source you are evaluating is a print resource, read the introduction and conclusion and also the table of contents to get a glimpse of what it covers. Look at the index to see what subject is covered with the most pages. Is it thorough enough to meet your information need?

Tip: Related Sites

Use Google to find other sites on the same topic by entering related:[the URL of the site you know] in the search box.

For example: related:guides.osu.edu

Use this technique to browse other sites Google turns up. Do other sites cover aspects of the topic that are missing from the site you are evaluating? Or does your site stack up pretty well against the competition?

Making the Inference

Consider the clues. Then decide the extent that the source’s thoroughness is acceptable for your purpose. It might help to grade the extent that this factor contributes to the source being suitable on a scale like this one:

  • A – Very Acceptable
  • B – Good, but could be better
  • C – OK in a pinch
  • D – Marginal
  • F – Unacceptable

You’ll want to make a note of the source’s grade for thoroughness so you can combine it later with the grades you give the other factors.

Combining the Factors

Once you’ve considered each factor used in evaluating a source, it’s important to take a look at the inferences you made about them. Now is the time to look at those grades all together—to average them if you’ve been assigning grades—and to make one more inference.

Taking the grade on each factor into account, can you infer that the source is credible enough for your purpose? If it isn’t, this is one source that can’t be helpful in your project. If it is relevant and credible enough, you can use information from that resource with confidence.

Making the Final Inference

Assume you’re writing a term paper and are considering using information from Site XYZ. You ran through the evaluation process as you looked over the site, and you made notes about the grades you assigned.

The grades you gave individual factors are:

  • Neighborhood: A
  • Author/publisher’s background: B
  • Degree of bias: A
  • Recognition from others: No Evidence
  • Thoroughness: C
  • Currency of the content: A

You average the grades (A=4, B=3, C=2, D=1, F=0), remembering not to include the factor on which you gave no grade. The score was 3.4, about a B, which is a “Good, but could be better” score on the scale we used in this tutorial.

Would you use this source in your paper? This answer may vary depending on what your other sources look like. Is this your highest grade of source? Will this be a heavily used source in your paper as you work to prove your point? The answers to questions like these may push you to keep looking or include this source in your final paper.

When Should You Stop?

Research shows that students often don’t know when they should stop trying to find and evaluate sources for a particular project.[8] How many sources are enough? It’s hard to say, exactly. But you’ll need enough to meet the information needs of your project and to meet the requirements your professor told you about.

Furthermore, you may change your mind as you continue working on your project. There is probably not a researcher alive who hasn’t thought he or she had enough relevant sources, only to change their mind later when they were actually writing the final product.

The process of finding, reading, and evaluating sources can very rewarding; however, as with most things in life, it can also be extremely frustrating and at times you may feel defeated. Learning how to find, read, and evaluate sources is a skill that academics (even your professor) are continually learning more about and developing their skills in. So, before you reach the point where you fear your hair may be close to falling out completely, reach out to your professor. They have not only been trained in these same skills and environment, but because they will serve as a new set of eyes on the topic, they can oftentimes point you in a direction you haven’t thought of.[9]

 

Media Attributions

  • Lindsey MacCallum and Teaching & Learning Ohio State University Libraries
  • Image source: John Atkinson, Wrong Hands https://wronghands1.com/2012/03/01/the-outernet/
  • Lindsey MacCallum and Teaching & Learning Ohio State University Libraries
  • Ibid.
  • Ibid.
  • Screenshot provided by Lindsey MacCallum and Teaching & Learning Ohio State University Libraries
  • Screenshot provided by Lindsey MacCallum and Teaching & Learning Ohio State University Libraries
  • Lindsey MacCallum and Teaching & Learning Ohio State University Libraries

  1. 2.4 (except where otherwise noted) was borrowed with minor edits and additions from Choosing & Using Sources: A Guide to Academic Research, 1st Canadian Edition by Lindsey MacCallum and Teaching & Learning Ohio State University Libraries which is licensed under a Creative Commons Attribution 4.0 International License
  2. Like we discussed in earlier sections, understanding who your audience is (i.e. part of academia, peers, experts, etc.) is vital in determining what type of sources you need and what qualifications make those sources credible for that audience.
  3. Go back to section 2.3 to review how to read a scholarly article to make determining the usefulness of a source a smoother process.
  4. 1. a; 2. c; 3. a
  5. Much of this advice comes from “Speedy Reading” in The Craft of Research, second edition, by Wayne Booth, Gregory Colomb, and Joseph Williams, 2003, pp. 108-109.
  6. 1. b; 2. c; 3. a
  7. 1. b; 2. c; 3. c
  8. Head, A., & Eisenber, M. (2010). Truth be told: How college students evaluate and use information in the digital age. SSRN. http://dx.doi.org/10.2139/ssrn.2281485
  9. To be honest, most of us (us being college instructors) find this stage of the research process to be enjoyable. So, do not avoid reaching out for fear of “bugging” us.

License

Share This Book