2.1 Overview of information

Because a literature review is a summary and analysis of the relevant publications on a topic, we first have to understand what is meant by ‘the literature’.  In this case, ‘the literature’ is a collection of all of the relevant written sources on a topic.  It will include both theoretical and empirical works.  Both types provide scope and depth to a literature review.

2.1.1 Disciplines of knowledge

When drawing boundaries around an idea, topic, or subject area, it helps to think about how and where the information for the field is produced. For this, you need to identify the disciplines of knowledge production in a subject area.

Information does not exist in the environment like some kind of raw material. It is produced by individuals working within a particular field of knowledge who use specific methods for generating new information. Disciplines are knowledge-producing and -disseminating systems which consume, produce and disseminate knowledge. Looking through a  course catalog of a post-secondary educational institution gives clues to the structure of a discipline structure. Fields such as political science, biology, history and mathematics are unique disciplines, as are education and nursing, with their own logic for how and where new knowledge is introduced and made accessible.

You will need to become comfortable with identifying the disciplines that might contribute information to any search strategy. When you do this, you will also learn how to decode the way how people talk about a topic within a discipline. This will be useful to you when you begin a  review of the literature in your area of study.

For example, think about the disciplines that might contribute information to a the topic such as  the role of sports in society. Try to anticipate the type of perspective each discipline might have on the topic. Consider the following types of questions as you examine what different disciplines might contribute:

• What is important about the topic to the people in that discipline?
• What is most likely to be the focus of their study about the topic?
• What perspective would they be likely to have on the topic?

In this example, we identify two disciplines that have something to say about the role of sports in society: allied health and education. What would each of these disciplines raise as key questions or issues related to that topic?

2.1.1.1 Nursing

• how sports affect individuals’ health and well-being
• assessing and treating sports injuries
• physical conditioning for athletes

2.1.1.2 Education

• how schools privilege or punish student athletes
• how young people are socialized into the ideal of team cooperation
• differences between boys’ and girls’ participation in organized sports

We see that a single topic can be approached from many different perspectives depending on how the disciplinary boundaries are drawn and how the topic is framed. This step of the research process requires you to make some decisions early on to focus the topic on a manageable and appropriate scope for the rest of the strategy. (Hansen & Paul, 2015).

‘The literature’ consists of the published works that document a scholarly conversation in a field of study. You will find, in ‘the literature,’ documents that explain the background of your topic so the reader knows where you found loose ends in the established research of the field and what led you to your own project.  Although your own literature review will focus on primary, peer-reviewed resources, it will begin by first grounding yourself in background subject information generally found in secondary and tertiary sources such as books and encyclopedias.  Once you have that essential overview, you delve into the seminal literature of the field. As a result, while your literature review may consist of research articles tightly focused on your topic with secondary and tertiary sources used more sparingly, all three types of information (primary, secondary, tertiary) are critical to your research.

2.1.2 Definitions

• Theoretical – discusses a theory, conceptual model or framework for understanding a problem.
• Empirical – applies theory to a behavior or event and reports derived data to findings.
• Seminal – “A classic work of research literature that is more than 5 years old and is marked by its uniqueness and contribution to professional knowledge.” (Houser, 4th ed., 2018, p. 112).
• Practical – “…accounts of how things are done” (Wallace & Wray, 3rd ed., 2016, p. 20). Action research, in Education, refers to a wide variety of methods used to develop practical solutions. (Great Schools Partnership, 2017).
• Policy – generally produced by policy-makers, such as government agencies.
• Primary – published results of original research studies .
• Secondary – interpret, discuss, summarize original sources
• Tertiary – synthesize or distill primary and secondary sources.  Examples include: encyclopedias, directories, dictionaries, handbooks, guides, classification, chronology, and other fact books.
• Grey literature – research and information released by non-commercial publishers, such as government agencies, policy organizations, and think-tanks.

‘The literature’ is published in books, journal articles, conference proceedings, theses and dissertations.  It can also be found in newspapers, encyclopedias, textbooks, as well as websites and reports written by government agencies and professional organizations. While these formats may contain what we define as ‘the literature’, not all of it will be appropriate for inclusion in your own literature review.

These sources are found through different tools that we will discuss later in this section. Although a discovery tool, such as a database or catalog, may link you to the ‘the literature’ not every tool is appropriate to every literature review.  No single source will have all of the information resources you should consult.  A comprehensive literature review should include searches in the following:

• Multiple subject and article databases
• Library and other book catalogs
• Grey literature sources

2.2 Information Cycle

To get a better idea of how the literature in a discipline develops, it’s useful to see how the information publication lifecycle works.  These distinct stages show how information is created, reviewed, and distributed over time.

Follow the image link to view the full tutorial.

The following chart can be used to guide you in searching literature existing at various stages of the scholarly communication process (freely accessible sources are linked, subscription or subscribed sources are listed but not linked):

Guide to searching for literature at various stages of the scholarly communication process

Steps in the Scholarly Communication Process	Publication Cycle	Access Points
Research and develop idea	Unpublished documents such as lab notebooks, personal correspondence, graphs, charts, grant proposals, and other ‘grey literature’	Limited access Google Scholar HSRR (Health Services and Sciences Research Resources) RePORTER (Database of NIH funded research projects) Institute of Education Sciences
Present preliminary findings	Preliminary reports: letters to the editor or journals, brief (short) communication submitted to a primary journal	PubMed (limiting search results to Letter under Limits) Web of Science (Science Citation Index)
Report research	Conference literature: preprints, conference proceedings	PapersFirst ProceedingsFirst Conference web sites Preprint services
	Research reports: master’s theses, doctoral dissertations, interim or technical reports	Dissertations & Theses British Library EThOS Theses Canada Portal Electronic Theses and Dissertations Center PubMed (limiting search results to Technical Report under Limits) Current Grey Literature Report Professional association web sites OpenDOAR
Publish research	Research paper (scholarly journal articles): research papers published in peer-reviewed/refereed journals	PubMed CINAHL PsycINFO Web of Science ERIC
Popularize research findings	Newspapers, popular magazines, TV news reports, trade publications, web sites	PubMed (limiting search results to News and Newspaper Article under Limits) Media outlets Internet search engines
Compact and repackage information	Reviews, systematic reviews, guidelines, textbooks, handbooks, yearbooks, encyclopedias	Cochrane Library Library Catalogs WorldCat

2.3 Information Types

To continue our discussion of information sources, there are two ways published information in the field can be categorized:

• Articles by the type of periodical in which an article it is published, for example, magazine, trade, or scholarly publications.
• Where the material is located in the information cycle, as in primary, secondary, or tertiary information sources.

2.3.1 Popular, Trade, or Scholarly publications

2.3.1.1 Types of Periodicals

Journals, trade publications, and magazines are all periodicals, and articles from these publications they can all look similar article by article when you are searching in the databases. It is good to review the differences and think about when to use information from each type of periodical.

2.3.1.2 Magazines

A magazine is a collection of articles and images about diverse topics of popular interest and current events.

Features of magazines:

• articles are usually written by journalists
• articles are written for the average adult
• articles tend to be short
• articles rarely provides a list of reference sources at the end of the article
• lots of color images and advertisements
• the decision about what goes into the magazine is made by an editor or publisher
• magazines can have broad appeal, like Time and Newsweek, or a narrow focus, like Sports Illustrated and Mother Earth News.

Popular magazines like Psychology Today, Sports Illustrated, and Rolling Stone can be good sources for articles on recent events or pop-culture topics, while Harpers, Scientific American, and The New Republic will offer more in-depth articles on a wider range of subjects. These articles are geared towards readers who, although not experts, are knowledgeable about the issues presented.

2.3.1.3 Trade Publications

Trade publications or trade journals are periodicals directed to members of a specific profession. They often have information about industry trends and practical information for people working in the field.

Features of trade publications:

• Authors are specialists in their fields
• Focused on members of a specific industry or profession
• No peer review process
• Include photographs, illustrations, charts, and graphs, often in color
• Technical vocabulary

Trade publications are geared towards professionals in a discipline. They report news and trends in a field, but not original research. They may provide product or service reviews, job listings, and advertisements.

2.3.1.4 Scholarly, Academic, and Scientific Publications

Scholarly, academic, and scientific publications are a collections of articles written by scholars in an academic or professional field. Most journals are peer-reviewed or refereed, which means a panel of scholars reviews articles to decide if they should be accepted into a specific publication. Journal articles are the main source of information for researchers and for literature reviews.

Features of journals:

• written by scholars and subject experts
• author’ credentials and institution will be identified
• written for other scholars
• dedicated to a specific discipline that it covers in depth
• often report on original or innovative research
• long articles, often 5-15 pages or more
• articles almost always include a list of sources at the end (Works Cited, References, Sources, or Bibliography) that point back to where the information was derived
• no or very few advertisements
• published by organizations or associations to advance their specialized body of knowledge

Scholarly journals provider articles of interest to experts or researchers in a discipline. An editorial board of respected scholars (peers) reviews all articles submitted to a journal. They decide if the article provides a noteworthy contribution to the field and should be published. There are typically few  little or no advertisements. Articles published in scholarly journals will include a list of references.

2.3.1.5 A word about open access journals

Increasingly, scholars are publishing findings and original research in open access journals.  Open access journals are scholarly and peer-reviewed and open access publishers provide unrestricted access and unrestricted use.  Open access is a means of disseminating scholarly research that breaks from the traditional subscription model of academic publishing. It is free of charge to readers and because it is online, it is available at anytime, anywhere in the world, to anyone with access to the internet.  The Directory of Open Access Journals (DOAJ) indexes and provides access to high-quality, peer-reviewed scholarly articles.

In summary, newspapers and other popular press publications are useful for getting general topic ideas. Trade publications are useful for practical application in a profession and may also be a good source of keywords for future searching. Scholarly journals are the conversation of the scholars who are doing research in a specific discipline and publishing their research findings.

2.3.1.6 Primary, Secondary, and Tertiary Sources

Primary sources of information are those types of information that come first. Some examples of primary sources are:

• original research, like data from an experiment with plankton.
• diaries, journals, photographs
• data from the census bureau or a survey you have done
• original documents, like the constitution or a birth certificate
• newspapers are primary sources when they report current events or current opinion
• speeches, interviews, email, letters
• religious books
• personal memoirs and autobiographies
• art work
• pottery or weavings

There are different types of primary sources for different disciplines.  In the discipline of history, for example, a diary or transcript of a speech is a primary source.  In education and nursing, primary sources will generally be original research, including data sets.

Secondary sources are written about primary sources to interpret or analyze them. They are a step or more removed from the primary event or item. Some examples of secondary sources are:

• commentaries on speeches
• critiques of plays, journalism, or books
• a journal article that talks about a primary source such as an interpretation of Steinbeck’s The Grapes of Wrath, or the flower symbolism of Monet’s water garden paintings
• textbooks (can also be considered tertiary)
• biographies
• encyclopedias
• websites

Tertiary sources are further removed from the original material and are a distillation and collection of primary and secondary sources. Some examples are:

• bibliography of critical works about an author
• textbooks (also considered secondary)
• factbooks
• guidebooks
• manuals

A comparison of information sources across disciplines:

2.4 Information Sources

In this section, we discuss how to find not only information, but the sources of information in your discipline or topic area.  As we see in the graphic and chart above, the information you need for your literature review will be located in multiple places.  How and where research and publication occurs drives how and where the information is located, which in turn determines how you will discover and retrieve it.  When we talk about information sources for a literature review in education or nursing, we generally mean these five areas: the internet, reference material and other books, empirical or evidence-based articles in scholarly, peer-reviewed journals, conference proceedings and papers, dissertations and theses, and grey literature.

2.4.1 Web

The World Wide Web can be an excellent place to satisfy some initial research needs.

• It is a good resource for background information and for finding keywords for searching in the library catalog and databases.
• It is a good tool for locating professional organizations and searching for information and the names of experts in a given discipline.
• Google Scholar is a useful discovery tool for citations, especially if you are trying to get the lay of the land surrounding your topic or if you are having a problem with keywords in the databases. You can find some information to refine your search terms. It is NOT acceptable to depend on Google Scholar for finding articles because of the spotty coverage and lack of adequate search features.

2.4.2 Books and Reference Sources

Reference materials and books are available in both print and electronic formats. They provide gateway knowledge to a subject area and are useful at the beginning of the research process to:

• Get an overview of the topic, learn the scope, key definitions, significant figures who are involved, and important timelines
• Discover the foundations of a topic
• Learn essential definitions, vocabulary terms, and keywords you can use in your literature searching strategy

2.4.3 Scholarly Articles in Journals

Another major category of information sources is scholarly information produced by subject experts working in academic institutions, research centers and scholarly organizations. Scholars and researchers generate information that advances our knowledge and understanding of the world. The research they do creates new opportunities for inventions, practical applications, and new approaches to solving problems or understanding issues.

Academics, researchers and students at universities make their contributions to scholarly knowledge available in many forms:

• masters’ theses
• doctoral dissertations
• conference papers
• journal articles and books
• individual scholars’ web pages
• web pages developed by the researcher’s’ home institution (Hansen & Paul, 2015).

Scholars and researchers introduce their discoveries to the world in a formal system of information dissemination that has developed over centuries. Because scholarly research undergoes a process of “peer review” before being published (meaning that other experts review the work and pass judgment about whether it is worthy of publication), the information you find from scholarly sources meets preset standards for accuracy, credibility and validity in that field.

Likewise, scholarly journal articles are generally considered to be among the most reliable sources of information because they have gone through a peer-review process.

2.4.5 Conference Papers & Proceedings

Conferences are a major source of  emerging research where researchers present papers on their current research and obtain feedback from the audience.  The papers presented in the conference are then usually published in a volume called a conference proceeding.  Conference proceedings highlight current discussion in a discipline and can lead you to scholars who are interested in specific research areas.

A word about conference papers: several factors contribute to making these documents difficult to find.  It may be months before a paper is published as a journal article, or it may never be published.  Publishers and professional associations are inconsistent in how they publish proceedings.  For example, the papers from an annual conference may be published as individual, stand-alone titles, which may be indexed in a library catalog, or the conference proceedings may be treated more like a periodical or serial and, therefore, indexed in a journal database.

It is not unusual that papers delivered at professional conferences are not published in print or electronic form, although an abstract may be available.  In these cases, the full paper may only be available from the author or authors.

The most important thing to remember is that if you have any difficulty finding a conference proceeding or paper, ask a librarian for assistance.

2.4.6 Dissertations and Theses

Dissertations and theses can be rich sources of information and have extensive reference lists to scan for resources. They are considered gray literature, so are not “peer reviewed”. The accuracy and validity of the paper itself may depend on the school that awarded the doctoral or master’s  degree to the author.

2.5 Conclusion

In thinking about ‘the literature’ of your discipline, you are beginning the first step in writing your own literature review.  By understanding what the literature in your field is, as well as how and when it is generated, you begin to know what is available and where to look for it.

We briefly discussed seven types of (sometimes overlapping) information:

• information found on the web
• information found in reference books and monographs
• information found in scholarly journals
• information found in conference proceedings and papers
• information found in dissertations and theses
• information found in magazines and trade journals
• information that is primary, secondary, or tertiary.

By conceptualizing or scoping how and where the literature of your discipline or topic area is generated, you have started on your way to writing your own literature review.

Finally, remember:

“All information sources are not created equal. Sources can vary greatly in terms of how carefully they are researched, written, edited, and reviewed for accuracy. Common sense will help you identify obviously questionable sources, such as tabloids that feature tales of alien abductions, or personal websites with glaring typos. Sometimes, however, a source’s reliability—or lack of it—is not so obvious…You will consider criteria such as the type of source, its intended purpose and audience, the author’s (or authors’) qualifications, the publication’s reputation, any indications of bias or hidden agendas, how current the source is, and the overall quality of the writing, thinking, and design.”  (Writing for Success, 2015, p. 448).

We will cover how to evaluate sources in more detail in Chapter 5.

References

Image Attribution

3.1 Topic selection

If the longest journey begins with the first step, most graduate-level literature reviews begin with choosing a relevant, appropriate, interesting topic about which to do the review.  Whether the topic is assigned, chosen from a list of possible options, or (most likely) developed on your own, a good way to begin your thinking is to take a general issue or subject and formulate it into a question. You may want to start to think about a single aspect in your field or discipline that might be interesting to pursue, such as ‘science education’ or ‘diabetes treatment.’

A good topic selection plan begins with a general orientation into the subject you are interested in pursuing in more depth. Although finding a good research question may initially feel like looking for a needle in a haystack, choosing a general topic is the first step.

Things to think about when choosing a topic area:

1. Pick an area of interest; pick an area of experience; or, pick an area where you know there is a need for more research.
2. It may be easier to start with “what” and “why” questions and expand on those. For example, in Nursing: what is current research on obesity and why is it significant to nursing and health sciences?  Or, in Education: what is media literacy and why is it significant to education sciences?
3. If you are a teacher or other education practitioner, you might think about a current problem in the workplace such as, classroom management or parent interaction and expand from there.  Nurses may want to consider a current issue in a clinical or hospital setting, like hand washing or patient falls.

Other suggestions for choosing a topic include:

• Ask a professor, preferably one active in research, about possible topics
• Read departmental information on research interests of the faculty. Faculty research interests areas vary widely, so do some research on their past publications. Most departmental websites post faculty CVs.
• Read a research paper that interests you.  The paper’s literature review or background section will provide insight into the research question the author was seeking to address with his/her study.  Is the research incomplete, imprecise, biased, or inconsistent?  As you’re reading the paper, look for what’s missing.  These may be “gaps in the literature” that you might explore in your own study. The conclusion or discussion section at the end may also offer some questions for future exploration.  A recent blog posting in Science (Pain, 2016) provides several tips from researchers and graduate students on how to effectively read these papers.
• Think about papers you enjoyed researching and writing as an undergraduate and choose a topic that reflects those interests
• Sift through the table of contents of annual reviews journals in your area of interest – such as, the Annual Review of Psychology, the Annual Review of Immunology, the Review of Research in Education, or the Annual Review of Nursing Research.
• Identify and browse journals related to your research interests. Faculty and librarians can help you identify relevant journals in your field and specific areas of interest.

Although it’s a good idea to avoid subjects that are too personal or emotional as these can interfere with an unbiased approach to the research, it’s also important to make sure you have more than a passing interest in the topic. You will be with this literature review for an extended period of time and it will be difficult to stick with it even under the best circumstances. A graduate student in psychology said, “‘My advice would be to NOT choose a topic that is an unappealing offshoot of your adviser’s work or a project that you have lukewarm feelings about in general…It’s important to remember that this is a marathon, not a sprint, and lukewarm feelings can turn cold quickly.’”  (Dittman, 2005).

3.2 Question formulation

Now, take that general idea and begin to think about it in terms of a question. What do you really want to know about the topic?  As a warm-up exercise, try dropping a possible topic idea into one of the blank spaces below.  The questions may help bring your subject into sharper focus and provide you with the first important steps towards developing your topic. The type of paper you want to write (Definition, Analysis, Narration, etc.) can also be a useful way to begin thinking about your research question.  For example, if you’re interested in parent involvement in early childhood education, your research question might be “What are the various features of parent involvement in early childhood education?”  Or, if you want to do an evaluative literature review, your research question could be “What is the value of infant vaccination?”

1. What does ___ mean? (Definition)
2. What are the various features of ___? (Description)
3. What are the component parts of ___? (Simple analysis)
4. How is ___ made or done? (Process analysis)
5. How should ___ by made or done? (Directional analysis)
6. What is the essential function of ___? (Functional analysis)
7. What are the causes of ___? (Causal analysis)
8. What are the consequences of ___? (Causal analysis)
9. What are the types of ___? (Classification)
10. How is ___ like or unlike ___? (Comparison)
11. What is the present status of ___? (Comparison)
12. What is the significance of ___? (Interpretation)
13. What are the facts about ___? (Reportage)
14. How did ___ happen? (Narration)
15. What kind of person is ___? (Characterization/Profile)
16. What is the value of ___? (Evaluation)
17. What are the essential major points or features of ___? (Summary)
18. What case can be made for or against ___? (Persuasion)
19. What is the relationship between  _____ and the outcome of ____? (Explorative)

For more information about how to form a research question, check out this video tutorial:

At this point, you will want to do an initial review of the existing literature to see what resources on your topic or question already exist.  Based on what you find, you may decide to alter your question in some way before going too far along a path that perhaps has already been well-covered by other scholars.

3.3 Research Question or Hypothesis?

Some things to keep in mind at this beginning stage of the research process is whether your literature review will be in the form of a research question or a hypothesis.  One way to determine that outcome is to compare the two and decide which format will work best for you.  For example, if the area you are researching is a relatively new field, and there is little or no existing literature or theory that indicates what you will find, then your literature review will likely be based on a research question.

3.3.1 Research question criteria:

The question should express a relationship between two or more variables – for example, how is  A related to B?  It should be clearly stated in a question form – such as, “How do grades (A) affect participation in class (B)?” or “How does parental education level (A) affect children’s vaccination status (B)?”  Your literature review, in turn, may become:

Grades as a classroom participation motivator: A literature review, or

Education level and vaccinations: A literature review

Your question should also imply possibilities for empirical testing–remember, metaphysical questions are not measurable and a variable that cannot be clearly defined cannot be tested.

3.3.2 Hypothesis criteria:

If, however, your literature review tests something based on the findings of a large amount of previous literature or a well-developed theory, your literature review will be to test of a hypothesis, rather than answer a question. The statement should indicate an expected relationship between variables and it must be testable.  State your hypothesis as simply and concisely as possible.  For example, if A, then B, as in: “If patient is obese, he/she will also be deaf.” (Dhanda & Taheri, 2017).  Or, “For those who stutter, unusual temperament or anxiety is a causal factor.” (Kefalianos, 2012)

Decide what type of relationship you would like to study between the variables.  Now, try to express the relationship between the concepts as a single sentence–in the form of either a research question or a hypothesis.

3.4 Refining the question

Once you have selected your topic area and reviewed literature related to it, you may need to narrow it to something that can be realistically researched and answered. In addition to asking Who, What, When, Where, Why, and How questions, other types of questions you might begin to ask to further refine your topic include those that are: Descriptive, Differential or Comparative, Associative or Relational.

You might beginning by asking a series of PICO questions. Although the PICO method is used primarily in the health sciences, it can also be useful for narrowing/refining a research question in the social sciences as well.   A way to formulate an answerable question using the PICO model could look something like this:

• Patient, Population or Problem: What are the characteristics of the patient or population?  For example, gender, age, other demographics. What is the situation or disease you are interested in?  For example, diabetes or classroom management

• Intervention or exposure: What do you want to do with the patient, person, or population (e.g. treat, diagnose, observe)?  Such as, observe classroom behavior or reaction to a specific type of treatment

• Comparison: What is the alternative to the intervention (e.g. placebo, different drug, surgery)?  For example, how does a sample group that is assigned homework compare to a similar group that is not assigned homework?

• Outcome: What are the relevant outcomes (e.g. morbidity, death, complications)?  For example, how do lower cholesterol numbers or improved scores in spelling impact the target population?

Some examples of how the PICO method is used to refine a research question include:

• Education: “Is play-based learning an effective approach in early childhood education? – Population (early childhood) / Intervention (play-based learning),” or “Can music therapy help autistic students improve their communication skills? – Population (autistic students  / intervention (music therapy)”
• Nursing: “What is the effect of a dressing with silver in its composition on the treatment of diabetic foot ulcers? – Population (patients with diabetes) / Intervention (dressings made with silver)” or “How effective are antidepressive medications on anxiety and depression? – Intervention (antidepressants) / Population (patients with anxiety and depression)”

Another mnemonic technique used in the social sciences for narrowing a topic is SPICE.  An example of how SPICE factors can be used to develop a research question is given below:

Setting – for example, Canada
Perspective – for example, Adolescents
Intervention – for example, Text message reminders
Comparisons – for example, Telephone message reminders
Evaluation – for example, Number of homework assignments turned in after text message reminder compared to the number of assignments turned in after a telephone reminder

Likewise, developing a concept map or mind map around your topic may help you analyze your question and determine more precisely what you want to research.  Using this technique, start with the broad topic, issue, or problem, and begin writing down all the words, phrases and ideas related to that topic that come to mind and then ‘map’ them to the original idea.

This mapping technique aims to improve  the “description of the breadth and depth of literature in a domain of inquiry. It also facilitates identification of the number and nature of studies underpinning mapped relationships among concepts, thus laying the groundwork for systematic research reviews and meta-analyses.” (Lesley, Floyd, & Oermann, 2002; D’Antoni & Pinto Zipp, G., 2006).  Its purpose, like the other methods of question refining, is to help you organize, prioritize, and integrate material into a workable research area; one that is interesting, answerable, realistic in terms of resource availability and time management, objective, scholarly, original, and clear.

Check out this YouTube video for more basic information on how to map your research question:

In addition to helping you get started with your own literature review, the techniques described here will give you some keywords and concepts that will be useful when you begin searching the literature for relevant studies and publications on your topic.

For example, perhaps your initial idea or interest is ‘how to prevent obesity.’ After an initial search of the relevant nursing literature, you realize the topic of ‘obesity’ is too broad to adequately cover in the time you have to do your literature review.  You decide to narrow your focus to ‘causes of childhood obesity.’ Using PICO factors you further narrow your search to ‘the influence of family factors on overweight children.’  A potential research question might then be “What maternal factors are associated with toddler obesity in the United States?”  You’re now ready to begin searching the literature for studies, reports, cases, and other information sources that relate to this question.

Similarly, for a broad topic like ‘school performance’ or ‘grades,’ and after an initial literature search that provides some variables, examples of a narrow research question might be:

• “To what extent does parental involvement in children’s education relate to school performance over the course of the early grades?”
• “Do parental involvement levels differ by family social, demographic, and contextual characteristics?”
• “ What forms of parent involvement are most highly correlated with children’s outcomes? What factors might influence the extent of parental involvement?” (Early Childhood Longitudinal Program, 2011).

References

Image Attribution

4.1 Overview of discovery

Discovery, or background research, is something that happens at the beginning of the research process when you are just learning about a topic. It is a search for general information to get the big picture of a topic for exploration, ideas about subtopics and context for the actual focused research you will do later. It is also a time to build a list of distinctive, broad, narrow, and related search terms.

Discovery happens again when you are ready to focus in on your research question and begin your own literature review. There are two crucial elements to discovering the literature for your review with the least amount of stress as possible: the places you look and the words you use in your search.

The places you look depend on:

• The stage you are in your research
• The disciplines represented in your research question
• The importance of currency in your research topic

Review the information and publication cycles discussed in Chapter 2 to put those sources of this information in context.

The words you use will help you locate existing literature on your topic, as well as topics that may be closely related to yours. There are two categories for these words:

• Keywords – the natural language terms we think of when we discuss and read about a topic
• Subject terms – the assigned vocabulary for a catalog or database

The words you use during both the initial and next stage of discovery should be recorded in some way throughout the literature search process. Additional terms will come to light as you read and as your question becomes more specific. You will want to keep track of those words and terms, as they will be useful in repeating your searches in additional databases, catalogs, and other repositories. Later in this chapter, we will discuss how putting the two elements (the places we look and the words we use) together can be enhanced by the use of Boolean operators and discipline-specific thesauri.

Discovery is an iterative process. There is not a straight, bright line from beginning to end. You will go back into the literature throughout the writing of your literature review as you uncover gaps in the evidence and as additional questions arise.

4.2 Finding sources: Places to look

Let’s take some time to look at where the information sources you need for your literature review are located, indexed, and stored. At this stage, you have a general idea of your research area and have done some background searching to learn the scope and the context of your topic. You have begun collecting keywords to use in your later searching. Now, as you focus in on your literature review topic, you will take your searches to the databases and other repositories to see what the other researchers and scholars are saying about the topic.

The following resources are ordered from the more general and established information to the more recent and specific. Although it is possible to find some of these resources by searching the open web, using a search engine like Google or Google Scholar, this is not the most efficient or effective way to search for and discover research material. As a result, most of the resources described in this section are found from within academic library catalogs and databases, rather than internet search engines.

4.2.1 Finding books and ebooks

4.2.1.1 Books

Look to books for broad and general information that is useful for background research. Books are “essential guides to understanding theory and for helping you to validate the need for your study, confirm your choice of literature, and certify (or contradict) its findings.” (Fink, 4th ed., 2014, p. 77). In this section, we will consider print and electronic books as well as print and electronic encyclopedias.

Most academic libraries use the Library of Congress classification system to organize their books and other resources. The Library of Congress classification system divides a library’s collection into 21 classes or categories. A specific letter of the alphabet is assigned to each class. More detailed divisions are accomplished with two and three letter combinations. Book shelves in most academic libraries are marked with a Library of Congress letter-number combination to correspond to the Library of Congress letter-number combination on the spines of library materials. This is often referred to as a call number and it is noted in the catalog record of every physical item on the library shelves. (Bennard et al, 2014a)

The Library of Congress (LC) classification for Education (General) is L7-991, with LA, LB, LC, LD, LE, LG, LH, LJ, and LT subclasses. For example,

LB3012.2.L36 1995
Beyond the Schoolhouse Gate: Free Speech and the Inculcation of Values

In Nursing, the LC subject range is RT1-120. A book with this LC call number might look like: R121.S8 1990 Stedman’s Medical Dictionary. Areas related to nursing that are outside that range include:

R121 Medical dictionaries

R726.8 Hospice care

R858-859.7 Medical informatics

RB37 Diagnostic and laboratory tests

RB115 Nomenclature (procedural coding – CPT, ICD9)

RC69-71 Diagnosis

RC86.7 Emergency medicine

RC266 Oncology nursing

RC952-954.6 Geriatrics

RD93-98 Wound care

RD753 Orthopedic nursing

RG951 Maternal child nursing / Obstetrical nursing

RJ245 Pediatric nursing

RM216 Nutrition and diet therapy

RM301.12 Drug guides

In most libraries, there is a collection of reference material kept in a specific section. These books, consisting of encyclopedias, dictionaries, thesauri, handbooks, atlases, and other material contain useful background or overview information about topics. Ask the librarian for help in finding an appropriate reference book. Although reference material can only be used in the library, other print books will likely be in what’s called the “circulating collection,” meaning they are available to check out.

4.2.1.2 Ebooks

The library also provides access to electronic reference material. Some are subject specific and others are general reference sources. Although each resource will have a different “look” just as different print encyclopedias and dictionaries look different, each should have a search box. Most will have a table of contents for navigation within the work. Content includes pages of text in books and encyclopedias and occasionally, videos. In all cases you will be able to collect background information and search terms to use later.

North American academic libraries buy or subscribe to individual ebook titles as well as collections of ebooks. Ebooks appear on various publisher and platforms, such as Springer, Cambridge, ebrary (ProQuest), EBSCO, and Safari to name a few. Although access to these ebooks varies by platform, you can find the ebook titles your library has access to through the library catalog. You can generally read the entire book online, and you can often download single chapters or a limited number of pages. You may be able to download an entire ebook without restrictions, or you may have to ‘check it out’ for a limited period of time. Some downloads will be in PDF format, others use another type of free ebook viewing software, like ePUB. Unlike public library ebook collections, most academic library ebooks are not be downloadable to ereader devices, such as Amazon’s Kindle

4.2.1.3 The Library Catalog

In general, everything owned or licensed by a library is indexed in “the library catalog”. Although most library catalogs are now sophisticated electronic products called ‘integrated library systems’, they began as wooden card filing cabinets where researchers could look for books by author, title, or subject.

While the look and feel of current integrated library systems vary between libraries, they operate in similar ways. Most library catalogs are quickly found from a library’s home page or website. The library catalog is the quickest way to find books and ebooks on your topic.

Here are some general tips for locating books in a library catalog:

• Use the search box generally found on a library’s home page to start a search.

• Type a book title, author name, or subject keywords into the search box.
• You will be directed to a results page.
• If you click on a book title or see an option to see more details about the book, you can look at its full bibliographic record, which provides more information about the book, as well as where to find the book. Pay particular attention to subjects associated with the item, adding relevant and appropriate terms to your list of search terms for future use.

• If you want to have more control over search results, you can try an “Advanced Search” within the library catalog
  • Look for an “Advanced Search” option near the basic or single search box
  • The options within the advanced catalog search window allow you to limit searches by:
    • Publication Year
    • Subject
    • Call number
    • And more…
• There is generally a “Format” list on the advanced search page screen. This list will give you options for limiting format to Print Books or Ebooks.
• You can limit searches to a specific library or libraries to narrow by location or ‘search everything’ to broaden your search.

OCLC WorldCat (https://www.worldcat.org/) is the world’s largest network of library content and it provides another way to search for books and ebooks. For students who do not have immediate access to an academic library catalog, WorldCat is a way to search many library catalogs at once for an item and then locate a library near you that may own or subscribe to it. Whether you will be able check the item out, request it, place an interlibrary loan request for it, or have it shipped will depend on local library policy. Note that like your own library catalog, WorldCat has a single search box, an Advanced search feature, and a way to limit by format and location.

4.2.2 Finding scholarly articles

While books and ebooks provide good background information on your topic, the main body of the literature in your research area will be found in academic journals. Scholarly journals are the main forum for research publication. Unlike books and professional magazines that may comment or summarize research findings, articles in scholarly journals are written by a researcher or research team. These authors will report in detail original study findings, and will include the data used. Articles in academic journals also go through a screening or peer-review process before publication,implying a higher level of quality and reliability. For the most current, authoritative information on a topic, scholars and researchers look to the published, scholarly literature. That said,

Journals, and the articles they contain, are often quite expensive. Libraries spend a large part of their collection budget subscribing to journals in both print and online formats. You may have noticed that a Google Scholar search will provide the citation to a journal article but will not link to the full text. This happens because Google does not subscribe to journals. It only searches and retrieves freely available web content. However, libraries do subscribe to journals and have entered into agreements to share their journal and book collections with other libraries. If you are affiliated with a library as a student, staff, or faculty member, you have access to many other libraries’ resources, through a service called interlibrary loan. Do not pay the large sums required to purchase access to articles unless you do not have another way to obtain the material, and you are unable to find a substitute resource that provides the information you need. (Bennard et al, 2014a)

4.2.2.1 Databases

A database is an electronic system for organizing information. Journal databases are where the scholarly articles are organized and indexed for searching. Anyone with an internet connection has free access to public databases such as PubMed and ERIC. Students can also search in library-subscribed general information databases (such as EBSCO’s Academic Search Premier) or a specialized or subject specific database (for example, a ProQuest version of CINAHL for Nursing or ERIC for Education).

Library databases store and display different types of information sets than a library catalog or Google Scholar. There are different types of databases that include:

• Indexes– with citations only
• Abstract databases – with citations and abstracts only
• Full text databases – with citations and the full text of articles, reports, and other materials

Library databases are often connected to each other by means of a “link resolver”, allowing different databases to “talk to each other.” For example, if you are searching an index database and discover an article you want to read in its entirety, you can click on a link resolver that takes you to another database where the full-text of the article is held. If the full-text is not available, an automated form to request the item from another library may be an option.

Why search a database instead of Google Scholar or your library catalog? Both can lead you to good articles BUT:

• The content is wide-ranging but not comprehensive or as current as a database that may be updated daily.
• Google Scholar doesn’t disclose its criteria for what makes the results “scholarly’ and search results often vary in quality and availability.
• Neither gives you as much control over your search as you get in a database.

4.2.2.2 Citation searches

Another way to find additional books and articles on your topic is to mine the reference lists of books and articles you already found. By tracing literature cited in published titles, you not only add to your understanding of the scholarly conversation about your research topic but also enrich your own literature search.

A citation is a reference to an item that gives enough information for you to identify it and find it again if necessary. You can use the citations in the material you found to lead you to other resources. Generally, citations include four elements:

• Author
• Title
• Source
• Date

For example,

For a good summary of how to read a citation for a book, book chapter, and journal article in both APA and MLA format, see this explanation  at: https://www.slideshare.net/opensunytextbooks/gathering-components-of-a-citation

4.2.3 Finding conference papers

Conference papers are often overlooked because they can be difficult to locate in full-text. Sometimes the papers from an annual proceeding are treated like an individual book, or a single special issue of a journal. Sometimes the papers from a conference are not published and must be requested from the original author. Despite publication inconsistency, conference papers may be the first place a scholar presents important findings and, as such, are relevant to your own research. Places to look for conference papers:

4.2.3.1 WorldCat

• use keywords from the conference name (NOT the article title)
• it often helps to leave out terms like: conference, proceedings, transactions, congresses, symposia/symposium, exposition, workshop or meeting
• include the year of the conference
• include the city in which the conference took place

4.2.3.2 Google Scholar

• Search by keyword and add the word ‘conference’ and the year to your search, for example: ‘conference education 2008′

4.2.3.3 Databases

• For Education: ERIC, limit to ‘Collected Works–Proceedings’ or ‘Speeches/Meeting papers’
• For Nursing: CINAHL, limit to proceedings in the “Publication Type” box
• For Education: Education Full Text, limit to ‘proceeding’ in the “Document Type” box
• PsychInfo: limit to ‘Conference Proceedings’ in the “Record Type” Box
• Web of Science: limit to ‘conference’

4.2.3.4 Professional Societies & Other Sponsoring Organizations

Check the web sites of the organizations that sponsor conferences. Listings of conference proceedings are often under a “Publications” or “Meetings” tab/link. The National Library of Medicine maintains a conference proceedings subject guide for health-related national and international conferences. Though many papers/proceedings are not available for free, the organization web site will often contain citations of proceedings that you can request through interlibrary loan.

4.2.4 Finding dissertations

In addition to journal articles, original research is also published in books, reports, conference proceedings, theses and dissertations. Both theses and dissertations are very detailed and comprehensive accounts of research work. Dissertations and theses are a primary source of original research and include “referencing, both in text and in the reference list, so that, in principle, any reference to the literature may be easily traced and followed up.” (Wallace & Wray, p. 187). Citation searching of the reference list or bibliography in a dissertation is another method for discovering the relevant literature for your own research area. Like conference papers, they are more difficult to locate and retrieve than books and articles. Some may be available electronically in full-text at no cost. Others may only be available to the affiliates of the university or college where a degree was granted. Others are behind paywalls and can only be accessed after purchasing. Both CINAHL and ERIC index dissertations. Individual universities and institutional repositories often list dissertations held locally. Other places to look for theses and dissertations include:

Dissertations Express – search for dissertations from around the world. Search by subject or keyword, results include author, title, date, and where the degree was granted. Some are available in full-text at no cost, however most requirement payment.

EThOS – the national thesis service for the United Kingdom, managed by the British Library. It is a national aggregated record of all doctoral theses awarded by UK Higher Education institutions, providing free access to the full text of many theses for use by all researchers to further their own study.

Theses Canada – a collaborative program between Library and Archives Canada (LAC) and nearly 70 accredited Canadian universities. The collection contains both microfiche and electronic theses and dissertations that are for personal or academic research purposes.

4.3 Advanced searching

Now that you have an idea of some of the places to look for information on your research topic and the form that information takes (books, ebooks, journals, conference papers, and dissertations), it’s time to consider not only how to use the specialized resources for your discipline but how to get the most out of those resources. To do a graduate-level literature review and find everything published on your topic, advanced search and retrieval skills are needed.

4.3.1 Search Operators

Literature review research often necessitates the use of Boolean operators to combine keywords. The operators – AND, OR, and NOT — are powerful tools for searching in a database or search engine. By using a combination of terms and one or more Boolean operator, you can focus your search and narrow your search results to a more specific area than a basic keyword search allows.

Boolean operators – allow you to combine your search terms using the keywords AND, OR and NOT. Look at the diagrams in Figure 4.6 to see how these terms will affect your results.

Truncation – If you use truncation (or wildcards), your search results will contain documents including variations of that term.

For example: light* will retrieve, of course, light, but also terms like: lighting, lightning, lighters and lights. Note that the truncation symbol varies depending on where you search. The most common truncation symbols are the asterisk (*) and question mark (?).

Phrase searching – Phrase searching is used to make sure your search retrieves a specific concept. For example “durable wood products” will retrieve more relevant documents than the same terms without quotation marks.

For a description of these more advanced search features, watch this short video tutorial on effective search strategies. (Clark, 2016).

4.3.2 Finding sources in your discipline or topic area

It’s time to put these tips and your search skills to use. This is the point, if you have not done so already, to talk to a librarian. The librarian will direct you to the resources you need, including research databases to which the library subscribes, for your discipline or subject area. Literature reviews rely heavily on data from online databases, such as CINAHL for Nursing and ERIC for Education. Unfortunately, the costs to subscribe to vendor-provided products is high. Students affiliated with large university libraries that can afford to subscribe to these products will have access to many databases, while those who do not have fewer options.

Students who do not have access to subscription databases such as CINAHL or ERIC through Ebsco and ProQuest should use PubMed for Nursing at https://www.ncbi.nlm.nih.gov/pubmed/ and the public version of ERIC at https://eric.ed.gov/ for literature review research.

Although a librarian is the best resource for learning how to use a specific tool, an online tutorial on how to search PubMed may be useful and informative for those who do not have access to a librarian or a subscription database: Likewise, this document, titled “How does the ERIC search work,” provided by the Institute of Education Sciences provides some helpful tips for searching the public version ERIC.

4.3.3 Specialized vocabulary

One major source of search terms in a database is a specialized dictionary, or thesaurus, used to index journal articles. Thesauri provide a consistent and standardized way to retrieve information, especially when different terms are used for the same concept. According to Fink (2014), “evidence exists that using thesaurus terms produces more of the available citations than does reliance on key words…Using the appropriate subject heading will enable the reviewer to find all citations regardless of how the author uses the term.” (p. 24).

In Education and Nursing, thesauri are available. In subscription databases, as well as in PubMed and the public version of ERIC, look for the thesaurus to guide you to appropriate and relevant subject terms.

4.3.4 Citation Searching

Citation searching works best when you already have a relevant work that is on topic. From the document you identified as useful for your own literature review, you can either search citations forward or backward to gather additional resources. Cited reference searching and reference or bibliography mining are advanced search techniques that may also help generate new ideas as well as additional keywords and subject areas.

For cited reference searching, use Google Scholar or library databases such as Web of Science or Scopus. These tools trace citations forward to link to newly published books, journal articles, book chapters, and reports that were written after the document you found. Through cited reference searching, you may also locate works that have been cited numerous times, indicating what may be a seminal work in your field.

With citation mining, you will look at the references or works cited list in the resource you located to identify other relevant works. In this type of search, you will be tracing citations backward to find significant books, journal articles, book chapters, and reports that were written before the document you found. For a brief discussion about citation searching, check out this article by Hammond & Brown (2008).

References

Image attributions

5.1 Overview of evaluation of sources

Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops. (Association of College & Research Libraries, 2016).

You developed a viable research question, compiled a list of subject headings and keywords and spent a great deal of time searching the literature of your discipline or topic for sources. It’s now time to evaluate all of the information you found. Not only do you want to be sure of the source and the quality of the information, but you also want to determine whether each item is appropriate fit for your own review. This is also the point at which you make sure that you have searched out publications for all areas of your research question and go back into the literature for another search, if necessary.

In general, when we discuss evaluation of sources we are talking about looking at quality, accuracy, relevance, bias, reputation, currency, and credibility factors in a specific work, whether it’s a book, ebook, article, website, or blog posting. Before you include a source in your literature review, you should clearly understand what it is and why you are including it. According to Bennard et al., (2014), “Using inaccurate, irrelevant, or poorly researched sources can affect the quality of your own work.” (para. 4).

When evaluating a work for inclusion in, or exclusion from, your literature review, ask yourself a series of questions about each source.

5.1.1 Evaluating books

For primary and secondary sources you located in your search, use the ASAP mnemonic to evaluate inclusion in your literature review:

5.1.1.1 Age

Is it outdated? The answer to this question depends on your topic. If you are comparing historical classroom management techniques, something from 1965 might be appropriate. In Nursing, unless you are doing a historical comparison, a textbook from 5 years ago might be too dated for your needs.

A General Rule of Thumb:

5 years, maximum: medicine, health, education, technology, science
10-20 years: history, literature, art

5.1.1.2 Sources

Check reference or bibliography sources as well as those listed in footnotes or endnotes. Skim the list to see what kinds of sources the author used. When were the sources published? If the author is primarily citing works from 10 or 15 years ago, the book may not be what you need.

5.1.1.3 Author

Does the author have the credentials to write on the topic? Does the author have an academic degree or research grant funding? What else has the author published on the topic?

5.1.1.4 Publisher

Look for academic presses, including university presses. Books published under popular press imprints (such as Random House or Macmillan, in the U.S.) will not present scholarly research in the same way as Sage, Oxford, Harvard, or the University of Washington Press.

Other questions to ask about the book you may want to include in your literature review:

• What is the book’s purpose? Why was it written? Who is the intended audience?
• What is the conclusion or argument? How well is the main argument or conclusion supported?
• Is it relevant to your research? How is it related to your research question?
• Do you see any evidence of bias or unsubstantiated data?

5.1.2 Evaluating websites

In your research, it is likely you will discover information on the web that you will want to include in your literature review. For example, if your review is related to the current policy issues in public education in the United States, a potentially relevant information source may be a document located on the National Center for Education Statistics (NCES) website titled The Condition of Education 2017. Likewise, for nursing, an article titled Discussing Vaccination with Concerned Patients: An Evidence-Based Resource for Healthcare Providers is available through the nursingcenter.com website. How do you evaluate these resources, and others like them?

Use the RADAR mnemonic (Mandalios, 2013) to evaluate internet sources:

5.1.2.1 Relevance

How did you find the website and how is it relevant to your topic?

• Was it recommended by a reliable source?
• Was it cited in a scholarly source, such as a peer-reviewed journal?
• Was it linked from a reputable site?

5.1.2.2 Authority

Look for the About page to find information about the purpose of the website . You may make a determination of its credibility based on what you find there. Does the page exhibit a particular point of view or bias? For example, a heart association or charter school may be promoting a particular perspective – how might that impact the objectivity of the information located on their site? Is there advertising or is there a product information attached to the content?

5.1.2.3 Date

• When was the page created?
• Is it kept up to date?
• Are the links current and functional?

5.1.2.4 Appearance

• Does the information presented appear to be factual?
• Is the language formal or academic?
• How does it compare to other information you have read on the topic?
• Are references or links to cited material included?

5.1.2.5 Reason

What is the web address or URL? This can give you a clue about the purpose of the website, which may be to debate, advocate, advertise or sell, campaign, or present information. Here are some common domains and their origins:

• .org – An advocacy website for an organization
• .com – A private or commercial site
• .net – A network organization or Internet provider/no longer frequently used
• .edu – The site of a higher educational institution
• .gov – A federal government site
• .wa.us – A state government site which may include public schools and community colleges
• .uk, .ca, .jm – A country site

Mike Caulfield (2017), the author of Web Literacy for Student Fact-Checkers, recommends a few simple strategies to evaluate a website (as well as social media):

• Check for previous work: Look around to see if someone else has already provided a synthesis of the research described.
• Go upstream to the source: Go “upstream” to the source of the claim. Most web content is not original. Get to the original source to understand the credibility and reliability of the information.
• Read laterally: Read laterally. Once you get to the source of a claim, read what other people say about the source (publication, author, etc.). The truth is in the network.

5.1.3 Evaluating journal articles

It is likely that most of the resources you locate for your review will be from the scholarly literature of your discipline or in your topic area. As we have already seen, peer-reviewed articles are written by and for experts in a field. They generally describe formal research studies or experiments with the purpose of providing insight on a topic. You may have located these articles through Google, Google Scholar, a subscription or open access database, or citation searching. You now may want to know how to evaluate the usefulness for your research. As with the other resources, you are again looking for authority, accuracy, reliability, relevance, currency, and scope. Looking at each article as a separate and unique artifact, consider these elements in your evaluation:

5.1.3.1 Credibility/Authority

ASK: Who is the author? Is this person considered an expert in their field?

• Search the author’s name in a general web search engine like Google.
• What are the researcher’s academic credentials?
• What else has this author written? Search by author in the databases and see how much they have published on any given subject.
• How often or frequently has this article been cited by other scholars?

Citation analysis is the study of the impact and assumed quality of an article, an author, or an institution, based on the number of times works and/or authors have been cited by others. Google Scholar is a good way to get at this information.

5.1.3.2 Accuracy

Check the facts. ASK:

• Can statistics be verified through other sources?
• Does this information seem to fit with what you have read in other sources?

5.1.3.3 Reliability/Objectivity

ASK: Is there an obvious bias? That doesn’t mean that you can’t use the information, it just means you need to take the bias into account.

• Is a particular point of view or bias immediately obvious, or does it seem objective at first glance?
• What point of view does the author represent? Are they clear about their point of view?
• Is the article an editorial that is trying to argue a position?
• Is the article in a publication with a particular editorial position?

5.1.3.4 Relevance

ASK: The hard questions:

• Is the information relevant to your topic/thesis?
• How does the article fit into the scope of the literature on this topic?
• Who is the intended audience for this source?
  • Is the material too technical or too clinical?
  • Is it too elementary or basic?
• Does the information support your thesis or help you answer your question, or is it a challenge to make some kind of connection?
• Does the information present an opposite point of view so you can show that you have addressed all sides of the argument in your paper?

5.1.3.5 Currency

ASK:

• When was the source published?
• How important is current information to your topic, discipline, or paper type?
• Does older material add to the history of the research? Or do you need something more current to support your thesis?

5.1.3.6 Scope and Purpose

To determine and evaluate in this category, ASK:

• Is it a general work that provides an overview of the topic or is it specifically focused on only one aspect of your topic?
• Does the breadth of the work match your expectations?
• Is the article meant to inform, explain, persuade or sell something. Be aware of the purpose as you read the content and take that into consideration when deciding whether to use it or not.

For Nursing and other medical articles ASK:

• What are the research methods used in the article?
• Where does the method fall in the evidence pyramid? Systematic reviews and meta-analyses are the most credible, with articles that are opinions the least credible.

• Meta Analysis: A systematic review that uses quantitative methods to summarize the results.
• Systematic Review: An article in which the authors have systematically searched for, appraised, and summarized all of the medical literature on a specific topic.
• Randomized Controlled Trials (RCTs): RCTs include a randomized group of patients in an experimental group, as well as a control group. These groups are monitored for the variables/outcomes of interest.
• Cohort Study: Research identifies two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and follows these cohorts for a specified duration of time, in order to measure the outcome of interest.
• Case Study: Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looks to see if they had the exposure of interest.
• Animal Research / Lab Studies: Information creation begins at the bottom of the pyramid: This is where ideas and laboratory research take place. Ideas turn into therapies and diagnostic tools, which are then tested with lab models and animals.
• Background Information / Expert Opinion: Handbooks, encyclopedias, and textbooks often provide a good foundation or introduction and often include generalized information about a condition. While background information presents a convenient summary, it typically takes about three years for this type of literature to be published.

5.1.4 Evaluating Social Media

Although social media (for example, Twitter or Facebook) is generally treated as an object under study rather than a source of information on a topic, the prevalence of social media as communication and sharing platforms must be acknowledged. It’s important to be skeptical of these sources, especially for inclusion in a literature review. However, as with any other web resource,you can evaluate a social media posting for authenticity by asking the following questions:

• Location of the source – Is the author in the place they are tweeting or posting about?
• Network – Who is in the author’s network and who follows the account?
• Content – Can the information be corroborated from other sources?
• Contextual updates – Does the author usually post or tweet on this topic? If so, what did past or updated posts say? Do they fill in more details?
• Reliability – does the author cite sources and are those sources reliable? (Sheridan Libraries, 2017)

5.2 In Summary

Another way to think about evaluation of sources is to ask the 5W questions:

• What type of document is it?
• Who created it?
• Why was the material published?
• When was it published?
• Where was the resource published?
• How was the information gathered and presented? (Radom, 2017)

Locating sources for your literature review by using discovery layers, library catalogs, databases, search engines, and other search platforms may take a great deal of time and effort. Does everything you found and retrieved have value or worth to you as you write your own literature review? If the resource has not met the criteria above and you can’t justify its place in your literature review, it doesn’t deserve to be mentioned in your work. Include high-quality materials that are current, accurate, credible, and most importantly relevant to your research question, hypothesis, or topic.

References

Image attribution

6.1 Overview of documenting sources

A graduate-level literature review is a significant undertaking and will require some decisions about information organization, record-keeping, and notes management. Make these decisions before you begin your intensive review of the literature. Some of the decisions you will need to make include things like document-naming conventions, choosing a citation management tool that fits your needs, and setting up journal alerts.

Once you have identified and located materials for your literature review, you will organize, analyze, and synthesize them as the next step in literature review process. Here are some general guidelines for how you treat the articles at this stage:

1. Skim the articles as you gather them to get an idea of the general purpose and content. Focus on the abstract, introduction, first few paragraphs, and the conclusion.
2. Record notes and impressions on the article directly in the citation management tool you choose. Record specific aspects or significant keywords of the article that are relevant to your review. General remarks, such as ‘good source’ or ‘interesting idea,’ won’t help you later on.
3. Pay special attention to major trends or patterns, possible gaps in the literature, and relationships among studies, especially noting or highlighting landmark studies that led to subsequent ones in the same area.
4. Group the articles into categories or folders, such as topics and subtopics. Also group articles that you have placed within these categories chronologically. You can print out each article and organize the paper copies into categories or you take advantage of technology by using citation management software to store and organize your articles.

Begin to group sources into broad categories and then organize chronologically or alphabetically by author’s last name. Broad general categories might include:

• Themes or Concepts
• Theories
• Policies
• Programs
• Populations
• Methodologies
• Questions for further research

Other broad organization schemes might relate to the PICO or SPICE models mentioned in Chapter 3. We will discuss organization and synthesis in more detail in Chapter 7.

5. Develop a standardized naming convention for folders and files. Names should be kept as short as possible whilst also being meaningful, concise, and standardized. For example, PolicyCttee2017 or GuidelinesRegulationsHarrison is more useful than LitReviewArticle1.

Other useful file naming conventions can be found The University of Edinburgh Records Management Office (2017). Examples include:

• Avoid unnecessary repetition and redundancy in file names and file paths.
• Use capital letters to delimit words, not spaces or underscores
• When including a personal name in a file name give the family name first followed by the initials.
• Avoid using common words such as ‘draft’ or ‘letter’ at the start of file names, unless doing so will make it easier to retrieve the record.
• Order the elements in a file name in the most appropriate way to retrieve the record.
• Avoid using non-alphanumeric characters in file names.

Take the time to think about your organizational system before you begin researching and compiling sources in earnest. “Organizing now will save much time and heartache later.” (Machi & McEvoy, 2012, p. 31).

6.2 Citation Management Tools

One of your first decisions – after selecting your topic – will be to determine which citation manager will work the best for you. Citation managers are software packages, such as EndNote or Zotero, used to create personalized databases of citations and notes. Citation management tools help users:

• import citations from databases, websites, and library catalogs
• create bibliographies
• format citations in a variety of styles such as APA, MLA, Chicago, and more
• manage, categorize, and organize citations and documents
• attach PDFs, images, and notes to citations in your collection.

While most current citation managers are generally similar, individual workflow may determine which tool to use. For example, if you will be working from multiple computers and locations, a web-based tool such as RefWorks and Mendeley will work better for you than a client-based or centrally hosted website. Other needs to consider when evaluating different citation managers:

• I need to work offline.
• I’ll be doing a lot of my research on freely available websites and need to be able to save copies of webpages.
• I’m working on a group project and need to share my references with others.
• I’m not so comfortable with technology and may potentially need a lot of help with my tool.
• I will be working on a mobile device.

6.2.1 Tips for choosing the right tool

There are many tools to choose from and you want to experiment with a few as well as discuss with professional colleagues, fellow students, or faculty before making a final decision on which to use. Choosing a tool ultimately depends on your personal workflow preferences and your needs.

General tips for choosing the right citation management tool:

• Consult Wikipedia’s detailed and updated comparison chart of citation management tools to determine if any tool is clearly the best fit for you.
• Talk to people in your department. Do individuals in your discipline tend to use one tool more often than another? Does your department or university already provide access to a specific tool?
• Talk to your subject librarian; s/he can recommend a tool based on your needs.
• Critically assess your technology skills and interests. Although all the tools advertise ease of use, there is a learning curve. Take a look at the free tutorials, help documents, and instruction manuals and rate your level of understanding and confidence.

Choose your citation management tool carefully. Try some out. Talk to colleagues. Once you’ve chosen a tool and started using it, changing to a different tool is problematic on several levels. If you save citations in two different products, it can be difficult to keep track of citations. Learning a new product or migrating information from one citation tool to another when you are in the middle of a project can also be difficult, time-consuming, and stressful. Choose carefully, but do choose and then stick with it.

6.2.2 Alerts

Alerts are an excellent way to keep up with the literature of your discipline. Alerts allow you to stay up to date with current research relevant to your topic. Once an alert is set up, you will automatically receive an email when an author’s publication, keywords, affiliations, or other search criteria appear in a database. You will be able to connect to the citation, download the citation and full text (when available) from the alert, and (if relevant) save to your citation manager. Alerts are a way to save time AND stay up-to-date in your topic area.

6.2.2.1 Why use alerts?

• Do you ever feel overwhelmed by the amount of time it takes to stay aware of the latest research and trends in your discipline?
• Do you have so many articles and journals in your “to read” pile that they end up being irrelevant by the time you get to them?
• Do you have a due date for your literature review, but can’t find time to check back for the latest updates on the topic?

If you answered yes to any of these questions, note that a number of database aggregators like ProQuest and EBSCO, as well as individual databases, such as ERIC and CINAHL, offer free alert services informing you of new journal issues, recently published articles related to your interests, and more. Most databases and journals use e-mail alerts to inform users of new content. Many researchers set up alerts through Google Scholar. For tips on how to set up alerts in Google Scholar, see the help page at: https://scholar.google.com/intl/en/scholar/help.html#alerts

6.2.2.2 Types of alerts

1. Table of Contents (TOC) Alerts – These alerts inform users about new journal issues. Depending on the database and your preferred method of delivery, you will receive a table of contents for the issue or links to the full-text articles. Most TOC alerts are delivered via email, but they can also be subscribed to via RSS. A directory of thousands of current and scholarly TOCs is browsable at http://www.journaltocs.hw.ac.uk/. For a short 2 minute tutorial on how to set up journal alerts through PubMed, see https://www.nlm.nih.gov/bsd/viewlet/myncbi/jourup/index.html
2. Saved Searches – A saved search alert will notify you when the database identifies new articles related to a customized search. You can specify how often you would like to receive updates (weekly, monthly, etc.).
3. Citation Alerts – These alerts will inform you when a specified article is cited in a new publication.

Within your citation manager, you can set up custom folders to not only store new articles but also to share both alerts and articles with colleagues or fellow students researching similar topics.

6.3 Bibliographic citation format

Once you begin gathering sources for your literature review, you will need to organize and document them. Citations document the source of an idea, statement, or study. A uniform citation style helps both the reader and the writer. A standardized editorial style removes the distraction and confusion of puzzling over the correct punctuation for every reference or the proper formatting for numbers and other data in text. Those elements are codified in the rules of the format style, allowing the reader to focus energy on the substance of the research, rather than how the paper is constructed.

An author writing for publication must follow the rules established by the publisher to avoid inconsistencies. Without established rules of style, each manuscript might use different spellings, notations, and citations, which would confuse and distract readers. The need for a consistent style becomes more apparent and more visible when complex material is presented, such as tables or statistics. Without standardized rules for presentation of data, the reader would spend too much time and energy looking for meaning among the structure.

Likewise, a systematic and standardized bibliographic citation format helps the writer of the literature review keep track of references as they accumulate and find them more efficiently later in the process. “You will be rewarded for your hard work, if not in heaven, then certainly when you come to write your report. You will be able to locate information easily, to regroup and reclassify evidence and to produce referenced quotations to support your arguments.” (Bell, 2005, p. 74).

There are numerous different bibliographic citation format styles. APA (American Psychological Association), MLA (Modern Language Association), Chicago, Turabian, ACS (American Chemical Society), AMA (American Medical Association) , and IEEE (Institute of Electrical and Electronic Engineers) are some of the more common formats in use, but there are many more. The different styles, and different versions within each style, are a source of stress for generations of students and researchers in all disciplines, including those in the health sciences and education. In the social sciences, APA style is frequently used as the default citation style. Your department or discipline may require another format and, if so, that is the one you should accustom yourself with using to document your sources.

As there are over a dozen different citation styles and different disciplines prefer different styles, always check to see if your instructor requires a particular style. Also because the rules for citation styles can change and can be extensive, it is best to refer to the official handbooks/style guides when you can. (Teaching & Learning, 2015, p. 6).

Whatever citation style and format you decide to use, now is the time to make that decision. Consistently documenting your sources as you read is another way to plan and organize information as you go along, rather than at the end or in the middle.

In addition to print and online manuals detailing the specifics of each citation style, there are numerous websites and other resources that provide document citation formatting help. The Online Writing Lab (OWL) at Purdue University, for example, can answer most questions about APA, MLA, and Chicago style. University writing labs and subject specialist librarians may also help with correctly documenting sources and formatting style.

A useful open resource for graduate students in the social sciences is Professional Writing in the Health Disciplines by Sandra Collins (2016). In addition to discussing how to structure a graduate-level paper, a chapter on APA citation and reference formatting provides extensive detail on how to document sources. Additionally, Choosing & Using Sources: A Guide to Academic Research (Teaching & Learning, 2015) provides examples and advice for documenting sources using APA style formatting.

References

Image attributions

7.1 Overview of synthesizing

7.1.1 Putting the Pieces Together

Combining separate elements into a whole is the dictionary definition of synthesis.  It is a way to make connections among and between numerous and varied source materials.  A literature review is not an annotated bibliography, organized by title, author, or date of publication.  Rather, it is grouped by topic to create a whole view of the literature relevant to your research question.

Your synthesis must demonstrate a critical analysis of the papers you collected as well as your ability to integrate the results of your analysis into your own literature review.  Each paper collected should be critically evaluated and weighed for “adequacy, appropriateness, and thoroughness” (Garrard, 2017) before inclusion in your own review.  Papers that do not meet this criteria likely should not be included in your literature review.

Begin the synthesis process by creating a grid, table, or an outline where you will summarize, using common themes you have identified and the sources you have found. The summary grid or outline will help you compare and contrast the themes so you can see the relationships among them as well as areas where you may need to do more searching. Whichever method you choose, this type of organization will help you to both understand the information you find and structure the writing of your review.  Remember, although “the means of summarizing can vary, the key at this point is to make sure you understand what you’ve found and how it relates to your topic and research question” (Bennard et al., 2014).

As you read through the material you gather, look for common themes as they may provide the structure for your literature review.  And, remember, research is an iterative process: it is not unusual to go back and search information sources for more material.

At one extreme, if you are claiming, ‘There are no prior publications on this topic,’ it is more likely that you have not found them yet and may need to broaden your search.  At another extreme, writing a complete literature review can be difficult with a well-trod topic.  Do not cite it all; instead cite what is most relevant.  If that still leaves too much to include, be sure to reference influential sources…as well as high-quality work that clearly connects to the points you make. (Klingner, Scanlon, & Pressley, 2005).

7.2 Creating a summary table

Literature reviews can be organized sequentially or by topic, theme, method, results, theory, or argument.  It’s important to develop categories that are meaningful and relevant to your research question.  Take detailed notes on each article and use a consistent format for capturing all the information each article provides.  These notes and the summary table can be done manually, using note cards.  However, given the amount of information you will be recording, an electronic file created in a word processing or spreadsheet is more manageable. Examples of fields you may want to capture in your notes include:

• Authors’ names
• Article title
• Publication year
• Main purpose of the article
• Methodology or research design
• Participants
• Variables
• Measurement
• Results
• Conclusions

 Other fields that will be useful when you begin to synthesize the sum total of your research:

• Specific details of the article or research that are especially relevant to your study
• Key terms and definitions
• Statistics
• Strengths or weaknesses in research design
• Relationships to other studies
• Possible gaps in the research or literature (for example, many research articles conclude with the statement “more research is needed in this area”)
• Finally, note how closely each article relates to your topic.  You may want to rank these as high, medium, or low relevance.  For papers that you decide not to include, you may want to note your reasoning for exclusion, such as ‘small sample size’, ‘local case study,’ or ‘lacks evidence to support assertion.’

This short video demonstrates how a nursing researcher might create a summary table.

7.2.1 Creating a Summary Table

 Summary tables can be organized by author or by theme, for example:

For a summary table template, see http://blogs.monm.edu/writingatmc/files/2013/04/Synthesis-Matrix-Template.pdf

7.3 Creating a summary outline

An alternate way to organize your articles for synthesis it to create an outline. After you have collected the articles you intend to use (and have put aside the ones you won’t be using), it’s time to identify the conclusions that can be drawn from the articles as a group.

 Based on your review of the collected articles, group them by categories.  You may wish to further organize them by topic and then chronologically or alphabetically by author.  For each topic or subtopic you identified during your critical analysis of the paper, determine what those papers have in common.  Likewise, determine which ones in the group differ.  If there are contradictory findings, you may be able to identify methodological or theoretical differences that could account for the contradiction (for example, differences in population demographics).  Determine what general conclusions you can report about the topic or subtopic as the entire group of studies relate to it.  For example, you may have several studies that agree on outcome, such as ‘hands on learning is best for science in elementary school’ or that ‘continuing education is the best method for updating nursing certification.’ In that case, you may want to organize by methodology used in the studies rather than by outcome.

Organize your outline in a logical order and prepare to write the first draft of your literature review.  That order might be from broad to more specific, or it may be sequential or chronological, going from foundational literature to more current.  Remember, “an effective literature review need not denote the entire historical record, but rather establish the raison d’etre for the current study and in doing so cite that literature distinctly pertinent for theoretical, methodological, or empirical reasons.” (Milardo, 2015, p. 22).

As you organize the summarized documents into a logical structure, you are also appraising and synthesizing complex information from multiple sources.  Your literature review is the result of your research that synthesizes new and old information and creates new knowledge.

7.4 Additional resources:

Literature Reviews: Using a Matrix to Organize Research / Saint Mary’s University of Minnesota

Literature Review: Synthesizing Multiple Sources / Indiana University

Writing a Literature Review and Using a Synthesis Matrix / Florida International University

 Sample Literature Reviews Grid / Complied by Lindsay Roberts

References

Image attribution

8.1 Writing the literature review

You have discovered, retrieved, evaluated, synthesized, and organized the information you need for your literature review.  It’s time to turn that stack of articles and papers and notes into a literature review.  It’s time to start writing.

The graduate-level literature review is about both content and form.  In terms of content, keep in mind that your literature review is intended to:

• Set up a theoretical framework for your own research
• Show a clear understanding of the key concepts/ideas/studies/models related to your topic
• Demonstrate knowledge about the history of your research area and any related controversies
• Illustrate that you are able to evaluate and synthesize the work of others
• Clarify significant definitions and terminology
• Develop a space in your discipline for your research

In other words, literature reviews

…clearly describe the questions that are being asked. They also locate the research within the ongoing scholarly dialogue. This is done by summarizing current understandings and by discussing why what we already knows leads to the need for the present research. Literature reviews also define the primary concepts. While this information can appear in any order, these are the elements in all literature reviews. (Loseke, 2017, p.67)

Some questions to ask yourself when you begin to write your first draft include:

• How will my literature review be organized: Chronologically, thematically, conceptually, methodologically or a combination?
• What section headings will I be using?
• How do the various studies relate to each other?
• What contributions do they make to the field?
• What are the limitations of a study/where are the gaps in the research?
• And finally but most importantly, how does my own research fit into what has already been done?

Some questions to ask after the first draft:

• Is there a logical flow from section to section, paragraph to paragraph, sentence to sentence?
• Does the content proceed from topic to topic?
• Does your conclusion match your introduction?
• Were you consistent in documenting and using the correct citation style?

8.2 Main features

No matter how you decide to organize your literature review (chronologically, thematically, etc.), it follows a format you will immediately recognize: Introduction, Body, Conclusion.  We will look at each section individually.

8.2.1 Introduction

The introduction to the literature review contains a statement or statements about the overall topic of your dissertation or theses.  This might be an paragraph or section that lets your reader know what your literature review is going to address.  You will describe how the literature review will be organized (for example, what are the main points you are going to address and in what order will they appear?).  You may choose to briefly describe search criteria (keywords, databases, journals) in this section, or you may do it in different parts of the review.  It is suggested that this introductory section be no longer than two pages in length.  The purpose is to lead your reader further into the body of the literature review.

In the introduction, you will:

• Define or identify the general topic, issue, or area of concern thereby providing an appropriate context for the remainder of the review
• Point out overall trends in what has been previously published on the topic; or conflicts in theory, methodology, evidence, conclusions, or gaps in research and scholarship
• Establish your reason for reviewing this research (point of view); explain the criteria used to search the literature; the organization of the review (sequence); and – if necessary – why certain literature either is or is not included (scope)
• Demonstrate how your research either closes a gap in the literature, extends earlier work, or replicates an important study thereby contributing new knowledge to your discipline.

More tips for the Introduction:

1. Consider presenting a historical frame of reference
2. Point out a landmark or seminal study
3. Provide definitions for important terms
4. Describe how your literature review was conducted
5. Describe any inclusion or exclusion criteria used

8.2.2 Body

Some general tips for writing the body of your literature review:

• Start broad and then narrow to show how past research relates to your project.
• Make it clear to your reader where you’re going, follow a logical progression of ideas
• When appropriate, cite two or more sources for a single point but avoid long strings of references for a single point.
• Use quotes sparingly.
• Keep your own formal academic voice throughout and keep the review focused and objective, following a logical structure.
• Point out consistent findings AND emphasize stronger studies over weaker ones.  Point out important strengths and weaknesses of research studies OR contradictions and inconsistent findings.
• Implications and suggestions for further research, or where there are gaps in the current literature, should be specific.

8.2.3 Conclusion

Summarize your literature review, discuss implications, and create a space for future or further research needed in this area.  Like the introduction, this section should be around 3-5 pages in length.  How do you know when you’re done?  Can you answer these 11 questions:

1. Have you clearly defined your topic and audience?
2. Did you search and re-search the literature?
3. Took notes while reading?
4. Chosen the type of review you want to write?
5. Have you kept the review focused throughout?
6. Were you critical and consistent in your evaluation and synthesis?
7. Is the structure of your review logical?
8. Did you make use of feedback?
9. Were you able to stay relevant and objective throughout?
10. Did you maintain an objective voice?
11. Did you cite current and older studies? (Pautasso, 2013).

8.2.4 List of references

The reference list of publications used in your literature review serves two purposes.  First, it provides your reader with a means to evaluate the quality of your research.  Second, accurately and correctly citing all the sources used in your work protects you from possible accusations of plagiarism.  Using the words or ideas of others without referencing your source  is a very serious academic offense.

The reference list is a  reflection of  the thoroughness of your review.  It also allows others to retrieve the publications you cite.  Errors made in authors’ names, journal or article titles, page numbers and dates may present barriers to retrieval of articles and may prevent giving credit to authors for their work.  Each reference should be checked carefully for errors.  Every in text citation must have a listing in the references and every title in the reference list should connect to an in-text citation.

8.3 Tips for Structure

The literature reviews generally move from general to more specific, taking in all the elements mentioned previously.

Build your story by identifying areas of consensus and areas of divergence.  For example

• It seems there is agreement among researchers…
• Much debate exists on the issue of…

Possible structures:

Distant to close – the most distantly related to your work leading to the most closely related to your work.

Chronological – earliest related work to most recent related work.

Compare and contrast valid approaches, features, characteristics, theories – that is, one approach, then a 2nd approach, followed by a 3rd approach.

Finally, consider the use of summary paragraphs throughout the body of the review.   For example:

• In summary, the evidence presented demonstrates that…
• Rather, this literature supports the theory that…
• Consequently, the population studied may experience…
• However, alternative ideas and findings suggest…

8.4 An Example and A Checklist

An example of the possible structure for a literature review:

Introduction
Establish the importance of the topic
Number and type of people affected
Seriousness of the impact
Physical, psychological, economic, social aspects
Definitions of key terms
Literature review strategies
Description of the extent and nature of the literature
Overview of the organization of the rest of the review
Body of the review

Topic 1

Supporting evidence

Topic 2

Supporting evidence

Topic 3

Supporting evidence
Summary of the review
Discussion
Conclusions
Implications
Suggestions for future research
List of references

After you have written your first draft, use this checklist to review your progress:

1. Fill in the topic outline with brief notes.
2. Do not write a string of annotations.
3. Cite two or more sources for a single point, but avoid long strings of references for a single point.  Consider using e.g. when there are a large number of sources for a single point.
4. Use quotations sparingly.
5. Emphasize stronger studies over weaker ones.
6. Point out strengths and weaknesses of the research cited.
7. Point out consistent findings in a body of literature.
8. Point out contradictions or inconsistent findings as well.
9. Identify gaps.
10. Indicate when previous literature reviews are cited.
11. Implications and suggestions for future research should be specific, not just ‘more research is needed.’

8.5 In Summary

Like any effective argument, the literature review must have some kind of structure. For example, it might begin by describing a phenomenon in a general way along with several studies that demonstrate it, then describing two or more competing theories of the phenomenon, and finally presenting a hypothesis to test one or more of the theories. Or it might describe one phenomenon, then describe another phenomenon that seems inconsistent with the first one, then propose a theory that resolves the inconsistency, and finally present a hypothesis to test that theory. In applied research, it might describe a phenomenon or theory, then describe how that phenomenon or theory applies to some important real-world situation, and finally suggest a way to test whether it does, in fact, apply to that situation.

Looking at the literature review in this way emphasizes a few things. First, it is extremely important to start with an outline of the main points that you want to make, organized in the order that you want to make them. The basic structure of your argument then should be apparent from the outline itself. Second, it is important to emphasize the structure of your argument in your writing. One way to do this is to begin the literature review by summarizing your argument even before you begin to make it, “In this article, I will describe two apparently contradictory phenomena, present a new theory that has the potential to resolve the apparent contradiction, and finally present a novel hypothesis to test the theory.” Another way is to open each paragraph with a sentence that summarizes the main point of the paragraph and links it to the preceding points. These opening sentences provide the “transitions” that many beginning researchers have difficulty with. Instead of beginning a paragraph by launching into a description of a previous study, such as “Williams (2004) found that…,” it is better to start by indicating something about why you are describing this particular study. Here are some simple examples:

Another example of this phenomenon comes from the work of Williams (2004).

Williams (2004) offers one explanation of this phenomenon.

An alternative perspective has been provided by Williams (2004).

We used a method based on the one used by Williams (2004).

Finally, remember that your goal is to construct an argument for why your research question is interesting and worth addressing—not necessarily why your favorite answer to it is correct. In other words, your literature review must be balanced. If you want to emphasize the generality of a phenomenon, then of course you should discuss various studies that have demonstrated it. However, if there are other studies that have failed to demonstrate it, you should discuss them too. Or if you are proposing a new theory, then of course you should discuss findings that are consistent with that theory. However, if there are other findings that are inconsistent with it, again, you should discuss them too. It is acceptable to argue that the balance of the research supports the existence of a phenomenon or is consistent with a theory (and that is usually the best that researchers in psychology can hope for), but it is not acceptable to ignore contradictory evidence. Besides, a large part of what makes a research question interesting is uncertainty about its answer. (University of Minnesota, 2016).

8.6 Additional resources

Doing a literature review / University of Leicester

Get Lit: The Literature Review / Texas A&M Writing Centre

References

Image Attribution

When you began looking through this book, you may have already been an accomplished researcher and writer.  As a student, you may have had both research and writing experiences as an undergraduate that prepared you for your first graduate-level literature review.  For most graduate students, however, many of the concepts and skills needed to successfully complete this high-stakes document will be new.  And, while developing these skills is not always a linear process, the effort put into acquiring them will serve you throughout both your academic and professional life.

Here is a quick review of the main points from each of the chapters in this book:

1. The purpose of a literature review is to survey the current state of knowledge in the area of inquiry; to identify key authors, articles, theories, and findings in that area; and to identify gaps in knowledge in that research area. (Chapter 1)
2. Some common errors in many first-time literature reviews include:
   1. Accepts another researcher’s finding as valid without evaluating methodology and data
   2. Neglects to consider or mention contrary findings and alternative interpretations
   3. Findings are not clearly related to one’s own study or findings are too general.
   4. Allows insufficient time to define best search strategies and writing
   5. Simply reports individual studies rather than synthesizing the results
   6. Problems with selecting and using most relevant keywords and descriptors are evident.
   7. Relies too heavily on secondary sources
   8. Does not record or report search procedures
   9. Summarizes rather than synthesizes (Chapter 1)
3. By understanding what the literature in your field is, as well as how and when it is generated, you begin to know what is available and where to look for it. (Chapter 2)
4. Most graduate-level literature reviews begin with choosing a relevant, appropriate, interesting topic and then changing it. (Chapter 3)
5. Search and discovery of the literature is an iterative process.  There are many places to look and many tools and techniques to use to find resources.  Advanced researchers master this skill early on and refine it with each project. (Chapter 4)
6. You searched the literature and found lots of relevant resources. How do you now determine whether each item is an appropriate fit for your own review? (Chapter 5)
7. How will your resources be organized (alphabetically or chronologically)?  By broad general theme or theory?  Based on a type of methodology or population?  What citation management program or software are you going to use to keep track of all your references? (Chapter 6)
8. Your literature review is not a summary of all the articles you read but rather a synthesis that demonstrates a critical analysis of the papers you collected as well as your ability to integrate the results of your analysis into your own literature review. (Chapter 7)
9. Like any effective argument, the literature review is about both content and form.  It should have logical and smooth flow, a clear introduction and conclusion, and use a consistent citation style throughout. (Chapter 8)

Remember: Writing a good literature review takes time.  Start early.  Begin thinking about your topic and collect references even while you work on other tasks.  Write a first draft and then revise.  Go over the language, style, and form.  Focus, sharpen, clarify, and search again.  When you are satisfied with the result, you’re done.

How is the literature review evaluated?

It is usually judged in three main areas:

1. Selection of the literature
   1. Have you clearly indicated the scope and purpose of the review?
   2. Have you included a balanced coverage of what is available?
   3. Have you included the most recent and relevant studies?
   4. Have you included enough material to show the development and limitations in this area?
   5. Have you indicated the source of the literature by referencing accurately?
   6. Have you used mostly primary sources or appropriate secondary sources?
2. Critique of the literature
   1. Have you clearly (and logically) ordered and sorted the research, focusing on themes or ideas rather than the authors?
   2. Does the review move from broader concepts to a more specific focus?
   3. Is there adequate critique of research limitations, including design and methodology?
   4. How do the studies compare or contrast with debates or controversies highlighted?
   5. Is the relevance to your problem clear?
3. Summary and interpretation of the literature
   1. Have you made an overall interpretation of what is available?
   2. Do the implications provide theoretical or empirical justification for your own research questions/hypothesis?
   3. Do the implications provide a rationale for your research design? (RMIT University)

We hope that this discussion about literature reviews is useful.  After reading this guide, and reviewing the additional resources and activities in each chapter, we hope you have a better understanding of the research and writing process.  What conclusions have you reached regarding the content and structure of a literature review that can answer the question, “How do I write a graduate-level literature review?”

Additional Resources

Bell, J. (2005).  Doing Your Research Project: A Guide for First-Time Researchers in Education, Health and Social Science (4th ed.). New York: Open University Press.

Booth, A., Sutton, Anthea, & Papaioannou, Diana. (2016). Systematic Approaches to a Successful Literature Review (2nd ed.). Los Angeles: Sage Publications.

Collins, S. (2016). Professional Writing in the Health Disciplines.  http://epub-fhd.athabascau.ca/professionalwriting/   CC-BY-NC-SA 4.0

Coughlan, M., & Cronin, Patricia. (2017). Doing a Literature Review in Nursing, Health and Social Care (2nd ed.). Los Angeles: Sage Publications.

Fink, A. (2014). Conducting Research Literature Reviews (4th ed.). Los Angeles, CA: Sage Publications.

Galvan, J.L. (2009). Writing Literature Reviews: A Guide for Students of the Social and Behavioral Sciences.  Glendale, CA : Pyrczak

Garrard, J. (2017). Health Sciences Literature Review Made Easy: The Matrix Method.  Burlington, MA: Jones & Bartlett Learning.

Machi, L.A., & McEvoy, B.T. (2012). The Literature Review: Six Steps to Success.  Thousand Oaks, CA: Corwin.

Milardo, R.M. (2015). Crafting Scholarship in the Behavioral and Social Sciences: Writing, Reviewing, and Editing.  New York: Routledge.

Pautasso M. (2013). Ten simple rules for writing a literature review. PLoS Computational Biology 9(7): e1003149. https://doi.org/10.1371/journal.pcbi.1003149

Petticrew, M., & Roberts, H. (2006). Systematic Reviews in the Social Sciences.  Malden, MA: Blackwell Publishing.

Wallace, M., & Wray, A. (2016). Critical Reading and Writing for Postgraduates  (3rd ed.).  Los Angeles, CA: Sage Publications.

References

RMIT University (n.d.). Writing the literature review/Using the literature.  https://www.dlsweb.rmit.edu.au/lsu/content/2_assessmenttasks/assess_pdf/PG%20lit%20review.pdf