Banner

Literature Review - what is a Literature Review, why it is important and how it is done

  • Strategies to Find Sources

Evaluating Literature Reviews and Sources

Reading critically, tips to evaluate sources.

  • Tips for Writing Literature Reviews
  • Writing Literature Review: Useful Sites
  • Citation Resources
  • Other Academic Writings
  • Useful Resources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

  • A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips. First part focus on Introductions and in page 10 in the PDF, 37 in the text, it focus on "literature reviews".
  • Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

To be able to write a good Literature Review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

  • Who is the author? What is his/her standing in the field.
  • What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique or clarify?
  • Note the experts in the field: are there specific names/labs that are frequently cited?
  • Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
  • Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
  • Theories: have they evolved overtime?
  • Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

  • How to Read a Paper (University of Waterloo, Canada) This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.

Criteria to evaluate sources:

  • Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?
  • Usefulness : How this source related to your topic? How current or relevant it is to your topic?
  • Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful site - Critically Analyzing Information Sources (Cornell University Library)

  • << Previous: Strategies to Find Sources
  • Next: Tips for Writing Literature Reviews >>
  • Last Updated: Nov 25, 2021 10:46 AM
  • URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

  • UConn Library
  • Literature Review: The What, Why and How-to Guide
  • Evaluating Sources & Lit. Reviews

Literature Review: The What, Why and How-to Guide — Evaluating Sources & Lit. Reviews

  • Getting Started
  • Introduction
  • How to Pick a Topic
  • Strategies to Find Sources
  • Tips for Writing Literature Reviews
  • Writing Literature Review: Useful Sites
  • Citation Resources
  • Other Academic Writings

Reading Critically

To be able to write a good literature review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

  • Who is the author? What is his/her standing in the field?
  • What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique, or clarify?
  • Note the experts in the field. Are there specific names/labs that are frequently cited?
  • Pay attention to methodology. Is it sound? What testing procedures, subjects, and materials were used?
  • Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
  • Theories: Have they evolved overtime?
  • Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

Tips to Evaluate Sources

Criteria to evaluate sources:

Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?

Usefulness : How this source related to your topic? How current or relevant it is to your topic?

Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful sites

  • Critically Analyzing Information Sources (Cornell University Library)

Evaluating Literature Reviews and Sources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

  • A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips.

literature review source evaluation

  • Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.
  • << Previous: Strategies to Find Sources
  • Next: Tips for Writing Literature Reviews >>
  • Last Updated: Sep 21, 2022 2:16 PM
  • URL: https://guides.lib.uconn.edu/literaturereview

Creative Commons

Banner

Conducting a Literature Review

  • Getting Started
  • Define your Research Question
  • Finding Sources
  • Evaluating Sources
  • Organizing the Review
  • Cite and Manage your Sources

Introduction

The process of evaluating sources can take place when you first encounter a source, when you're reading it over, and as you incorporate it into your project. In general, some level of evaluation should take place at all of these stages, with different goals for each. The last evaluation will be discussed in "Organizing the Review," but we'll go over the first two here.

The overall purpose of evaluating sources is to make sure that your review has the most relevant, accurate, and unbiased literature in the field, so that you can determine what has already been learned about your topic and where further research may be needed.

Additional Resources

Cover Art

  • Evaluate Sources by Kansas State University Libraries
  • How to Evaluate Any Source by Skyline College Library

Evaluating Sources During the Initial Search Process

When you first encounter a potential source, you'll want to know very quickly whether it is worth reading in detail and considering for your literature review. To avoid wasting time on unhelpful sources as much as possible, it's generally best to run each article, book, or other resource you find through a quick checklist, using information you can find by skimming through the summary and introduction.

The two most common forms of early source evaluation are the "Big 5 Criteria" and the "CRAAP Test." These cover the same most significant variables for evaluation, and which one to use comes down to preference.

Big 5 Criteria:

The most important criteria for evaluating a potential resource are:

  • Currency : When the source was published
  • Coverage/Relevance : How closely related the source is to your topic and research question
  • Authority : Who wrote the source and whether they are likely to be credible on the subject
  • Accuracy : Whether the information is accurate or not (this will be more heavily evaluated further on in the research process, but you should skim through quickly for any obviously inaccurate information as a means of disqualifying the source)
  • Objectivity/Purpose : Whether or not the source presents a biased point of view or agenda

A good rule of thumb for  Currency  is that medical, scientific, and technology resources should be published within the last 5 years to prevent the information from being out-of-date; for less time-sensitive topics like history or the humanities, resources published within the last 5-10 years are often acceptable.

CRAAP Test:

A helpful mnemonic to remember the evaluation criteria, CRAAP is an acronym for:

Helpful questions for initial evaluation:

  • When was this source published?
  • Is this source relevant to your topic?
  • What are the author or authors' qualifications?
  • Is the resource scholarly/peer-reviewed?
  • Are sources cited to support the author's claims?
  • Does the website or journal the source comes from have a bias to their reporting?
  • Do you notice biased or emotional language in the summary or introduction?
  • Do you notice spelling or grammatical errors in a quick examination of the source?
  • Writing a Research Paper: Evaluate Sources by Kansas State University

Evaluating Sources During the Reading Process

Once a resource has passed the initial evaluation, you are ready to begin reading through it to more carefully determine if it belongs in your project. In addition to the questions posed above, which are always relevant to evaluating sources, you should look at your potential sources of literature with an eye to the following questions:

1. Is there any bias visible in the work?

You already began this process in the previous step and hopefully eliminated the most obviously unreliable sources, but as you read it is always important to keep an eye out for potential blind spots the author might have based on their own perspective. Bias is not inherently disqualifying -- a biased article may still have accurate information -- but it is essential to know if a bias exists and be aware of how it might impact how the information was gathered, evaluated, or delivered.

Peer-reviewed sources tend to be less likely to have this risk, because multiple editors had to go through the resource looking for mistakes or biases. They need to meet a much higher academic threshold.

2. How was the research conducted? Are there any strengths or weaknesses in its methodology?

It is important to understand how the study in your source was administered; a significant part of the literature review will be about potential gaps in the current research, so you need to understand how the existing research was done.

3. How does the author justify their conclusions?

Either through the results of their own research or by citing external evidence, an article, book, or other type of resource should provide proof of its claims. In the initial research process you checked to make sure that there was evidence supporting the author's assertions; now it is time to take a look at that evidence and see if you find it compelling, or if you think it doesn't justify the conclusions drawn by the article.

4. What similarities do these articles share?

Grouping your literature review by categories based on subtopic, findings, or chronology can be an extremely helpful way to organize your work. Take notes while you're reading of common themes and results to make planning the review easier.

5. Where does this research differ from the other sources?

While very close to the previous question, this one emphasizes what unique information, methodology, or insights a particular source brings to the overall understanding of the topic. What  new  knowledge is being brought to the table by this source that would justify it appearing in your literature review?

6. Does this source leave any unanswered questions or opportunities for further research?

Most scholarly journal articles will include a section near the end of the article addressing limits of their study and opportunities for further research. Examine these closely and see where other sources you have fill in the gaps, and where perhaps additional research needs to be done to gain a more complete understanding of your topic.

As you go through this process, you might find yourself eliminating certain sources that no longer seem like they fit with your project's goals, or getting inspired to search for additional sources based on new information you've found. This is a normal part of the research process, and additional searching may be necessary to fill in any gaps left after evaluating the sources you already have.

literature review source evaluation

  • << Previous: Finding Sources
  • Next: Organizing the Review >>
  • Last Updated: Dec 19, 2022 1:17 PM
  • URL: https://libraryservices.acphs.edu/lit_review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates, and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and its scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position your work in relation to other researchers and theorists
  • Show how your research addresses a gap or contributes to a debate
  • Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Literature review guide

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

literature review source evaluation

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models, and methods?
  • Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

  • Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically evaluate: mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarize yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved February 19, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Banner

Literature Review: The What, Why and How-to Guide: Evaluating Sources & Literature Reviews

  • Literature Reviews?
  • Strategies to Finding Sources
  • Keeping up with Research!
  • Evaluating Sources & Literature Reviews
  • Organizing for Writing
  • Writing Literature Review
  • Other Academic Writings

Evaluating Literature Reviews and Sources

  • Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

Criteria to evaluate sources:

  • Authority : Who is the author? What are the author's credentials and areas of expertise? Is he or she affiliated with a university?
  • Usefulness : How this source related to your topic? How current or relevant it is to your topic?
  • Reliability : Does the information comes from a reliable, trusted source such as an academic journal?
  • Critically Analyzing Information Sources: Critical Appraisal and Analysis (Cornell University Library) Ten things to look for when you evaluate an information source.

Reading Critically

Reading critically (summary from how to read academic texts critically).

  • Who is the author? What is his/her standing in the field?
  • What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, critique or clarify?
  • Note the experts in the field: are there specific names/labs that are frequently cited?
  • Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
  • Note conflicting theories, methodologies, and results. Are there any assumptions being made by most/some researchers?
  • Theories: have they evolved over time?
  • Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?
  • How to Read Academic Texts Critically Excellent document about how best to read critically academic articles and other texts.
  • How to Read an Academic Article This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.
  • << Previous: Keeping up with Research!
  • Next: Organizing for Writing >>
  • Last Updated: Jan 24, 2024 11:25 AM
  • URL: https://guides.library.ucsb.edu/litreview

Logo for Rebus Press

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 5: Evaluating Sources

Learning objectives.

At the conclusion of this chapter, you will be able to:

  • Critically evaluate the sources of the information you have found.
  • Evaluate the content of selected material for your purposes.

5.1 Overview of evaluation of sources

Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops. ( Association of College & Research Libraries, 2016 ).

You developed a viable research question, compiled a list of subject headings and keywords and spent a great deal of time searching the literature of your discipline or topic for sources. It’s now time to evaluate all of the information you found. Not only do you want to be sure of the source and the quality of the information, but you also want to determine whether each item is appropriate fit for your own review. This is also the point at which you make sure that you have searched out publications for all areas of your research question and go back into the literature for another search, if necessary.

In general, when we discuss evaluation of sources we are talking about looking at quality, accuracy, relevance, bias, reputation, currency, and credibility factors in a specific work, whether it’s a book, ebook, article, website, or blog posting. Before you include a source in your literature review, you should clearly understand what it is and why you are including it. According to Bennard et al., ( 2014 ), “Using inaccurate, irrelevant, or poorly researched sources can affect the quality of your own work.” (para. 4).

When evaluating a work for inclusion in, or exclusion from, your literature review, ask yourself a series of questions about each source.

5.1.1 Evaluating books

For primary and secondary sources you located in your search, use the ASAP mnemonic to evaluate inclusion in your literature review:

5.1.1.1 Age

Is it outdated? The answer to this question depends on your topic. If you are comparing historical classroom management techniques, something from 1965 might be appropriate. In Nursing, unless you are doing a historical comparison, a textbook from 5 years ago might be too dated for your needs.

A General Rule of Thumb:

5 years, maximum: medicine, health, education, technology, science 10-20 years: history, literature, art

5.1.1.2 Sources

Check reference or bibliography sources as well as those listed in footnotes or endnotes. Skim the list to see what kinds of sources the author used. When were the sources published? If the author is primarily citing works from 10 or 15 years ago, the book may not be what you need.

5.1.1.3 Author

Does the author have the credentials to write on the topic? Does the author have an academic degree or research grant funding? What else has the author published on the topic?

5.1.1.4 Publisher

Look for academic presses, including university presses. Books published under popular press imprints (such as Random House or Macmillan, in the U.S.) will not present scholarly research in the same way as Sage, Oxford, Harvard, or the University of Washington Press.

Other questions to ask about the book you may want to include in your literature review:

  • What is the book’s purpose? Why was it written? Who is the intended audience?
  • What is the conclusion or argument? How well is the main argument or conclusion supported?
  • Is it relevant to your research? How is it related to your research question?
  • Do you see any evidence of bias or unsubstantiated data?

5.1.2 Evaluating websites

In your research, it is likely you will discover information on the web that you will want to include in your literature review. For example, if your review is related to the current policy issues in public education in the United States, a potentially relevant information source may be a document located on the National Center for Education Statistics (NCES) website titled The Condition of Education 2017 . Likewise, for nursing, an article titled Discussing Vaccination with Concerned Patients: An Evidence-Based Resource for Healthcare Providers is available through the nursingcenter.com website. How do you evaluate these resources, and others like them?

Use the RADAR mnemonic ( Mandalios, 2013 ) to evaluate internet sources:

5.1.2.1 Relevance

How did you find the website and how is it relevant to your topic?

  • Was it recommended by a reliable source?
  • Was it cited in a scholarly source, such as a peer-reviewed journal?
  • Was it linked from a reputable site?

5.1.2.2 Authority

Look for the About page to find information about the purpose of the website . You may make a determination of its credibility based on what you find there. Does the page exhibit a particular point of view or bias? For example, a heart association or charter school may be promoting a particular perspective – how might that impact the objectivity of the information located on their site? Is there advertising or is there a product information attached to the content?

5.1.2.3 Date

  • When was the page created?
  • Is it kept up to date?
  • Are the links current and functional?

5.1.2.4 Appearance

  • Does the information presented appear to be factual?
  • Is the language formal or academic?
  • How does it compare to other information you have read on the topic?
  • Are references or links to cited material included?

5.1.2.5 Reason

What is the web address or URL? This can give you a clue about the purpose of the website, which may be to debate, advocate, advertise or sell, campaign, or present information. Here are some common domains and their origins:

  • .org – An advocacy website for an organization
  • .com – A private or commercial site
  • .net – A network organization or Internet provider/no longer frequently used
  • .edu – The site of a higher educational institution
  • .gov – A federal government site
  • .wa.us – A state government site which may include public schools and community colleges
  • .uk, .ca, .jm – A country site

Mike Caulfield ( 2017 ), the author of Web Literacy for Student Fact-Checkers , recommends a few simple strategies to evaluate a website (as well as social media):

  • Check for previous work: Look around to see if someone else has already provided a synthesis of the research described.
  • Go upstream to the source: Go “upstream” to the source of the claim. Most web content is not original. Get to the original source to understand the credibility and reliability of the information.
  • Read laterally: Read laterally. Once you get to the source of a claim, read what other people say about the source (publication, author, etc.). The truth is in the network.

5.1.3 Evaluating journal articles

It is likely that most of the resources you locate for your review will be from the scholarly literature of your discipline or in your topic area. As we have already seen, peer-reviewed articles are written by and for experts in a field. They generally describe formal research studies or experiments with the purpose of providing insight on a topic. You may have located these articles through Google, Google Scholar, a subscription or open access database, or citation searching. You now may want to know how to evaluate the usefulness for your research. As with the other resources, you are again looking for authority, accuracy, reliability, relevance, currency, and scope. Looking at each article as a separate and unique artifact, consider these elements in your evaluation:

5.1.3.1 Credibility/Authority

ASK: Who is the author? Is this person considered an expert in their field?

  • Search the author’s name in a general web search engine like Google.
  • What are the researcher’s academic credentials?
  • What else has this author written? Search by author in the databases and see how much they have published on any given subject.
  • How often or frequently has this article been cited by other scholars?

Citation analysis is the study of the impact and assumed quality of an article, an author, or an institution, based on the number of times works and/or authors have been cited by others. Google Scholar is a good way to get at this information.

Figure 5.1 shows a screen from Google Scholar for a scholarly article. Under the article citation information, the number of times the article has been cited by others is indicated.An example search result in Google Scholar, which lists the Article title (links to article), a brief description, and information about how many people cited the article, related articles, and a web search for the article. The image shows an article titled "The Anatomy of the Grid: Enabling Scalable..." that has been "Cited by 4030"

5.1.3.2 Accuracy

Check the facts. ASK:

  • Can statistics be verified through other sources?
  • Does this information seem to fit with what you have read in other sources?

5.1.3.3 Reliability/Objectivity

ASK: Is there an obvious bias? That doesn’t mean that you can’t use the information, it just means you need to take the bias into account.

  • Is a particular point of view or bias immediately obvious, or does it seem objective at first glance?
  • What point of view does the author represent? Are they clear about their point of view?
  • Is the article an editorial that is trying to argue a position?
  • Is the article in a publication with a particular editorial position?

5.1.3.4 Relevance

ASK: The hard questions:

  • Is the information relevant to your topic/thesis?
  • How does the article fit into the scope of the literature on this topic?
  • Is the material too technical or too clinical?
  • Is it too elementary or basic?
  • Does the information support your thesis or help you answer your question, or is it a challenge to make some kind of connection?
  • Does the information present an opposite point of view so you can show that you have addressed all sides of the argument in your paper?

5.1.3.5 Currency

  • When was the source published?
  • How important is current information to your topic, discipline, or paper type?
  • Does older material add to the history of the research? Or do you need something more current to support your thesis?

5.1.3.6 Scope and Purpose

To determine and evaluate in this category, ASK:

  • Is it a general work that provides an overview of the topic or is it specifically focused on only one aspect of your topic?
  • Does the breadth of the work match your expectations?
  • Is the article meant to inform, explain, persuade or sell something. Be aware of the purpose as you read the content and take that into consideration when deciding whether to use it or not.

For Nursing and other medical articles ASK:

  • What are the research methods used in the article?
  • Where does the method fall in the evidence pyramid? Systematic reviews and meta-analyses are the most credible, with articles that are opinions the least credible.

Figure 5.2 shows a triangle with different types of research studies listed in order of reliability and credibility. Meta analysis and systematic reviews are at the top of the pyramid, while animal research and editorials and opinions are at the bottom.

  • Meta Analysis: A systematic review that uses quantitative methods to summarize the results.
  • Systematic Review: An article in which the authors have systematically searched for, appraised, and summarized all of the medical literature on a specific topic.
  • Randomized Controlled Trials (RCTs): RCTs include a randomized group of patients in an experimental group, as well as a control group. These groups are monitored for the variables/outcomes of interest.
  • Cohort Study: Research identifies two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and follows these cohorts for a specified duration of time, in order to measure the outcome of interest.
  • Case Study: Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looks to see if they had the exposure of interest.
  • Animal Research / Lab Studies: Information creation begins at the bottom of the pyramid: This is where ideas and laboratory research take place. Ideas turn into therapies and diagnostic tools, which are then tested with lab models and animals.
  • Background Information / Expert Opinion: Handbooks, encyclopedias, and textbooks often provide a good foundation or introduction and often include generalized information about a condition. While background information presents a convenient summary, it typically takes about three years for this type of literature to be published.

5.1.4 Evaluating Social Media

Although social media (for example, Twitter or Facebook) is generally treated as an object under study rather than a source of information on a topic, the prevalence of social media as communication and sharing platforms must be acknowledged. It’s important to be skeptical of these sources, especially for inclusion in a literature review. However, as with any other web resource,you can evaluate a social media posting for authenticity by asking the following questions:

  • Location of the source – Is the author in the place they are tweeting or posting about?
  • Network – Who is in the author’s network and who follows the account?
  • Content – Can the information be corroborated from other sources?
  • Contextual updates – Does the author usually post or tweet on this topic? If so, what did past or updated posts say? Do they fill in more details?
  • Reliability – does the author cite sources and are those sources reliable? ( Sheridan Libraries, 2017 )

5.2 In Summary

Another way to think about evaluation of sources is to ask the 5W questions:

  • What type of document is it?
  • Who created it?
  • Why was the material published?
  • When was it published?
  • Where was the resource published?
  • How was the information gathered and presented? ( Radom, 2017 )

Locating sources for your literature review by using discovery layers, library catalogs, databases, search engines, and other search platforms may take a great deal of time and effort. Does everything you found and retrieved have value or worth to you as you write your own literature review? If the resource has not met the criteria above and you can’t justify its place in your literature review, it doesn’t deserve to be mentioned in your work. Include high-quality materials that are current, accurate, credible, and most importantly relevant to your research question, hypothesis, or topic.

Evaluate a Website

Watch this short video :

Using a search engine like Google, do a quick search for a topic that interests you. Select a website from your list of results and evaluate it using the elements of website evaluation listed earlier in this chapter.

  • How did you find the website?
  • What is the domain name (the URL) of the site?
  • What can you learn about the author/s of the site?
  • When was the site last updated?
  • Is it accurate based on what you know about the topic?
  • Are there references?
  • Do you notice any bias?
  • Is the site functional? (re links working? Or do they lead to non-functional pages?)

Evaluate a Book

Select a subject specific book or ebook that you can access quickly and evaluate it based on the ASAP criteria.

Evaluate an Article

You can practice evaluation using the attached articles. You don’t need to spend a lot of time with the article, but see if you can identify each of the elements of evaluation. Remember the elements of evaluation for articles are:

  • Authority/Credibility or Study Design for Nursing
  • Reliability/Objectivity
  • Scope and Purpose

For Education: Quality standards in e-learning: A matrix of analysis ( Frydenberg, 2002 ).

For Nursing: Beliefs and attitudes towards participating in genetic research ( Kerath et al, 2013 ).

Test Yourself

Check the Answer Key

For Nursing students: Your topic is the relationship between autism and vaccinations. Which of the two resources would you include in your literature review? Why?

  • Hviid, Anders, Michael Stellfield, Jan Wohlfart, and Mads Melbye. “Association Between Thimerosal-Containing Vaccine and Autism.” Journal of the American Medical Association 290, no. 13 (October 1, 2003): 1763–1766. http://jama.jamanetwork.com/article.aspx?articleid=197365
  • Chepkemoi Maina, Lillian, Simon Karanja, and Janeth Kombich. “Immunization Coverage and Its Determinants among Children Aged 12–23 Months in a Peri-Urban Area of Kenya.” Pan-African Medical Journal 14, no.3 (February 1, 2013). http://www.panafrican-med-journal.com/content/article/14/3/full/

For Education students: Your topic is music therapy in kindergarten classrooms in the United States. Which of the two resources would you include in your literature review? Why?”

  • Simpson, Kate, and Deb Keen. “Music Interventions for Children with Autism: Review of the Literature.” Journal of Autism and Developmental Disorders 41, no. 11 (November 2011): 1507-1514.
  • Bowman, Robert. “Approaches for Counseling Children Through Music.” Elementary School Guidance and Counseling 21, no. 4 (April 1987): 284-91.

Image attribution

Literature Reviews for Education and Nursing Graduate Students Copyright © by Linda Frederiksen is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Banner

Literature Review - Finding the Resources

  • The Literature
  • Search Tools
  • Formulating your search statement
  • Buliding on what you have found
  • Keeping Track

Evaluating your sources

Scholarly journals vs. non-scholarly journals, evaluate websites, critical reading.

  • Academic Reading
  • Citing Sources

Before deciding whether or not to incorporate what you have found into your literature review, you need to evaluate the resources to make sure that they contain information which is valuable and pertinent . This is especially true when the resources you retrieved are not collected by an academic library, but conveniently accessible through Internet search. Web resources need more careful thought to ensure their quality. Thus it is always a good practice to begin your search using CityU LibraryFind and databases for more authoritative and reliable resources.

Evaluation Criteria

Accuracy, authority, objectivity, currency and coverage are the five basic criteria for evaluating information from any sources.

It has been mentioned on " The Literature " page of this guide that a literature review generally consists of scholarly works. In addition to dissertations and theses, scholarly journal articles are another important sources to be incorporated in a literature review.

Many Library databases contain articles of various types of periodicals, including scholarly journals, magazines and newspapers. Most of these databases allow you to further limit your search results to "Scholarly Articles" so that you can view only academic research articles that in general report current original research.

  • EBSCOhost Databases
  • ProQuest Databases

The document below assists you in distinguishing scholarly journals from non-scholarly journals:

  • Types of Periodicals - Scholarly vs. Non-Scholarly Periodicals

Bearing in mind that the Web is a vast network of unfiltered information sources, (i.e., anyone can put anything on it, bypassing editorial or peer review). It is of utmost importance that we evaluate information on the Web before it is used and cited.

Here are some quick hints that can help you decide whether the information given in a particular web page is reliable or not:

  • Look for information about the author, e.g., links that say "Who we are", "About this site", etc.  
  • See if the author/web master provides e-mail address or other contact information so that he or she can be contacted for enquiries or further information.  
  • .com / .co -- a commercial site (may be trying to sell a product)
  • .edu  / .ac -- an educational institution (usually reliable but may not if it is a personal web page of a member of the institution)
  • .gov  -- a government department or agent
  • .net -- network access provider
  • .org -- a non-profit organization (may or may not be biased)  
  • a "~" in the URL usually indicates it is a personal web page e.g., http://personal.univ.edu/~smith/abc.htm The quality of information can vary greatly among personal web pages.

For more about evaluating information, visit the following sites:

Critically Analyzing Information Sources , from Research & Learning Serivces, Cornell University Library.

Evaluating Resources , from UC Berkeley Library.

Fake News, Propaganda, and Disinformation: Learning to Critically Evaluate Media Sources , from Cornell University Library.

  • << Previous: Keeping Track
  • Next: Academic Reading >>
  • Last Updated: Jan 17, 2024 7:25 PM
  • URL: https://libguides.library.cityu.edu.hk/litreview

© City University of Hong Kong | Copyright | Disclaimer

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Banner

Source Evaluation

  • Source Examples
  • Online Sources
  • Lesson Plan
  • Library Resources
  • Citing Sources
  • Literature Reviews

How to Develop a Literature Review

Watch this short video for an introduction to literature review!

literature review source evaluation

Literature Review Links

  • PhD on Track: Types of Reviews Narrative & Systematic
  • Purdue Owl: Literature Reviews
  • Purdue OWL: Writing a Literature Review

What is a Literature Review?

A  literature review  is a comprehensive study and interpretation of literature that addresses a specific topic.

literature review source evaluation

Literature reviews are generally conducted in one of two ways:

1) As a preliminary review before a larger study in order to critically evaluate the current literature and justify why further study and research is required.

2) As a project in itself that provides a comprehensive survey of the works published in a particular discipline or area of research over a specified period of time.  

Why conduct a literature review? They provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone.

More:   different types of literature reviews  on how to conduct a literature review.

What is a Primary Source?

A  primary source  is an original object or document -- the raw material or first-hand information, source material that is closest to what is being studied. Scientific and other peer reviewed journals are excellent sources for primary research.

literature review source evaluation

Primary sources vary by discipline and can include historical and legal documents, eye witness accounts, results of an experiment, statistical data, pieces of creative writing, and art objects.

In the sciences, the results of an experiment or study are typically found in scholarly articles or papers delivered at conferences, so those articles and papers that present the original results are considered primary sources.

A  secondary source  is something written about a primary source. Secondary sources include comments on, interpretations of, or discussions about the original material. You can think of secondary sources as second-hand information. If I tell you something, I am the primary source. If you tell someone else what I told you, you are the secondard source.

Secondary source materials can be articles in newspapers or popular magazines, book or movie reviews, or articles found in scholarly journals that evaluate or criticize someone else's original research.

Tertiary sources have the most fluid definition of the three levels of analysis.  Generally speaking, tertiary resources analyze and synthesize information about a given topic.  In other words, tertiary sources are information about information.  They summarize the research on a particular topic in a user-friendly form or list primary and secondary sources. 

Backward and Forward Citation Searching

Working from a reputable resource using forward and backward citation searching is one way to find related, reliable sources. 

Forward citation searching  involves seeing where the source has been cited AFTER it's publication. In an online platform, that may include a "cited by" or "times cited" icon on the page.

Backward citation searching  looks back at the research the author cites in their work in the form of a bibliography, which is generally found at the very end of a publication. 

Looking at both sources that cite the work after publication, and works cited by the author in the publication are great ways to both evaluate the source as a whole, and find other relates sources for your research. 

literature review source evaluation

https://scuhdrtraining.com/citation-searching/ 

  • Last Updated: Jan 5, 2024 12:10 PM
  • URL: https://libguides.kettering.edu/evaluation
  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

  • Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
  • Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
  • Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

  • << Previous: Theoretical Framework
  • Next: Citation Tracking >>
  • Last Updated: Feb 8, 2024 1:57 PM
  • URL: https://libguides.usc.edu/writingguide

Literature reviews as independent studies: guidelines for academic practice

  • Original Paper
  • Open access
  • Published: 14 October 2022
  • Volume 16 , pages 2577–2595, ( 2022 )

Cite this article

You have full access to this open access article

  • Sascha Kraus   ORCID: orcid.org/0000-0003-4886-7482 1 , 2 ,
  • Matthias Breier 3 ,
  • Weng Marc Lim 4 , 8 , 22 ,
  • Marina Dabić 5 , 6 ,
  • Satish Kumar 7 , 8 ,
  • Dominik Kanbach 9 , 10 ,
  • Debmalya Mukherjee 11 ,
  • Vincenzo Corvello 12 ,
  • Juan Piñeiro-Chousa 13 ,
  • Eric Liguori 14 ,
  • Daniel Palacios-Marqués 15 ,
  • Francesco Schiavone 16 , 17 ,
  • Alberto Ferraris 18 , 21 ,
  • Cristina Fernandes 19 , 20 &
  • João J. Ferreira 19  

55k Accesses

239 Citations

4 Altmetric

Explore all metrics

Review articles or literature reviews are a critical part of scientific research. While numerous guides on literature reviews exist, these are often limited to the philosophy of review procedures, protocols, and nomenclatures, triggering non-parsimonious reporting and confusion due to overlapping similarities. To address the aforementioned limitations, we adopt a pragmatic approach to demystify and shape the academic practice of conducting literature reviews. We concentrate on the types, focuses, considerations, methods, and contributions of literature reviews as independent, standalone studies. As such, our article serves as an overview that scholars can rely upon to navigate the fundamental elements of literature reviews as standalone and independent studies, without getting entangled in the complexities of review procedures, protocols, and nomenclatures.

Similar content being viewed by others

literature review source evaluation

How to Write and Publish a Research Paper for a Peer-Reviewed Journal

Clara Busse & Ella August

literature review source evaluation

What is Qualitative in Qualitative Research

Patrik Aspers & Ugo Corte

literature review source evaluation

Criteria for Good Qualitative Research: A Comprehensive Review

Drishti Yadav

Avoid common mistakes on your manuscript.

1 Introduction

A literature review – or a review article – is “a study that analyzes and synthesizes an existing body of literature by identifying, challenging, and advancing the building blocks of a theory through an examination of a body (or several bodies) of prior work (Post et al. 2020 , p. 352). Literature reviews as standalone pieces of work may allow researchers to enhance their understanding of prior work in their field, enabling them to more easily identify gaps in the body of literature and potential avenues for future research. More importantly, review articles may challenge established assumptions and norms of a given field or topic, recognize critical problems and factual errors, and stimulate future scientific conversations around that topic. Literature reviews Footnote 1 come in many different formats and purposes:

Some review articles conduct a critical evaluation of the literature, whereas others elect to adopt a more exploratory and descriptive approach.

Some reviews examine data, methodologies, and findings, whereas others look at constructs, themes, and theories.

Some reviews provide summaries by holistically synthesizing the existing research on a topic, whereas others adopt an integrative approach by assessing related and interdisciplinary work.

The number of review articles published as independent or standalone studies has been increasing over time. According to Scopus (i.e., search database ), reviews (i.e., document type ) were first published in journals (i.e., source type ) as independent studies in 1945, and they subsequently appeared in three digits yearly from the late 1980s to the late 1990s, four digits yearly from the early 2000s to the late 2010s, and five digits in the year 2021 (Fig.  1 ). This increase is indicative that reviewers and editors in business and management research alike see value and purpose in review articles to such a level that they are now commonly accepted as independent, standalone studies. This development is also reflected in the fact that some academic journals exclusively publish review articles (e.g., the Academy of Management Annals , or the  International Journal of Management Reviews ), and journals publishing in various fields often have special issues dedicated to literature reviews on certain topic areas (e.g., the Journal of Management and the Journal of International Business Studies ).

figure 1

Full-year publication trend of review articles on Scopus (1945–2021)

One of the most important prerequisites of a high-quality review article is that the work follows an established methodology, systematically selects and analyzes articles, and periodically covers the field to identify latest developments (Snyder 2019 ). Additionally, it needs to be reproducible, well-evidenced, and transparent, resulting in a sample inclusive of all relevant and appropriate studies (Gusenbauer and Haddaway 2020; Hansen et al. 2021 ). This observation is in line with Palmatier et al. ( 2018 ), who state that review articles provide an important synthesis of findings and perspectives in a given body of knowledge. Snyder ( 2019 ) also reaffirmed this rationale, pointing out that review articles have the power to answer research questions beyond that which can be achieved in a single study. Ultimately, readers of review articles stand to gain a one-stop, state-of-the-art synthesis (Lim et al. 2022a ; Popli et al. 2022) that encapsulates critical insights through the process of re-interpreting, re-organizing, and re-connecting a body knowledge (Fan et al. 2022 ).

There are many reasons to conduct review articles. Kraus et al. ( 2020 ) explicitly mention the benefits of conducting systematic reviews by declaring that they often represent the first step in the context of larger research projects, such as doctoral dissertations. When carrying out work of this kind, it is important that a holistic overview of the current state of literature is achieved and embedded into a proper synthesis. This allows researchers to pinpoint relevant research gaps and adequately fit future conceptual or empirical studies into the state of the academic discussion (Kraus et al., 2021 ). A review article as an independent or standalone study is a viable option for any academic – especially young scholars, such as doctoral candidates – who wishes to delve into a specific topic for which a (recent) review article is not available.

The process of conducting a review article can be challenging, especially for novice scholars (Boell and Cecez-Kecmanovic 2015 ). Therefore, it is not surprising that numerous guides have been written in an attempt to improve the quality of review studies and support emerging scholars in their endeavors to have their work published. These guides for conducting review articles span a variety of academic fields, such as engineering education (Borrego et al. 2014 ), health sciences (Cajal et al. 2020 ), psychology (Laher and Hassem 2020 ), supply chain management (Durach et al. 2017 ), or business and entrepreneurship (Kraus et al. 2020 ; Tranfield et al. 2003 ) – the latter were among the first scholars to recognize the need to educate business/management scholars on the roles of review studies in assembling, ascertaining, and assessing the intellectual territory of a specific knowledge domain. Furthermore, they shed light on the stages (i.e., planning the review, conducting the review, reporting, and dissemination) and phases (i.e., identifying the need for a review, preparation of a proposal for a review, development of a review protocol, identification of research, selection of studies, study quality assessment, data extraction and monitoring progress, data synthesis, the report and recommendations, and getting evidence into practice) of conducting a systematic review. Other scholars have either adapted and/or developed new procedures (Kraus et al. 2020 ; Snyder 2019 ) or established review protocols such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram (Moher et al. 2015 ). The latter provides a checklist that improves transparency and reproducibility, thus reducing questionable research practices. The declarative and procedural knowledge of a checklist allows users to derive value from (and, in some cases, produce) methodological literature reviews.

Two distinct and critical gaps or issues provide impetus for our article. First, while the endeavors of the named scholars are undoubtedly valuable contributions, they often encourage other scholars to explain the methodology of their review studies in a non-parsimonious way ( 1st issue ). This can become problematic if this information distracts and deprives scholars from providing richer review findings, particularly in instances in which publication outlets impose a strict page and/or word limit. More often than not, the early parts (i.e., stages/phases, such as needs, aims, and scope) of these procedures or protocols are explained in the introduction, but they tend to be reiterated in the methodology section due to the prescription of these procedures or protocols. Other parts of these procedures or protocols could also be reported more parsimoniously, for example, by filtering out documents, given that scientific databases (such as Scopus or Web of Science ) have since been upgraded to allow scholars to select and implement filtering criteria when conducting a search (i.e., criterion-by-criterion filtering may no longer be necessary). More often than not, the procedures or protocols of review studies can be signposted (e.g., bracket labeling) and disclosed in a sharp and succinct manner while maintaining transparency and replicability.

Other guides have been written to introduce review nomenclatures (i.e., names/naming) and their equivalent philosophical underpinnings. Palmatier et al. ( 2018 ) introduced three clearly but broadly defined nomenclatures of literature reviews as independent studies: domain-based reviews, theory-based reviews, and method-based reviews. However, such review nomenclatures can be confusing due to their overlapping similarities ( 2nd issue ). For example, Lim et al. ( 2022a ) highlighted their observation that the review nomenclatures associated with domain-based reviews could also be used for theory-based and method-based reviews.

The two aforementioned issues – i.e., the lack of a parsimonious understanding and the reporting of the review methodology , and the confusion emerging from review nomenclatures – are inarguably the unintended outcomes of diving into an advanced (i.e., higher level) understanding of literature review procedures, protocols, and nomenclatures from a philosophical perspective (i.e., underpinnings) without a foundational (i.e., basic level) understanding of the fundamental (i.e., core) elements of literature reviews from a pragmatic perspective. Our article aims to shed light on these issues and hopes to provide clarity for future scholarly endeavors.

Having a foundational understanding of literature reviews as independent studies is (i) necessary when addressing the aforementioned issues; (ii) important in reconciling and scaffolding our understanding, and (iii) relevant and timely due to the proliferation of literature reviews as independent studies. To contribute a solution toward addressing this gap , we aim to demystify review articles as independent studies from a pragmatic standpoint (i.e., practicality). To do so, we deliberately (i) move away from review procedures, protocols, and nomenclatures, and (ii) invest our attention in developing a parsimonious, scaffolded understanding of the fundamental elements (i.e., types, focuses, considerations, methods, and contributions) of review articles as independent studies.

Three contributions distinguish our article. It is worth noting that pragmatic guides (i.e., foundational knowledge), such as the present one, are not at odds with extant philosophical guides (i.e., advanced knowledge), but rather they complement them. Having a foundational knowledge of the fundamental elements of literature reviews as independent studies is valuable , as it can help scholars to (i) gain a good grasp of the fundamental elements of literature reviews as independent studies ( 1st contribution ), and (ii) mindfully adopt or adapt existing review procedures, protocols, and nomenclatures to better suit the circumstances of their reviews (e.g., choosing and developing a well-defined review nomenclature, and choosing and reporting on review considerations and steps more parsimoniously) ( 2nd contribution ). Therefore, this pragmatic guide serves as (iii) a foundational article (i.e., preparatory understanding) for literature reviews as independent studies ( 3rd contribution ). Following this, extant guides using a philosophical approach (i.e., advanced understanding) could be relied upon to make informed review decisions (e.g., adoption, adaptation) in response to the conventions of extant review procedures, protocols, and nomenclatures (Fig.  2 ).

figure 2

Foundational and advanced understanding of literature reviews as independent studies

2 Fundamental elements of literature reviews as independent studies

A foundational understanding of literature reviews as independent studies can be acquired through the appreciation of five fundamental elements – i.e., types, focuses, considerations, methods, and contributions – which are illustrated in Fig.  3 and summarized in the following sections.

figure 3

Fundamental elements of literature reviews as independent studies

There are two types of literature reviews as independent studies: systematic literature reviews ( SLRs ) and non-systematic literature reviews ( non-SLRs ). It is important to recognize that SLRs and non-SLRs are not review nomenclatures (i.e., names/naming) but rather review types (i.e., classifications).

In particular, SLRs are reviews carried out in a systematic way using an adopted or adapted procedure or protocol to guide data curation and analysis, thus enabling transparent disclosure and replicability (Lim et al. 2022a ; Kraus et al. 2020 ). Therefore, any review nomenclature guided by a systematic methodology is essentially an SLR. The origin of this type of literature review can be traced back to the evidence-based medicine movement in the early 1990s, with the objective being to overcome the issue of inconclusive findings in studies for medical treatments (Boell and Cecez-Kecmanovic 2015 ).

In contrast, non-SLRs are reviews conducted without any systematic procedure or protocol; instead, they weave together relevant literature based on the critical evaluations and (subjective) choices of the author(s) through a process of discovery and critique (e.g., pointing out contradictions and questioning assertions or beliefs); they are shaped by the exposure, expertise, and experience (i.e., the “3Es” in judgement calls) of the author(s). Therefore, non-SLRs are essentially critical reviews of the literature (Lim and Weissmann 2021 ).

2.2 Focuses

Unlike Palmatier et al. ( 2018 ) who considered domain-based reviews, theory-based reviews, and method-based reviews as review nomenclatures, we consider domain , theory , and method as three substantive focuses that can take center stage in literature reviews as independent studies. This is in line with our attempt to move away from review nomenclatures when providing a foundational understanding of literature reviews as independent studies.

A review that is domain-focused can examine: (i) a  concept (e.g., customer engagement; Lim et al. 2022b ; digital transformation; Kraus et al. 2021 ; home sharing; Lim et al. 2021 ; sharing economy; Lim 2020 ), (ii) a context (e.g., India; Mukherjee et al. 2022a ), (iii) a discipline (e.g., entrepreneurship; Ferreira et al. 2015 ; international business; Ghauri et al. 2021 ), (iv) a field (e.g., family business; Lahiri et al. 2020 ; Rovelli et al. 2021 ; female entrepreneurship; Ojong et al. 2021 ), or (v) an outlet (e.g., Journal of Business Research ; Donthu et al. 2020 ; Management International Review ; Mukherjee et al. 2021 ; Review of Managerial Science ; Mas-Tur et al. 2020 ), which typically offer broad, overarching insights.

Domain-focused hybrids , such as the between-domain hybrid (e.g., concept-discipline hybrid, such as digital transformation in business and management; Kraus et al. 2022 ; religion in business and entrepreneurship; Kumar et al. 2022a ; personality traits in entrepreneurship; Salmony and Kanbach 2022 ; and policy implications in HR and OB research; Aguinis et al., 2022 ) and the within-domain hybrid (e.g., the concept-concept hybrid, such as customer engagement and social media; Lim and Rasul 2022 ; and global business and organizational excellence; Lim 2022 ; and the discipline-discipline hybrid, such as neuromarketing; Lim 2018 ) are also common as they can provide finer-grained insights.

A review that is theory-focused can explore a standalone theory (e.g., theory of planned behavior; Duan and Jiang 2008 ), as well as a theory in conjunction with a domain , such as the concept-theory hybrid (e.g., behavioral control and theory of planned behavior; Lim and Weissmann 2021 ) and the theory-discipline hybrid (e.g., theory of planned behavior in hospitality, leisure, and tourism; Ulker-Demirel and Ciftci 2020 ), or a theory in conjunction with a method (e.g., theory of planned behavior and structural equation modeling).

A review that is method-focused can investigate a standalone method (e.g., structural equation modeling; Deng et al. 2018 ) or a method in conjunction with a domain , such as the method-discipline hybrid (e.g., fsQCA in business and management; Kumar et al. 2022b ).

2.3 Planning the review, critical considerations, and data collection

The considerations required for literature reviews as independent studies depend on their type: SLRs or non-SLRs.

For non-SLRs, scholars often rely on the 3Es (i.e., exposure, expertise, and experience) to provide a critical review of the literature. Scholars who embark on non-SLRs should be well versed with the literature they are dealing with. They should know the state of the literature (e.g., debatable, underexplored, and well-established knowledge areas) and how it needs to be deciphered (e.g., tenets and issues) and approached (e.g., reconciliation proposals and new pathways) to advance theory and practice. In this regard, non-SLRs follow a deductive reasoning approach, whereby scholars initially develop a set of coverage areas for reviewing a domain, theory, or method and subsequently draw on relevant literature to shed light and support scholarly contentions in each area.

For SLRs, scholars often rely on a set of criteria to provide a well-scoped (i.e., breadth and depth), structured (i.e., organized aspects), integrated (i.e., synthesized evidence) and interpreted/narrated (i.e., describing what has happened, how and why) systematic review of the literature. Footnote 2 In this regard, SLRs follow an inductive reasoning approach, whereby a set of criteria is established and implemented to develop a corpus of scholarly documents that scholars can review. They can then deliver a state-of-the-art overview, as well as a future agenda for a domain, theory, or method. Such criteria are often listed in philosophical guides on SLR procedures (e.g., Kraus et al. 2020 ; Snyder 2019 ) and protocols (e.g., PRISMA), and they may be adopted/adapted with justifications Footnote 3 . Based on their commonalities they can be summarized as follows:

Search database (e.g., “Scopus” and/or “Web of Science”) can be defined based on justified evidence (e.g., by the two being the largest scientific databases of scholarly articles that can provide on-demand bibliographic data or records; Pranckutė 2021 ). To avoid biased outcomes due to the scope covered by the selected database, researchers could utilize two or more different databases (Dabić et al. 2021 ).

Search keywords may be developed by reading scholarly documents and subsequently brainstorming with experts. The expanding number of databases, journals, periodicals, automated approaches, and semi-automated procedures that use text mining and machine learning can offer researchers the ability to source new, relevant research and forecast the citations of influential studies. This enables them to determine further relevant articles.

Boolean operators (e.g., AND, OR) should be strategically used in developing the  string   of search keywords (e.g., “engagement” AND “customer” OR “consumer” OR “business”). Furthermore, the correct and precise application of quotation marks is important but is very frequently sidestepped, resulting in incorrect selection processes and differentiated results.

Search period (e.g., between a specified period [e.g., 2000 to 2020] or up to the latest full year at the time or writing [e.g., up to 2021]) can be defined based on the justified scope of study (e.g., contemporary evolution versus historical trajectory).

Search field (e.g., “article title, abstract, keywords”) can be defined based on justified assumptions (e.g., it is assumed that the focus of relevant documents will be mentioned in the article title, abstract, and/or keywords).

Subject area (e.g., “business, management, and accounting”) can be defined based on justified principles (e.g., the focus of the review is on the marketing discipline, which is located under the “business, management, and accounting” subject area in Scopus).

Publication stage (e.g., “final”) can be defined based on justified grounds (e.g., enabling greater accuracy in replication).

Document type (e.g., “article” and/or “review”), which reflects the type of scientific/practical contributions (e.g., empirical, synthesis, thought), can be defined based on justified rationales (e.g., articles selected because they are peer-reviewed; editorials not selected because they are not peer-reviewed).

Source type (e.g., “journal”) can be defined based on justified reasons (e.g., journals selected because they publish finalized work; conference proceedings not selected because they are work in progress, and in business/management, they are usually not being considered as full-fledged “publications”).

Language (e.g., “English”) can be determined based on justified limitations (e.g., nowadays, there are not many reasons to use another language besides the academic lingua franca English). Different spellings should also be considered, as the literature may contain both American and British spelling variants (e.g., organization and organisation). Truncation and wildcards in searches are recommended to capture both sets of spellings. It is important to note that each database varies in its symbology.

Quality filtering (e.g., “A*” and “A” or “4*”, “4”, and “3”) can be defined based on justified motivations (e.g., the goal is to unpack the most originally and rigorously produced knowledge, which is the hallmark of premier journals, such as those ranked “A*” and “A” by the Australian Business Deans Council [ABDC] Journal Quality List [JQL] and rated “4*”, “4”, and “3” by the Chartered Association of Business Schools [CABS] Academic Journal Guide [AJG]).

Document relevance (i.e., within the focus of the review) can be defined based on justified judgement (e.g., for a review focusing on customer engagement, articles that mention customer engagement as a passing remark without actually investigating it would be excluded).

Others: Screening process should be accomplished by beginning with the deduction of duplicate results from other databases, tracked using abstract screening to exclude unfitting studies, and ending with the full-text screening of the remaining documents.

Others: Exclusion-inclusion criteria interpretation of the abstracts/articles is obligatory when deciding whether or not the articles dealt with the matter. This step could involve removing a huge percentage of initially recognized articles.

Others: Codebook building pertains to the development of a codebook of the main descriptors within a specific field. An inductive approach can be followed and, in this case, descriptors are not established beforehand. Instead, they are established through the analysis of the articles’ content. This procedure is made up of several stages: (i) the extraction of important content from titles, abstracts, and keywords; (ii) the classification of this content to form a reduced list of the core descriptors; and (iii) revising the codebook in iterations and combining similar categories, thus developing a short list of descriptors (López-Duarte et al. 2016 , p. 512; Dabić et al. 2015 ; Vlacic et al. 2021 ).

2.4 Methods

Various methods are used to analyze the pertinent literature. Often, scholars choose a method for corpus analysis before corpus curation. Knowing the analytical technique beforehand is useful, as it allows researchers to acquire and prepare the right data in the right format. This typically occurs when scholars have decided upon and justified pursuing a specific review nomenclature upfront (e.g., bibliometric reviews) based on the problem at hand (e.g., broad domain [outlet] with a large corpus [thousands of articles], such as a premier journal that has been publishing for decades) (Donthu et al. 2021 ). However, this may not be applicable in instances where (i) scholars do not curate a corpus of articles (non-SLRs), and (ii) scholars only know the size of the corpus of articles once that corpus is curated (SLRs). Therefore, scholars may wish to decide on a method of analyzing the literature depending on (i) whether they rely on a corpus of articles (i.e., yes or no), and (ii) the size of the corpus of articles that they rely on to review the literature (i.e., n  = 0 to ∞).

When analytical techniques (e.g., bibliometric analysis, critical analysis, meta-analysis) are decoupled from review nomenclatures (e.g., bibliometric reviews, critical reviews, meta-analytical reviews), we uncover a toolbox of the following methods for use when analyzing the literature:

Bibliometric analysis measures the literature and processes data by using algorithm, arithmetic, and statistics to analyze, explore, organize, and investigate large amounts of data. This enables scholars to identify and recognize potential “hidden patterns” that could help them during the literature review process. Bibliometrics allows scholars to objectively analyze a large corpus of articles (e.g., high hundreds or more) using quantitative techniques (Donthu et al. 2021 ). There are two overarching categories for bibliometric analysis: performance analysis and science mapping. Performance analysis enables scholars to assess the productivity (publication) and impact (citation) of the literature relating to a domain, method, or theory using various quantitative metrics (e.g., average citations per publication or year, h -index, g -index, i -index). Science mapping grants scholars the ability to map the literature in that domain, method, or theory based on bibliographic data (e.g., bibliographic coupling generates thematic clusters based on similarities in shared bibliographic data [e.g., references] among citing articles; co-citation analysis generates thematic clusters based on commonly cited articles; co-occurrence analysis generates thematic clusters based on bibliographic data [e.g., keywords] that commonly appear together; PageRank analysis generates thematic clusters based on articles that are commonly cited in highly cited articles; and topic modeling generates thematic clusters based on the natural language processing of bibliographic data [e.g., article title, abstract, and keywords]). Footnote 4 Given the advancement in algorithms and technology, reviews using bibliometric analysis are considered to be smart (Kraus et al. 2021 ) and technologically-empowered (Kumar et al. 2022b ) SLRs, in which a review has harnessed the benefits of (i) the machine learning of the bibliographic data of scholarly research from technologically-empowered scientific databases, and (ii) big data analytics involving various science mapping techniques (Kumar et al. 2022c ).

Content analysis allows scholars to analyze a small to medium corpus of articles (i.e., tens to low hundreds) using quantitative and qualitative techniques. From a quantitative perspective , scholars can objectively carry out a content analysis by quantifying a specific unit of analysis . A useful method of doing so involves adopting, adapting, or developing an organizing framework . For example, Lim et al. ( 2021 ) employed an organizing (ADO-TCM) framework to quantify content in academic literature based on: (i) the categories of knowledge; (ii) the relationships between antecedents, decisions, and outcomes; and (iii) the theories, contexts, and methods used to develop the understanding for (i) and (ii). The rapid evolution of software for content analysis allows scholars to carry out complex elaborations on the corpus of analyzed articles, so much so that the most recent software enables the semi-automatic development of an organizing framework (Ammirato et al. 2022 ). From a qualitative perspective , scholars can conduct a content analysis or, more specifically, a thematic analysis , by subjectively organizing the content into themes. For example, Creevey et al. ( 2022 ) reviewed the literature on social media and luxury, providing insights on five core themes (i.e., luxury brand strategy, luxury brand social media communications, luxury consumer attitudes and perceptions, engagement, and the influence of social media on brand performance-related outcomes) generated through a content (thematic) analysis. Systematic approaches for inductive concept development through qualitative research are similarly applied in literature reviews in an attempt to reduce the subjectivity of derived themes. Following the principles of the approach by Gioia et al. ( 2012 ), Korherr and Kanbach ( 2021 ) develop a taxonomy of human-related capabilities in big data analytics. Building on a sample of 75 studies for the literature review, 33 first-order concepts are identified. These are categorized into 15 second-order themes and are finally merged into five aggregate dimensions. Using the same procedure, Leemann and Kanbach ( 2022 ) identify 240 idiosyncratic dynamic capabilities in a sample of 34 studies for their literature review. They then categorize these into 19 dynamic sub-capabilities. The advancement of technology also makes it possible to conduct content analysis using computer assisted qualitative data analysis (CAQDA) software (e.g., ATLAS.ti, Nvivo, Quirkos) (Lim et al. 2022a ).

Critical analysis allows scholars to subjectively use their 3Es (i.e., exposure, expertise, and experience) to provide a critical evaluation of academic literature. This analysis is typically used in non-SLRs, and can be deployed in tandem with other analyses, such as bibliometric analysis and content analysis in SLRs, which are used to discuss consensual, contradictory, and underexplored areas of the literature. For SLRs, scholars are encouraged to engage in critical evaluations of the literature so that they can truly contribute to advancing theory and practice (Baker et al. 2022 ; Lim et al. 2022a ; Mukherjee et al. 2022b ).

Meta-analysis allows scholars to objectively establish a quantitative estimate of commonly studied relationships in the literature (Grewal et al. 2018 ). This analysis is typically employed in SLRs intending to reconcile a myriad of relationships (Lim et al. 2022a ). The relationships established are often made up of conflicting evidence (e.g., a positive or significant effect in one study, but a negative or non-significant effect in another study). However, through meta-analysis, scholars are able to identify potential factors (e.g., contexts or sociodemographic information) that may have led to the conflict.

Others: Multiple correspondence analysis helps to map the field, assessing the associations between qualitative content within a matrix of variables and cases. Homogeneity Analysis by Means of Alternating Least Squares ( HOMALS ) is also considered useful in allowing researchers to map out the intellectual structure of a variety of research fields (Gonzalez-Loureiro et al. 2015 ; Gonzalez-Louriero 2021; Obradović et al. 2021 ). HOMALS can be performed in R or used along with a matrix through SPSS software. In summary, the overall objective of this analysis is to discover a low dimensional representation of the original high dimensional space (i.e., the matrix of descriptors and articles). To measure the goodness of fit, a loss function is used. This function is used minimally, and the HOMALS algorithm is applied to the least squares loss functions in SPSS. This analysis provides a proximity map, in which articles and descriptors are shown in low-dimensional spaces (typically on two axes). Keywords are paired and each couple that appears together in a large number of articles is shown to be closer on the map and vice-versa.

When conducting a literature review, software solutions allow researchers to cover a broad range of variables, from built-in functions of statistical software packages to software orientated towards meta-analyses, and from commercial to open-source solutions. Personal preference plays a huge role, but the decision as to which software will be the most useful is entirely dependent on how complex the methods and the dataset are. Of all the commercial software providers, we have found the built-in functions of (i) R and VOSviewer most useful in performing bibliometric analysis (Aria and Cuccurullo 2017 ; R Core Team 2021 ; Van Eck and Waltman 2014 ) and (ii) Stata most useful in performing meta-analytical tasks.

Many different analytical tools have been used. These include simple document counting, citation analysis, word frequency analysis, cluster analysis, co-word analysis, and cooperation analysis (Daim et al. 2006 ). Software has also been produced for bibliometric analysis, such as the Thomson Data Analyzer (TDA), which Thomson Reuters created, and CiteSpace developed by Chen ( 2013 ). VOSviewer helps us to construct and visualize bibliometric networks, which can include articles, journals, authors, countries, and institutions, among others (Van Eck and Waltman 2014 ). These can be organized based on citations, co-citations, bibliographic coupling, or co-authorship relations. In addition, VOSviewer provides text mining functions, which can be used to facilitate a better understanding of co-occurrence networks with regards to the key terms taken from a body of scientific literature (Donthu et al. 2021 ; Wong 2018 ). Other frequently used tools include for bibliometric analysis include Bibliometrix/Biblioshiny in R, CitNetExplorer, and Gephi, among others.

2.5 Contributions

Well-conducted literature reviews may make multiple contributions to the literature as standalone, independent studies.

Generally, there are three primary contributions of literature reviews as independent studies: (i) to provide an overview of current knowledge in the domain, method, or theory, (ii) to provide an evaluation of knowledge progression in the domain, method, or theory, including the establishment of key knowledge, conflicting or inconclusive findings, and emerging and underexplored areas, and (iii) to provide a proposal for potential pathways for advancing knowledge in the domain, method, or theory (Lim et al. 2022a , p. 487). Developing theory through literature reviews can take many forms, including organizing and categorizing the literature, problematizing the literature, identifying and exposing contradictions, developing analogies and metaphors, and setting out new narratives and conceptualizations (Breslin and Gatrell 2020 ). Taken collectively, these contributions offer crystalized, evidence-based insights that both ‘mine’ and ‘prospect’ the literature, highlighting extant gaps and how they can be resolved (e.g., flags paradoxes or theoretical tensions, explaining why something has not been done, what the challenges are, and how these challenges can be overcome). These contributions can be derived through successful bibliometric analysis, content analysis, critical analysis, and meta-analysis.

Additionally, the deployment of specific methods can bring in further added value. For example, a performance analysis in a bibliometric analysis can contribute to: (i) objectively assessing and reporting research productivity and impact ; (ii) ascertaining reach for coverage claims ; (iii) identifying social dominance and hidden biases ; (iv) detecting anomalies ; and (v) evaluating ( equitable ) relative performance ; whereas science mapping in bibliometric analysis can contribute to: (i) objectively discovering thematic clusters of knowledge ; (ii) clarifying nomological networks ; (iii) mapping social patterns ; (iv) tracking evolutionary nuances ; and (v) recognizing knowledge gaps (Mukherjee et al. 2022b , p. 105).

3 Conclusion

Independent literature reviews will continue to be written as a result of their necessity, importance, relevance, and urgency when it comes to advancing knowledge (Lim et al. 2022a ; Mukherjee et al. 2022b ), and this can be seen in the increasing number of reviews being published over the last several years. Literature reviews advance academic discussion. Journal publications on various topics and subject areas are becoming more frequent sites for publication. This trend will only heighten the need for literature reviews. This article offers directions and control points that address the needs of three different stakeholder groups: producers (i.e., potential authors), evaluators (i.e., journal editors and reviewers), and users (i.e., new researchers looking to learn more about a particular methodological issue, and those teaching the next generation of scholars). Future producers will derive value from this article’s teachings on the different fundamental elements and methodological nuances of literature reviews. Procedural knowledge (i.e., using control points to assist in decision-making during the manuscript preparation phase) will also be of use. Evaluators will be able to make use of the procedural and declarative knowledge evident in control points as well. As previously outlined, the need to cultivate novelty within research on business and management practices is vital. Scholars must also be supported to choose not only safe mining approaches; they should also be encouraged to attempt more challenging and risky ventures. It is important to note that abstracts often seem to offer a lot of potential, stating that authors intend to make large conceptual contributions, broadening the horizons of the field.

Our article offers important insights also for practitioners. Noteworthily, our framework can support corporate managers in decomposing and better understanding literature reviews as ad-hoc and independent studies about specific topics that matter for their organization. For instance, practitioners can understand more easily what are the emerging trends within their domain of interest and make corporate decisions in line with such trends.

This article arises from an intentional decoupling from philosophy, in favor of adopting a more pragmatic approach. This approach can assist us in clarifying the fundamental elements of literature reviews as independent studies. Five fundamental elements must be considered: types, focuses, considerations, methods, and contributions. These elements offer a useful frame for scholars starting to work on a literature review. Overview articles (guides) such as ours are thus invaluable, as they equip scholars with a solid foundational understanding of the integral elements of a literature review. Scholars can then put these teachings into practice, armed with a better understanding of the philosophy that underpins the procedures, protocols, and nomenclatures of literature reviews as independent studies.

Data availability

Our manuscript has no associate data.

Our focus here is on standalone literature reviews in contrast with literature reviews that form the theoretical foundation for a research article.

Scoping reviews, structured reviews, integrative reviews, and interpretive/narrative reviews are commonly found in review nomenclature. However, the philosophy of these review nomenclatures essentially reflects what constitutes a good SLR. That is to say, a good SLR should be well scoped, structured, integrated, and interpreted/narrated. This observation reaffirms our position and the value of moving away from review nomenclatures to gain a foundational understanding of literature reviews as independent studies.

Given that many of these considerations can be implemented simultaneously in contemporary versions of scientific databases, scholars may choose to consolidate them into a single (or a few) step(s), where appropriate, so that they can be reported more parsimoniously. For a parsimonious but transparent and replicable exemplar, see Lim ( 2022 ).

Where keywords are present (e.g., author keywords or keywords derived from machine learning [e.g., natural language processing]), it is assumed that each keyword represents a specific meaning (e.g., topic [concept, context], method), and that a collection of keywords grouped under the same cluster represents a specific theme.

Aguinis H, Jensen SH, Kraus S (2022) Policy implications of organizational behavior and human resource management research. Acad Manage Perspect 36(3):1–22

Article   Google Scholar  

Ammirato S, Felicetti AM, Rogano D, Linzalone R, Corvello V (2022) Digitalising the systematic literature review process: The My SLR platform. Knowl Manage Res Pract. doi: https://doi.org/10.1080/14778238.2022.2041375

Aria M, Cuccurullo C (2017) bibliometrix: An R-tool for comprehensive science mapping analysis. J Informetrics 11(4):959–975

Baker WE, Mukherjee D, Perin MG (2022) Learning orientation and competitive advantage: A critical synthesis and future directions. J Bus Res 144:863–873

Boell SK, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature reviews. J Inform Technol 30:161–173

Borrego M, Foster MJ, Froyd JE (2014) Systematic literature reviews in engineering education and other developing interdisciplinary fields. J Eng Educ 103(1):45–76

Breslin D, Gatrell C (2020) Theorizing through literature reviews: The miner-prospector continuum. Organizational Res Methods. https://doi.org/10.1177/1094428120943288 (in press)

Cajal B, Jiménez R, Gervilla E, Montaño JJ (2020) Doing a systematic review in health sciences. Clínica y Salud 31(2):77–83

Chen C (2013) Mapping scientific frontiers: The quest for knowledge visualization. Springer Science & Business Media

Creevey D, Coughlan J, O’Connor C (2022) Social media and luxury: A systematic literature review. Int J Manage Reviews 24(1):99–129

Dabić M, González-Loureiro M, Harvey M (2015) Evolving research on expatriates: what is ‘known’after four decades (1970–2012). Int J Hum Resource Manage 26(3):316–337

Dabić M, Vlačić B, Kiessling T, Caputo A, Pellegrini M(2021) Serial entrepreneurs: A review of literature and guidance for future research.Journal of Small Business Management,1–36

Daim TU, Rueda G, Martin H, Gerdsri P (2006) Forecasting emerging technologies: Use of bibliometrics and patent analysis. Technol Forecast Soc Chang 73(8):981–1012

Deng L, Yang M, Marcoulides KM (2018) Structural equation modeling with many variables: A systematic review of issues and developments. Front Psychol 9:580

Donthu N, Kumar S, Pattnaik D (2020) Forty-five years of Journal of Business Research: A bibliometric analysis. J Bus Res 109:1–14

Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM (2021) How to conduct a bibliometric analysis: An overview and guidelines. J Bus Res 133:285–296

Duan W, Jiang G (2008) A review of the theory of planned behavior. Adv Psychol Sci 16(2):315–320

Google Scholar  

Durach CF, Kembro J, Wieland A (2017) A new paradigm for systematic literature reviews in supply chain management. J Supply Chain Manage 53(4):67–85

Fan D, Breslin D, Callahan JL, Szatt-White M (2022) Advancing literature review methodology through rigour, generativity, scope and transparency. Int J Manage Reviews 24(2):171–180

Ferreira MP, Reis NR, Miranda R (2015) Thirty years of entrepreneurship research published in top journals: Analysis of citations, co-citations and themes. J Global Entrepreneurship Res 5(1):1–22

Ghauri P, Strange R, Cooke FL (2021) Research on international business: The new realities. Int Bus Rev 30(2):101794

Gioia DA, Corley KG, Hamilton AL (2012) Seeking qualitative rigor in inductive research: Notes on the gioia methodology. Organizational Res Methods 16(1):15–31

Gonzalez-Loureiro M, Dabić M, Kiessling T (2015) Supply chain management as the key to a firm’s strategy in the global marketplace: Trends and research agenda. Int J Phys Distribution Logistics Manage 45(1/2):159–181. https://doi.org/10.1108/IJPDLM-05-2013-0124

Grewal D, Puccinelli N, Monroe KB (2018) Meta-analysis: Integrating accumulated knowledge. J Acad Mark Sci 46(1):9–30

Hansen C, Steinmetz H, Block J(2021) How to conduct a meta-analysis in eight steps: a practical guide.Management Review Quarterly,1–19

Korherr P, Kanbach DK (2021) Human-related capabilities in big data analytics: A taxonomy of human factors with impact on firm performance. RMS. https://doi.org/10.1007/s11846-021-00506-4 (in press)

Kraus S, Breier M, Dasí-Rodríguez S (2020) The art of crafting a systematic literature review in entrepreneurship research. Int Entrepreneurship Manage J 16(3):1023–1042

Kraus S, Durst S, Ferreira J, Veiga P, Kailer N, Weinmann A (2022) Digital transformation in business and management research: An overview of the current status quo. Int J Inf Manag 63:102466

Kraus S, Jones P, Kailer N, Weinmann A, Chaparro-Banegas N, Roig-Tierno N (2021) Digital transformation: An overview of the current state of the art of research. Sage Open 11(3):1–15

Kraus S, Mahto RV, Walsh ST (2021) The importance of literature reviews in small business and entrepreneurship research. J Small Bus Manage. https://doi.org/10.1080/00472778.2021.1955128 (in press)

Kumar S, Sahoo S, Lim WM, Dana LP (2022a) Religion as a social shaping force in entrepreneurship and business: Insights from a technology-empowered systematic literature review. Technol Forecast Soc Chang 175:121393

Kumar S, Sahoo S, Lim WM, Kraus S, Bamel U (2022b) Fuzzy-set qualitative comparative analysis (fsQCA) in business and management research: A contemporary overview. Technol Forecast Soc Chang 178:121599

Kumar S, Sharma D, Rao S, Lim WM, Mangla SK (2022c) Past, present, and future of sustainable finance: Insights from big data analytics through machine learning of scholarly research. Ann Oper Res. https://doi.org/10.1007/s10479-021-04410-8 (in press)

Laher S, Hassem T (2020) Doing systematic reviews in psychology. South Afr J Psychol 50(4):450–468

Leemann N, Kanbach DK (2022) Toward a taxonomy of dynamic capabilities – a systematic literature review. Manage Res Rev 45(4):486–501

Lahiri S, Mukherjee D, Peng MW (2020) Behind the internationalization of family SMEs: A strategy tripod synthesis. Glob Strategy J 10(4):813–838

Lim WM (2018) Demystifying neuromarketing. J Bus Res 91:205–220

Lim WM (2020) The sharing economy: A marketing perspective. Australasian Mark J 28(3):4–13

Lim WM (2022) Ushering a new era of Global Business and Organizational Excellence: Taking a leaf out of recent trends in the new normal. Global Bus Organizational Excellence 41(5):5–13

Lim WM, Rasul T (2022) Customer engagement and social media: Revisiting the past to inform the future. J Bus Res 148:325–342

Lim WM, Weissmann MA (2021) Toward a theory of behavioral control. J Strategic Mark. https://doi.org/10.1080/0965254X.2021.1890190 (in press)

Lim WM, Kumar S, Ali F (2022a) Advancing knowledge through literature reviews: ‘What’, ‘why’, and ‘how to contribute’. Serv Ind J 42(7–8):481–513

Lim WM, Rasul T, Kumar S, Ala M (2022b) Past, present, and future of customer engagement. J Bus Res 140:439–458

Lim WM, Yap SF, Makkar M (2021) Home sharing in marketing and tourism at a tipping point: What do we know, how do we know, and where should we be heading? J Bus Res 122:534–566

López-Duarte C, González-Loureiro M, Vidal-Suárez MM, González-Díaz B (2016) International strategic alliances and national culture: Mapping the field and developing a research agenda. J World Bus 51(4):511–524

Mas-Tur A, Kraus S, Brandtner M, Ewert R, Kürsten W (2020) Advances in management research: A bibliometric overview of the Review of Managerial Science. RMS 14(5):933–958

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Reviews 4(1):1–9

Mukherjee D, Kumar S, Donthu N, Pandey N (2021) Research published in Management International Review from 2006 to 2020: A bibliometric analysis and future directions. Manage Int Rev 61:599–642

Mukherjee D, Kumar S, Mukherjee D, Goyal K (2022a) Mapping five decades of international business and management research on India: A bibliometric analysis and future directions. J Bus Res 145:864–891

Mukherjee D, Lim WM, Kumar S, Donthu N (2022b) Guidelines for advancing theory and practice through bibliometric research. J Bus Res 148:101–115

Obradović T, Vlačić B, Dabić M (2021) Open innovation in the manufacturing industry: A review and research agenda. Technovation 102:102221

Ojong N, Simba A, Dana LP (2021) Female entrepreneurship in Africa: A review, trends, and future research directions. J Bus Res 132:233–248

Palmatier RW, Houston MB, Hulland J (2018) Review articles: Purpose, process, and structure. J Acad Mark Sci 46(1):1–5

Post C, Sarala R, Gatrell C, Prescott JE (2020) Advancing theory with review articles. J Manage Stud 57(2):351–376

Pranckutė R (2021) Web of Science (WoS) and Scopus: The titans of bibliographic information in today’s academic world. Publications 9(1):12

R Core Team (2021) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/ Accessed 20th July 2022

Rovelli P, Ferasso M, De Massis A, Kraus S(2021) Thirty years of research in family business journals: Status quo and future directions.Journal of Family Business Strategy,100422

Salmony FU, Kanbach DK (2022) Personality trait differences across types of entrepreneurs: a systematic literature review. RMS 16:713–749

Snyder H (2019) Literature review as a research methodology: An overview and guidelines. J Bus Res 104:333–339

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222

Ulker-Demirel E, Ciftci G (2020) A systematic literature review of the theory of planned behavior in tourism, leisure and hospitality management research. J Hospitality Tourism Manage 43:209–219

Van Eck NJ, Waltma L (2014) CitNetExplorer: A new software tool for analyzing and visualizing citation networks. J Informetrics 8(4):802–823

Vlačić B, Corbo L, Silva e, Dabić M (2021) The evolving role of artificial intelligence in marketing: A review and research agenda. J Bus Res 128:187–203

Wong D (2018) VOSviewer. Tech Serv Q 35(2):219–220. https://doi.org/10.1080/07317131.2018.1425352

Download references

Open access funding provided by Libera Università di Bolzano within the CRUI-CARE Agreement.

Author information

Authors and affiliations.

Faculty of Economics & Management, Free University of Bozen-Bolzano, Bolzano, Italy

Sascha Kraus

Department of Business Management, University of Johannesburg, Johannesburg, South Africa

School of Business, Lappeenranta University of Technology, Lappeenranta, Finland

Matthias Breier

Sunway University Business School, Sunway University, Sunway City, Malaysia

Weng Marc Lim

Faculty of Economics and Business, University of Zagreb, Zagreb, Croatia

Marina Dabić

School of Economics and Business, University of Ljubljana, Ljubljana, Slovenia

Department of Management Studies, Malaviya National Institute of Technology Jaipur, Jaipur, India

Satish Kumar

Faculty of Business, Design and Arts, Swinburne University of Technology, Kuching, Malaysia

Weng Marc Lim & Satish Kumar

Chair of Strategic Management and Digital Entrepreneurship, HHL Leipzig Graduate School of Management, Leipzig, Germany

Dominik Kanbach

School of Business, Woxsen University, Hyderabad, India

College of Business, The University of Akron, Akron, USA

Debmalya Mukherjee

Department of Engineering, University of Messina, Messina, Italy

Vincenzo Corvello

Department of Finance, Santiago de Compostela University, Santiago de Compostela, Spain

Juan Piñeiro-Chousa

Rowan University, Rohrer College of Business, Glassboro, NJ, USA

Eric Liguori

School of Engineering Design, Universidad Politécnica de Valencia, Valencia, Spain

Daniel Palacios-Marqués

Department of Management and Quantitative Studies, Parthenope University, Naples, Italy

Francesco Schiavone

Paris School of Business, Paris, France

Department of Management, University of Turin, Turin, Italy

Alberto Ferraris

Department of Management and Economics & NECE Research Unit in Business Sciences, University of Beira Interior, Covilha, Portugal

Cristina Fernandes & João J. Ferreira

Centre for Corporate Entrepreneurship and Innovation, Loughborough University, Loughborough, UK

Cristina Fernandes

Laboratory for International and Regional Economics, Graduate School of Economics and Management, Ural Federal University, Yekaterinburg, Russia

School of Business, Law and Entrepreneurship, Swinburne University of Technology, Hawthorn, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sascha Kraus .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Kraus, S., Breier, M., Lim, W.M. et al. Literature reviews as independent studies: guidelines for academic practice. Rev Manag Sci 16 , 2577–2595 (2022). https://doi.org/10.1007/s11846-022-00588-8

Download citation

Received : 15 August 2022

Accepted : 07 September 2022

Published : 14 October 2022

Issue Date : November 2022

DOI : https://doi.org/10.1007/s11846-022-00588-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Literature reviews
  • Bibliometrics
  • Meta Analysis
  • Contributions

JEL classification

  • Find a journal
  • Publish with us
  • Track your research

The University of Edinburgh home

  • Schools & departments

Institute for Academic Development

Literature review

A general guide on how to conduct and write a literature review.

Please check course or programme information and materials provided by teaching staff , including your project supervisor, for subject-specific guidance.

What is a literature review?

A literature review is a piece of academic writing demonstrating knowledge and understanding of the academic literature on a specific topic placed in context.  A literature review also includes a critical evaluation of the material; this is why it is called a literature review rather than a literature report. It is a process of reviewing the literature, as well as a form of writing.

To illustrate the difference between reporting and reviewing, think about television or film review articles.  These articles include content such as a brief synopsis or the key points of the film or programme plus the critic’s own evaluation.  Similarly the two main objectives of a literature review are firstly the content covering existing research, theories and evidence, and secondly your own critical evaluation and discussion of this content. 

Usually a literature review forms a section or part of a dissertation, research project or long essay.  However, it can also be set and assessed as a standalone piece of work.

What is the purpose of a literature review?

…your task is to build an argument, not a library. Rudestam, K.E. and Newton, R.R. (1992) Surviving your dissertation: A comprehensive guide to content and process. California: Sage, p49.

In a larger piece of written work, such as a dissertation or project, a literature review is usually one of the first tasks carried out after deciding on a topic.  Reading combined with critical analysis can help to refine a topic and frame research questions.  Conducting a literature review establishes your familiarity with and understanding of current research in a particular field before carrying out a new investigation.  After doing a literature review, you should know what research has already been done and be able to identify what is unknown within your topic.

When doing and writing a literature review, it is good practice to:

  • summarise and analyse previous research and theories;
  • identify areas of controversy and contested claims;
  • highlight any gaps that may exist in research to date.

Conducting a literature review

Focusing on different aspects of your literature review can be useful to help plan, develop, refine and write it.  You can use and adapt the prompt questions in our worksheet below at different points in the process of researching and writing your review.  These are suggestions to get you thinking and writing.

Developing and refining your literature review (pdf)

Developing and refining your literature review (Word)

Developing and refining your literature review (Word rtf)

Writing a literature review has a lot in common with other assignment tasks.  There is advice on our other pages about thinking critically, reading strategies and academic writing.  Our literature review top tips suggest some specific things you can do to help you submit a successful review.

Literature review top tips (pdf)

Literature review top tips (Word rtf)

Our reading page includes strategies and advice on using books and articles and a notes record sheet grid you can use.

Reading at university

The Academic writing page suggests ways to organise and structure information from a range of sources and how you can develop your argument as you read and write.

Academic writing

The Critical thinking page has advice on how to be a more critical researcher and a form you can use to help you think and break down the stages of developing your argument.

Critical thinking

As with other forms of academic writing, your literature review needs to demonstrate good academic practice by following the Code of Student Conduct and acknowledging the work of others through citing and referencing your sources.  

Good academic practice

As with any writing task, you will need to review, edit and rewrite sections of your literature review.  The Editing and proofreading page includes tips on how to do this and strategies for standing back and thinking about your structure and checking the flow of your argument.

Editing and proofreading

Guidance on literature searching from the University Library

The Academic Support Librarians have developed LibSmart I and II, Learn courses to help you develop and enhance your digital research skills and capabilities; from getting started with the Library to managing data for your dissertation.

Searching using the library’s DiscoverEd tool: DiscoverEd

Finding resources in your subject: Subject guides

The Academic Support Librarians also provide one-to-one appointments to help you develop your research strategies.

1 to 1 support for literature searching and systematic reviews

Advice to help you optimise use of Google Scholar, Google Books and Google for your research and study: Using Google

Managing and curating your references

A referencing management tool can help you to collect and organise and your source material to produce a bibliography or reference list. 

Referencing and reference management

Information Services provide access to Cite them right online which is a guide to the main referencing systems and tells you how to reference just about any source (EASE log-in may be required).

Cite them right

Published study guides

There are a number of scholarship skills books and guides available which can help with writing a literature review.  Our Resource List of study skills guides includes sections on Referencing, Dissertation and project writing and Literature reviews.

Study skills guides

Next generation now

  • Study resources
  • Calendar - Graduate
  • Calendar - Undergraduate
  • Class schedules
  • Class cancellations
  • Course registration
  • Important academic dates
  • More academic resources
  • Campus services
  • IT services
  • Job opportunities
  • Mental health support
  • Student Service Centre (Birks)
  • Calendar of events
  • Latest news
  • Media Relations
  • Faculties, Schools & Colleges
  • Arts and Science
  • Gina Cody School of Engineering and Computer Science
  • John Molson School of Business
  • School of Graduate Studies
  • All Schools, Colleges & Departments
  • Directories

Concordia University logo

  • My Library Account (Sofia) View checkouts, fees, place requests and more
  • Interlibrary Loans Request books from external libraries
  • Zotero Manage your citations and create bibliographies
  • E-journals via BrowZine Browse & read journals through a friendly interface
  • Article/Chapter Scan & Deliver Request a PDF of an article/chapter we have in our physical collection
  • Course Reserves Online course readings
  • Spectrum Deposit a thesis or article
  • WebPrint Upload documents to print with DPrint
  • Sofia Discovery tool
  • Databases by subject
  • Course Reserves
  • E-journals via BrowZine
  • E-journals via Sofia
  • Article/Chapter Scan & Deliver
  • Intercampus Delivery of Bound Periodicals/Microforms
  • Interlibrary Loans
  • Spectrum Research Repository
  • Special Collections
  • Additional resources & services
  • Loans & Returns (Circulation)
  • Subject & course guides
  • Open Educational Resources Guide
  • General guides for users
  • Evaluating...
  • Ask a librarian
  • Research Skills Tutorial
  • Quick Things for Digital Knowledge
  • Bibliometrics & research impact guide
  • Concordia University Press
  • Copyright Guide
  • Copyright Guide for Thesis Preparation
  • Digital Scholarship
  • Digital Preservation
  • Open Access
  • ORCID at Concordia
  • Research data management guide
  • Scholarship of Teaching & Learning
  • Systematic Reviews
  • How to get published speaker series
  • Borrow (laptops, tablets, equipment)
  • Connect (netname, Wi-Fi, guest accounts)
  • Desktop computers, software & availability maps
  • Group study, presentation practice & classrooms
  • Printers, copiers & scanners
  • Technology Sandbox
  • Visualization Studio
  • Webster Library
  • Vanier Library
  • Grey Nuns Reading Room
  • Book a group study room/scanner
  • Study spaces
  • Floor plans
  • Room booking for academic events
  • Exhibitions
  • Librarians & staff
  • University Librarian
  • Memberships & collaborations
  • Indigenous Student Librarian program
  • Wikipedian in residence
  • Researcher-in-Residence
  • Feedback & improvement
  • Annual reports & fast facts
  • Annual Plan
  • Library Services Fund
  • Giving to the Library
  • Webster Transformation blog
  • Policies & Code of Conduct

The Campaign for Concordia

Library Research Skills Tutorial

Log into...

  • My Library account (Sofia)
  • Interlibrary loans
  • Article/chapter scan
  • Course reserves

Quick links

How to write a literature review

What is a literature review.

The literature review is a written overview of major writings and other sources on a selected topic. Sources covered in the review may include scholarly journal articles, books, government reports, Web sites, etc. The literature review provides a description, summary and evaluation of each source. It is usually presented as a distinct section of a graduate thesis or dissertation.

Back to top

Purpose of the literature review

The purpose of the literature review is to provide a critical written account of the current state of research on a selected topic:

  • Identifies areas of prior scholarship
  • Places each source in the context of its contribution to the understanding of the specific issue, area of research, or theory under review.
  • Describes the relationship of each source to the others that you have selected
  • Identifies new ways to interpret, and shed light on any gaps in, previous research
  • Points the way forward for further research.

Components of the literature review

The literature review should include the following:

  • Objective of the literature review
  • Overview of the subject under consideration.
  • particular position, those opposed, and those offering completely different arguments.
  • Discussion of both the distinctiveness of each source and its similarities with the others.

Steps in the literature review process

Preparation of a literature review may be divided into four steps:

  • Define your subject and the scope of the review.
  • Search the library catalogue, subject specific databases and other search tools to find sources that are relevant to your topic.
  • Read and evaluate the sources and to determine their suitability to the understanding of topic at hand (see the Evaluating sources section).
  • Analyse, interpret and discuss the findings and conclusions of the sources you selected.

Evaluating sources

In assessing each source, consideration should be given to:

  • What is the author's expertise in this particular field of study (credentials)?
  • Are the author's arguments supported by empirical evidence (e.g. quantitative/qualitative studies)?
  • Is the author's perspective too biased in one direction or are opposing studies and viewpoints also considered?
  • Does the selected source contribute to a more profound understanding of the subject?

Examples of a published literature review

Literature reviews are often published as scholarly articles, books, and reports. Here is an example of a recent literature review published as a scholarly journal article:

Ledesma, M. C., & Calderón, D. (2015). Critical race theory in education: A review of past literature and a look to the future. Qualitative Inquiry, 21(3), 206-222. Link to the article

Additional sources on writing literature reviews

Further information on the literature review process may be found below:

  • Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review
  • Fink, A. (2010). Conducting research literature reviews: From the Internet to paper
  • Galvin, J. (2006). Writing literature reviews: A guide for students of the social and behavioral sciences
  • Machi, L. A., & McEvoy, B. T. (2012). The literature review: Six steps to success

Adapted with permission and thanks from How to Write a Literature Review originally created by Kenneth Lyons, McHenry Library, University of California, Santa Cruz.

arrow up, go to top of page

Literature Reviews

  • Tools & Visualizations
  • Literature Review Examples
  • Videos, Books & Links

Business & Econ Librarian

Profile Photo

Click to Chat with a Librarian

Text: (571) 248-7542

What is a literature review?

A literature review discusses published information in a particular subject area. Often part of the introduction to an essay, research report or thesis, the literature review is literally a "re" view or "look again" at what has already been written about the topic, wherein the author analyzes a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles. Literature reviews provide the reader with a bibliographic history of the scholarly research in any given field of study. As such,  as new information becomes available, literature reviews grow in length or become focused on one specific aspect of the topic.

A literature review can be just a simple summary of the sources, but usually contains an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, whereas a synthesis is a re-organization, or a reshuffling, of that information. The literature review might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. Depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

A literature review is NOT:

  • An annotated bibliography – a list of citations to books, articles and documents that includes a brief description and evaluation for each citation. The annotations inform the reader of the relevance, accuracy and quality of the sources cited.
  • A literary review – a critical discussion of the merits and weaknesses of a literary work.
  • A book review – a critical discussion of the merits and weaknesses of a particular book.
  • Teaching Information Literacy Reframed: 50+ Framework-Based Exercises for Creating Information-Literate Learners
  • The UNC Writing Center – Literature Reviews
  • The UW-Madison Writing Center: The Writer’s Handbook – Academic and Professional Writing – Learn How to Write a Literature Review

What is the difference between a literature review and a research paper?

The focus of a literature review is to summarize and synthesize the arguments and ideas of others without adding new contributions, whereas academic research papers present and develop new arguments that build upon the previously available body of literature.

How do I write a literature review?

There are many resources that offer step-by-step guidance for writing a literature review, and you can find some of them under Other Resources in the menu to the left. Writing the Literature Review: A Practical Guide suggests these steps:

  • Chose a review topic and develop a research question
  • Locate and organize research sources
  • Select, analyze and annotate sources
  • Evaluate research articles and other documents
  • Structure and organize the literature review
  • Develop arguments and supporting claims
  • Synthesize and interpret the literature
  • Put it all together

Cover Art

What is the purpose of writing a literature review?

Literature reviews serve as a guide to a particular topic: professionals can use literature reviews to keep current on their field; scholars can determine credibility of the writer in his or her field by analyzing the literature review.

As a writer, you will use the literature review to:

  • See what has, and what has not, been investigated about your topic
  • Identify data sources that other researches have used
  • Learn how others in the field have defined and measured key concepts
  • Establish context, or background, for the argument explored in the rest of a paper
  • Explain what the strengths and weaknesses of that knowledge and ideas might be
  • Contribute to the field by moving research forward
  • To keep the writer/reader up to date with current developments in a particular field of study
  • Develop alternative research projects
  • Put your work in perspective
  • Demonstrate your understanding and your ability to critically evaluate research in the field
  • Provide evidence that may support your own findings
  • Next: Tools & Visualizations >>
  • Last Updated: Sep 7, 2023 8:35 AM
  • URL: https://subjectguides.library.american.edu/literaturereview

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • PLoS Comput Biol
  • v.9(7); 2013 Jul

Logo of ploscomp

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

  • interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
  • an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
  • a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

  • keep track of the search items you use (so that your search can be replicated [10] ),
  • keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
  • use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
  • define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
  • do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

An external file that holds a picture, illustration, etc.
Object name is pcbi.1003149.g001.jpg

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

  • discussing in your review the approaches, limitations, and conclusions of past reviews,
  • trying to find a new angle that has not been covered adequately in the previous reviews, and
  • incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

  • be thorough,
  • use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
  • look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

  • the major achievements in the reviewed field,
  • the main areas of debate, and
  • the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

  • Open access
  • Published: 20 February 2024

The burden, risk factors and prevention strategies for drowning in Türkiye: a systematic literature review

  • Ali Işın   ORCID: orcid.org/0000-0003-4666-2117 1 &
  • Amy E. Peden 2 , 3  

BMC Public Health volume  24 , Article number:  528 ( 2024 ) Cite this article

Metrics details

Introduction

Drowning is a public health problem in Türkiye, as in the rest of the world. This study aims to systematically review the literature on drowning in Türkiye with a focus on data sources, epidemiology, risk factors and prevention strategies. Methods: Literature searches were conducted using PubMed, SPORTSDiscus, Scopus, Web of Science, Turk MEDLINE, Google Scholar and Google Akademik (Turkish language). Studies (limited to original research written in English and Turkish) reporting drowning (unintentional and intentional; fatal and non-fatal) of residents and tourists in Türkiye were independently dual screened at the title and abstract and full text stages. Study quality was assessed using JBI checklists and evidence level assessed based on study design. Results: From a total of 917 studies, 49 met the inclusion criteria. Most (51%) focused on unintentional fatal drowning. Included studies were most commonly analytical cross-sectional studies ( n  = 23) and case series ( n  = 20) meaning the evidence level was low or very low for 48 (98%) studies. Fifteen studies examined drowning at the national level, while sub-national studies ( n  = 30) focused on urban areas across three provinces: Antalya ( n  = 6), Istanbul ( n  = 6), Izmir ( n  = 4). There was little consensus on risk factors beyond male drowning risk, and no data reported on implemented or evaluated drowning prevention interventions. Discussion: There is a need for more national-level studies to identify the causes of drowning and to guide intervention implementation and evaluation to inform policy makers and donors. Currently official data is limited in its detail, providing age and gender data only, hampering efforts to identify, and thus address, causal factors for drowning. Practical applications: There is currently very little evidence to inform investment in effective drowning prevention interventions in Türkiye. To improve this, data collection systems on drowning in Türkiye need to be strengthened via the development a national drowning registry.

Trial Registration

#CRD42022382615.

Peer Review reports

Drowning is recognised as a serious public health problem worldwide. In 2019, more than 230,000 people died due to drowning, mostly in low- and middle-income countries, making drowning the third leading cause of unintentional injury death globally (accounting for 7% of all injury-related deaths) [ 1 ]. Studies from several countries identify that such figures likely underreport the true burden of drowning due to the exclusion of water transport and disaster-related drowning [ 2 , 3 , 4 ], as well as intentional drowning [ 5 ]. Drowning can occur in any type of water, such as rivers, lakes, oceans, pools, bathtubs or buckets, and can be classified as fatal or non-fatal depending on whether the outcome of the initial drowning incident [ 6 ].

Türkiye, a Eurasian country with 783,577 km2 of land, is surrounded by four seas (the Mediterranean, the Aegean, the Black Sea, and the Marmara) and has many lakes, streams and rivers [ 7 ]. The country’s total coastline is 8,592 km long and the area of the coastal provinces accounts for 30% of the whole country. Türkiye’s most populous provinces are generally along the coast [ 8 ]. This gives more people access to the sea, thus increasing drowning risk. Moreover, with the rising temperatures in the summer months, more people participate in aquatic activities such as swimming, boating, etc. This leads to fatal and non-fatal drowning incidents in Türkiye [ 9 ]. Although there are lifeguards on all major beaches, people may choose to enter the water in more rural areas. Also, in rural areas, irrigation canals, lakes, dams, rivers and streams are seen as significant risk factors for drowning. It is thought that the number of drownings increases in these areas due to the lack of protective measures (such as warning signs and rescue equipment) [ 10 ].

Drowning is a significant issue across the European region [ 11 ], including in Türkiye, where the prevention of drowning is challenging due to a lack of reliable and comprehensive data on its burden and risk factors [ 7 ]. The number of drowning deaths and crude mortality rate in Türkiye is uncertain due to different data sources (media data, clinic reports and autopsy records) which use different definitions thus affect the accuracy of estimates of drowning mortality in Türkiye. Further, the exclusion of flood-related drowning deaths and water transportation-related drownings [ 10 ] also risks underrepresenting the true burden. Further, there is no total population level data capture on non-fatal drowning in Türkiye. Therefore, more comprehensive and consistent data on drowning in Türkiye are needed to inform prevention strategies and policies [ 7 ].

Given of the lack of consolidated information on drowning in Türkiye, this systematic literature review aimed to identify and synthesise the published literature on drowning burden, data sources, risk factors and prevention strategies in Türkiye, with the aim of informing next steps for drowning prevention in the country.

Materials and methods

The protocol for this systematic review was prospectively registered with PROSPERO (#CRD42022382615) and conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [ 12 ].

Literature search

Searches were conducted using PubMed, SPORTSDiscus, Scopus, Web of Science, and Turk MEDLINE from inception to 9th December 2022. The inclusion and exclusion criteria are presented in Table  1 . Search terms included drown*, immers*, submers*, swim*, a variety of aquatic locations (i.e., river, lake, sea, beach, pool) and Turk*. Full search strategies can be found in Table S 1 . These were tailored to suit each journal and based on consultations with a specialist librarian and a previous literature review of drowning [ 13 ]. Search strings were also devised in such a way as to capture more relevant information, for example, cases classified as drowning not just deaths or incidents in water, and swimming as it pertains to drowning prevention and not competitive swimming or the biomechanics of swimming.

After database searches were run, additional searches of boğulma* AND Türk were run using Google Akademik (Turkish language by author AI) and drown* AND Turk* in Google Scholar (English language by author AP) to identify any articles not found via database searches. Authors screened results until 10 pages of nil results. As a result of these searches, no new articles were identified. Databases were chosen based on their relevance to drowning from a previous review of drowning in a neighbouring region [ 13 ], in addition to the use of Turk MEDLINE and Google Akademik to capture Turkish literature not indexed in the other databases.

Study selection

Two authors (AI and AP) conducted a dual independent review of the title and abstract followed by full-text screening with conflicts resolved via consensus. One Turkish-speaking author (AI) reviewed Turkish language literature, clarifying any concerns with author (AP). Study screening was performed using Covidence literature screening software [ 14 ].

Data extraction

Data extraction was undertaken by one author (AI) with an independent quality check of 20% of included records undertaken by a second author (AP). Data extraction was undertaken using a Microsoft Excel Spreadsheet custom-built for this purpose. Data were extracted on the following aspects: Study characteristics (which included author name, year published, years of study, study population, study design and data source(s)), epidemiology, risk factors, and prevention strategies.

The epidemiology of drowning was reported as numbers, proportions, or rates per 100,000 for each population reported (overall, by age group, by year, by gender, etc.) in the included studies. No inferred rates were calculated. Drowning was described by outcome (fatal, non-fatal, both, not specified), and intent (unintentional, intentional, both, not specified) and examined at a total population level, as well as by age group and gender.

We coded the free text description of data sources, risk/protective factors, and prevention strategies by consensus. Risk/protective factors were those that had a significant association with the risk of drowning or drowning outcome (e.g., chi square tests of significance [ p  < 0.05], odds ratio, relative risk). We extracted prevention strategies that were proposed, implemented and/or evaluated. We classified prevention strategies as being primary (before the drowning occurs), secondary (reduce the impact of a drowning which has already occurred), or tertiary prevention (reduce the ongoing effects of a drowning incident) [ 15 ] and also aligned strategies to the Hierarchy of Control [ 16 ]. We also noted if the prevention strategy involved multi-sectoral action (as recommended by the WHO [ 17 ]) and which sectors were involved.

Quality appraisal

Quality assessment of included studies was performed by two members of the review team using the Joanna Briggs Institute (JBI) Critical Appraisal Tools based on study type. The first author (X1) assessed all articles and then the other author (X2) randomly assessed 20% of the articles. Disagreements between the two authors were resolved by discussion. Checklists provide a score based on assessment of a range of study design criteria. Study design of the included studies were graded according to the National Health and Medical Research Council's (NHMRC) levels of evidence, which range from level I (a systematic review of Level II studies [randomised controlled trial]) to level IV (case studies with either post-test or pre-test/post-test outcomes) (Table S 2 ).

Database searching yielded 917 studies. After removal of 79 duplicates, 838 studies were screened by title and abstract for inclusion. Of these, 735 studies were deemed irrelevant and excluded. The remaining 103 full text studies were screened for eligibility. In total, 54 studies were removed at full text review and data were extracted from the remaining 49 studies which satisfied the inclusion criteria (Fig.  1 ).

figure 1

PRISMA flow chart

Study characteristics

Among the 49 included studies, the publication dates ranged from 2004 to 2022. The included studies were predominately analytical cross-sectional studies ( n  = 23; 47%) and case series ( n  = 20; 41%). The remaining studies comprised four prevalence studies, one qualitative study and one quasi-experimental study. Included studies mostly used autopsy data ( n  = 21; 43%) or medical reports ( n  = 13; 27%), followed by media reports ( n  = 8; 16%). Based on study design, the overall level of evidence was low, with almost all studies ( n  = 48) ranked as low or very low on the NHMRC Levels of Evidence criteria (Table S 2 ). When assessed using JBI checklists based on study type, 23 studies (47%) recorded a score of 7 or above.

Fifteen included studies reported drowning at the national level, while 16 studies reported drowning at the provincial level, most commonly in Antalya ( n  = 6), Istanbul ( n  = 6) and Izmir ( n  = 4) (Fig.  2 ). More than half of the included studies reported data at the sub-national level ( n  = 30; 61%), followed by 16 studies (33%) reporting national data and 3 studies (6%) reporting on foreign visitors to Türkiye. No studies examined drowning among migrants, either once they had arrived in Türkiye or while in transit. Most of the studies (27 out of 49; 55%) reported data from urban areas, while two studies (4%) reported data from rural areas. Some studies (20 out of 49; 41%) reported data from both urban and rural areas. Fatal drownings were the focus of 36 studies, while both fatal and non-fatal drownings were included in 12 studies. While 18 of these studies examined unintentional drownings only, seven examined both intentional and unintentional drownings. The remaining 24 studies did not distinguish between intentional and unintentional drownings (Table  2 ).

figure 2

Heatmap of sub-national studies by location

Burden and risk factors

One of the more commonly reported risk factors for drowning in Türkiye was gender [ 7 , 10 , 27 , 31 , 34 ]. Three studies presented drowning mortality rates per 100,000 people [ 7 , 9 , 34 ]. In these studies, the drowning rates for males were 1.8 (between 2005 and 2017), 0.52 (2015–2019) and 1.44 (2007–2011), respectively, while the corresponding rates for females were 0.48, 0.06 and 0.28 (Table  3 ). Only one study reported a higher proportion of females drowning (60%) than males (40%), though case numbers were small [ 47 ].

Studies showed different rates in different age groups, with different data sources, and focusing on different regions. However, the general trend was that about 70% of drowning cases were male. Işın et al. (2020) reported that the drowning rate for children under 18 years of age was 1.18 per 100,000 for males and 0.48 for females. It also showed that the risk of fatal drowning was almost four times higher for males (relative risk: 3.98) than females [ 7 ].

Few and varied mortality rates were reported in the included studies because of differences in data sets and populations. Turgut and Turgut (2014) found that drowning rate of 0.89 per 100,000 people in Türkiye based on media reports [ 9 ]. A similar study by Işın et al. (2020) found a rate of 1.17 per 100,000 children aged 0–18 years [ 7 ]. Çaylan et al. [ 27 ] found that the rate in children under 5 years of age decreased from 1.1 per 100,000 population in 2014 to 0.7 in 2017. In another study also conducted in children [ 7 ], it was reported that the rate of drowning, which was in an upward trend from 2005 to 2010, decreased every year until 2017 to 0.78 per 100,000 children after peaking in 2010. In a study conducted on the whole population [ 10 ], it was found that the drowning rate has been on a downward trend every year since 2015 (1.24 per 100,000 people) and decreased to 0.64 in 2019 (Table  3 ).

Out of 16 studies that reported number of deaths in age groups, only 10 presented data for the 0–19 years age group. The total number of deaths reported varied from 1 to 1,086. There was no consensus on the age group with the highest burden of drowning; a population-based study showed the 65 + years age group as recording the highest number of drowning cases [ 10 ], while a population-based drowning study showed high drowning numbers in the 10–14 years age group [ 7 ] (Table  4 ). A study focusing on child drowning found that the drowning rate per 100,000 children by age group varied from a low of 0.73 for 0–4 year-olds, increasing with age to a high of 2.11 for adolescents aged 15–17 years [ 7 ]. According to a study of rescue-related drowning, the age group with the highest risk of drowning per 100 000 persons was 15–24 years (1.28), followed by 25–34 years with 0.78 [ 34 ].

Fatal drownings by water location were reported in 16 studies. In Türkiye, the most common environments where fatal drownings occur were Beach/Sea, Stream/River/Creek, and Irrigation channel, respectively. The Beach/Sea was the most common drowning location in 5 studies, followed by Stream/River/Creek in 4 studies and lake in 2 studies. Bucket, Irrigation Channel, Hole/Well and Pool were each the most frequent drowning location in 1 study (Table  5 ). The sea/beach was the most common place for drowning across all age groups, but buckets were the main cause of drowning for children aged 0–4 years, while streams/rivers/creeks and irrigation channels were more prevalent for older children. Among rescuers, lakes/ponds and rivers were frequent drowning sites (Table  6 ).

Beyond gender, age and water location, several other drowning risk and protective factors were identified in the included literature. Results differed with respect to season, with winter found to have statistically significant lower drowning risk than Summer [ 27 ], while in Summer drowning rescues were more likely to be successful when compared to other seasons [ 34 ]. Among fatal and non-fatal drowning of children < 18 years, receipt of CPR and Noninvasive ventilation (NIV) treatment were associated with survival to hospital admission and a shorter stay in hospital respectively, whereas poorer vital signs led to poorer outcomes [ 44 ] (Table  6 ).

Prevention strategies

Identified prevention strategies included supervision for children aged ≤ 18 years, first aid education, data/research, rescue skills education (including throw rescues) and training, and swimming education. All were proposed strategies with no implementation or evaluation reported. All were classified as administrative on the Hierarchy of Control, representing lower order strategies in terms of likely effectiveness. Strategies were reasonably evenly spread across primary (four strategies), secondary (four strategies), and tertiary (three strategies) prevention. Most of the strategies (nine out of 11) involved more than one sector, with education and health being the most commonly co-occurring sectors (Table  7 ).

This study aimed to identify and synthesise the studies that have addressed drowning in Türkiye to date, examining data sources, epidemiology, risk factors and prevention strategies. Despite being a public health concern across Europe [ 11 ], our review identifies limited literature on drowning in Türkiye and low consensus on drowning risk factors. This lack of understanding on causal factors for drowning in Türkiye is thus manifest with no implementation or evaluation of drowning prevention strategies identified in the included literature [ 7 , 64 ].

Little is known about the crude drowning rate in Türkiye. The most important reason for this is that most of the conducted studies in Türkiye were based on autopsy or clinical/medical reports. Studies using these sources are not generalisable as they usually focus on a single centre (hospital, forensic medicine) or a province/region. This was insufficient data presented in many of the included studies to calculate mortality rates. In addition, many population-based studies used media reports of drowning as their source of data. While media reporting can be useful in the absence of routinely collected data, and in Türkiye supplements the meagre detail provided from the national statistics authority [ 10 ], it is not without its limitations. Previous research has indicated a bias towards more newsworthy incidents and incidents which occurred in urban settings [ 65 ]. Therefore, such data must be interpreted with caution and provides further support for the establishment of detailed and timely routine data collection on drowning such as via a national registry [ 66 ].

Where drowning mortality rates were reported, the rates among children were lower than those of neighbouring countries such as Iran, albeit with different data capture methods used [ 13 ].

Studies presenting crude drowning rates of different years and populations in Türkiye showed that drowning in Türkiye has been on a decreasing trend recently. Declining drowning rates in Türkiye appear to mirror those reported globally [ 67 ], as greater effort and funding is directed toward the issue [ 68 ], particularly investment in those interventions known to be effective in young children [ 69 ]. However, there is a need to expand this investment into the adolescent age group who experience high drowning rates with relatively lower investment [ 70 ]. Additionally, there is a need to ensure drowning fatalities across both urban and rural settings are captured [ 10 ], as well as better exploration of the impact of non-fatal drowning, particularly on the Turkish health system.

There was little consensus on risk factors for drowning in Türkiye, within the identified literature, aside from the consensus regarding male drowning risk being greater than female [ 7 , 10 , 34 ]. This is broadly consistent with many other studies globally [ 67 , 71 , 72 , 73 ].Based on the included studies, three possible reasons may account for the higher drowning rates among male in Türkiye; first, being males are more exposed to water than female. Thus, they spend more time in the water doing activities such as fishing, swimming, cooling off, boating, etc. [ 7 , 9 , 38 ]. Another reason could be that males are less likely to wear life jackets than female [ 7 ]. Finally, it is believed that male’s participation in the above activities under the influence of alcohol and drugs increases the risk of drowning in favour of male. Although the data didn’t meet the criteria to be included as a risk factor in our analysis, three studies suggested that alcohol consumption may be a preventable risk factor for drowning in Türkiye [ 37 , 46 , 59 ], particularly among males [ 37 ]. However, studies examining the impact of alcohol on drowning in Türkiye should consider the use of objective measures of alcohol consumption and intoxication such as recording blood alcohol concentration.

Another risk factor was age [ 7 , 10 , 27 , 34 ]. Most of the included studies focused on children and adolescents, but some also evaluated all age groups. The results of these studies showed that children, adolescents, and individuals over 65 years of age had a higher risk of drowning than other ages. Effective drowning prevention interventions for young children are well understood, comprising active supervision, restricting access to water, water familiarisation [ 7 ] and cardiopulmonary resuscitation (CPR) as a tertiary response [ 44 ]. It may be that greater public education on strategies to reduce child drowning risk are needed within Türkiye [ 51 ], though the baseline knowledge is not currently known. The results of a study on drowning in swimming pools in Türkiye highlight the need to provide safer environments to prevent drowning in swimming pools. It is stated that lack of adequate safety measures and supervision is the cause of a significant proportion of child drownings. It was concluded that there is a need to close the pool edges with safety fences and to raise awareness of public by hanging information and warning signs at the edge of the pool [ 64 ], although such legislation is yet to have been implemented. While such approaches are likely to reduce drowning among young children, globally there is little evidence regarding effective drowning prevention interventions for adolescents and older people [ 70 , 74 ].

Seasons were another risk factor for drownings in Türkiye. Fatal drowning cases increased in summer and decreased in winter [ 7 , 27 , 34 ]. People attended water environments for activities such as cooling off, swimming, fishing, boat trips, etc. [ 7 ]. Especially in the summer months, when the temperatures rose and the schools closed, people visited water environments such as sea, dam, lake, etc. more often [ 9 , 64 ]. This led to more drownings due to a lack of supervision [ 7 ], inability to swim [ 64 ], swimming in areas without lifeguards [ 49 ], etc. Therefore, the authors recommended swimming education as a prevention strategy [ 51 ]. In addition, we recommend increased public education and awareness campaigns regarding drowning risk reduction strategies ahead of high-risk periods such as summer and school holidays.

Aquatic location was also identified as a risk factor for drowning, although there was little consensus in the included studies. Türkiye has many natural water bodies including four seas, numerous lakes, dams, and rivers [ 7 ], leading to natural water bodies being a leading location for all-age drowning. Results of this review show that drownings in areas close to the coastline were mostly in the sea [ 19 , 24 , 31 ], while lake, rivers and irrigation canals were the main drowning places in landlocked or inland areas [ 58 ]. the findings of the current systematic review showed that most studies focused on drownings in a single province or region. This resulted in different locations being the leading sites for drowning cases/deaths for children and adults, based on their geographical location. Drowning prevention interventions in Türkiye must be tailored to the accessible water bodies and practices around interaction with water in the different localities.

Although still facing drowning risk from natural waterbodies where adults drown, such as the sea and rivers, creeks and streams, this review also highlight the drowning risk for children posed by buckets [ 18 ] and irrigation channels [ 7 , 28 ]. Updated research is needed to determine whether water storage practices have changed over time since Asurdizer et al.’s analysis of cases between 1996 and 2000 [ 18 ], including the potential role of water and sanitation hygiene advancements in changing child drowning risk profile. An absence of adult supervision combined with a lack of swimming ability contribute to drowning risk in irrigation channels. Therefore, parental education campaigns on supervision, as well as the provision of basic swimming and water safety education at the primary school level in Türkiye may assist in preventing future drowning incidents [ 50 ].

With respect to the ocean, Işın et al. [ 34 ] analysed the drownings of rescuers and found that rescues were more successful in the sea. The main reason for this is that seas are places where lifeguards are present and are visited by more people than other water environments. This increases the chance of rescue when more professionals intervene to save drowning people. Therefore, Işın et al. [ 34 ] suggested that rescue skill training and education would be an important prevention strategy, especially to prevent multiple drowning deaths.

Finally, lack of official data and limited data are considered as a barrier to the calculation of the burden of drowning in Türkiye [ 7 ]. Failure or limited determination of the burden of drowning and its underlying causes delays the planning of drowning prevention strategies. Previous studies in Türkiye have reported inadequate official records on drowning [ 7 , 9 , 34 , 64 ]. Due to the limited availability of official sources, most studies on drowning in Türkiye have obtained data either from autopsy reports [ 19 , 59 ] or medical reports (patient information form, electronic medical records, medical charts, and nursing records, etc.) [ 24 , 43 , 44 ]. However, such studies investigated drowning by analysing patient records or autopsy data from a region, a province, or one or more hospitals. Therefore, these studies were not successful in providing generalisable data on the total population burden of drowning in Türkiye due to their relatively small samples. Due to the lack of official records or limited information available to analyse the burden of drowning, researchers have analysed drowning in Türkiye from cases obtained from media reports [ 7 , 9 , 34 , 64 ]. Although this type of research has some reported limitations, it has provided important findings because of its generalisable conclusions and its contribution to revealing the main gaps in Türkiye to prevent drowning. Official mortality data, triangulated with police and media reports, are needed to identify causal factors to inform, and in future evaluate, risk reduction initiatives. Although Işın and Peden (2022) obtained data from TurkSTAT, which use the death notification system, only gender and age group were analysed in the study because TurkSTAT provides very limited information [ 10 ]. While this contributes to the epidemiology of drowning in Türkiye, it is insufficient to formulate prevention strategies. As has been proposed in other European countries, a National Drowning Registry needs to be developed in Türkiye in order to collect drowning data efficiently [ 66 ]. The adoption of a non-fatal drowning definition that is consistently applied to capture non-fatal drowning cases in Türkiye in this registry would also be advisable.

Strengths and limitations

To the best of our knowledge, this is the first systematic review of the published literature on drowning in Türkiye in terms of data sources, epidemiology, risk factors and prevention strategies. It is bolstered by examining publications in both English and Turkish language, as well as exploring publications from inception. However, it is not without its limitations. Turkish language studies could only be screened by one author due to the native language of the second author, which may have weakened the rigour around article screening and data extraction. Search strategies using different terms or combinations of terms, may have produced different results in terms of literature yielded. This review included primary studies published in peer-reviewed literature only. There may also be relevant information on the issue of drowning and its prevention within Türkiye published in the grey literature. The heterogenous nature of the studies made comparison difficult and a meta-analysis not possible.

This research has highlighted the need for more generalised studies to better understand and estimate the burden of drowning deaths in Türkiye. Most of the studies were autopsy-based and focused on specific regions, or cities, which limited their generalisability. Thus, the burden of drowning in Türkiye was mostly calculated with media reports, which had some limitations and biases. There is a need for more research to support greater consensus on risk factors, to inform prevention interventions. However, the lack of accurate and comprehensive data remains a significant barrier to advancing drowning prevention efforts in Türkiye. We recommend the establishment of a national drowning data registry to capture fatal drowning incidents, before considering the inclusion of non-fatal drowning events. The consistent collection and timely analysis of such data are vital to saving lives from drowning in Türkiye.

Availability of data and materials

All data generated or analysed during this study are included in this published article (and its Supplementary information files).

World Health Organization. Drowning. 2021 [cited 2023 January 27, 2023]; Available from: https://www.who.int/news-room/fact-sheets/detail/drowning .

Peden AE, et al. Using a retrospective cross-sectional study to analyse unintentional fatal drowning in Australia: ICD-10 coding-based methodologies verses actual deaths. BMJ Open. 2017;7(12): e019407.

Article   PubMed   PubMed Central   Google Scholar  

Lunetta P, Penttilä A, Sajantila A. Drowning in Finland:“external cause” and “injury” codes. Inj Prev. 2002;8(4):342–4.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Passmore J, Ozanne-Smith J, Clapperton A. True burden of drowning: compiling data to meet the new definition. Int J Inj Contr Saf Promot. 2007;14(1):1–3.

Haw C, Hawton K. Suicide and self-harm by drowning: a review of the literature. Arch Suicide Res. 2016;20(2):95–112.

Article   PubMed   Google Scholar  

Peden AE, et al. Understanding the full burden of drowning: a retrospective, cross-sectional analysis of fatal and non-fatal drowning in Australia. BMJ Open. 2018;8(11):e024868.

Işın A, Akdağ E, Turgut A. The epidemiology of fatal drowning in children: a 13-year retrospective study in Turkey. Int J Inj Contr Saf Promot. 2020;27(4):465–71.

Uzun B, Celik N. Sustainable management of coastal lands: A new approach for Turkish Coasts. Ocean Coast Manag. 2014;95:53–62.

Article   Google Scholar  

Turgut A, Turgut T. A population-based study on deaths by drowning incidents in Turkey. Int J Inj Contr Saf Promot. 2014;21(1):61–7.

Işın A, Peden AE. Assessing variations in estimates of drowning mortality in Turkey from 2013 to 2019. Archives of public health. 2022;80(1):178.

Peden AE, et al. Closing the gap for drowning prevention across Europe. Lancet Public Health. 2022;7(9):e728–9.

Moher D, et al. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. BMJ. 2009;339:2535.

Peden AE, Işın A. Drowning in the Eastern Mediterranean region: a systematic literature review of the epidemiology, risk factors and strategies for prevention. BMC Public Health. 2022;22(1):1477.

Veritas Health Innovation. Covidence systematic review software. 2021 19–09–2021]; Available from: www.covidence.org .

Pless IB, Hagel B. Injury prevention: a glossary of terms. J Epidemiol Community Health. 2005;59(3):182.

National Safety Council. The Hierarchy of Controls. 2019 03–02–2019]; Available from: https://www.safetyandhealthmagazine.com/articles/16790-the-hierarchy-of-controls .

World Health Organization, Preventing drowning: an implementation guide. Geneva: World Health Organization; 2017.

Aşırdizer M, et al. Infant and adolescent deaths in Istanbul due to home accidents. Turk J Pediatr. 2005;47:141–9.

PubMed   Google Scholar  

Atilgan M, et al. Deadly stays: A 10-year autopsy study of deaths in hotels in Antalya. Turkey Journal of forensic sciences. 2022;67(3):1116–23.

Azmak D. Asphyxial deaths: a retrospective study and review of the literature. Am J Forensic Med Pathol. 2006;27(2):134–44.

Barlas B, Beji S. Rip current fatalities on the Black Sea beaches of Istanbul and effects of cultural aspects in shaping the incidents. Nat Hazards. 2016;80:811–21.

Başol N, Baydın A, Yardan T. A retrospective evaluation of patients admitted emergency department due to drowning. Journal of Experimental and Clinical Medicene. 2012;29:121–5.

Beydilli I, et al. Mortality across drowning in the view of the meteorological parameters: Relative humudity and sea wavelength. Biomed Res. 2017;28(1):61–5.

Google Scholar  

Çakmakcı S, et al. Therapeutic Approaches and Mortality in Acute Respiratory Failure due to Drowning. Turkish Thoracic Journal. 2021;22(6):477.

Canturk N, et al. Evaluation of the deaths due to drowning autopsied between 2003 and 2006 in Ankara. TURKIYE KLINIKLERI TIP BILIMLERI DERGISI. 2009;29(5):1198–205.

Canturk N, et al. Medico-legal child deaths in Istanbul: data from the Morgue Department. Pediatr Int. 2007;49(1):88–93.

Çaylan N, et al. Evaluation of injury-related under-five mortality in Turkey between 2014–2017. Turk J Pediatr. 2021;63(1):37–47.

Dirlik M, Bostancıoğlu B. Child drowning deaths in Aydin province, western Turkey, 2002–2012. Eur J Trauma Emerg Surg. 2015;41:683–8.

Article   CAS   PubMed   Google Scholar  

Dogan KH, et al. Dead bodies found in wells. Am J Forensic Med Pathol. 2010;31(3):208–12.

Esiyok B, Balci Y, Ozbay M. Bodies recovered from wells, sewerage systems and pits: what is the cause of death? Annals-Academy of Medicine Singapore. 2006;35(8):547.

Güzel A, et al. Drowning and near-drowning: experience of a university hospital in the Black Sea region. Turk J Pediatr. 2013;55(6):620.

Hsieh W-H, Wang C-H, Lu T-H. Drowning mortality by intent: a population-based cross-sectional study of 32 OECD countries, 2012–2014. BMJ Open. 2018;8(7):e021501.

Işik M, Eşitti Ş. Content analysis of printed news media related to drowning incidents in Turkey. Hum Ecol Risk Assess Int J. 2015;21(4):1050–61.

Işın A, Turgut A, Peden AE. Descriptive epidemiology of Rescue-Related fatal drowning in turkey. Int J Environ Res Public Health. 2021;18(12):6613.

Ketenci HÇ, et al. Drowning in submerged cars caused by traffic accidents. Turk J Trauma Emerg Surg/Ulusal Travma ve Acil Cerrahi Dergisi. 2022;28(8):1115–21.

Koca E, et al. Evaluation of fatal diving accidents in Turkey. Undersea Hyperb Med. 2019;45(6):633–8.

Lakadamyalı H, Doğan T. Türkiye’de bir turizm yöresinde suda boğulma olgularının irdelenmesi. Türkiye Klinikleri Tıp Bilimleri Dergisi. 2008;28(2):143–8.

Lapa TY, Turgut A, Turgut T. Deaths by drowning incidents during recreational boating and similar activities. World Appl Sci J. 2012;17(2):233–8.

Lin C-Y, et al. Unintentional drowning mortality, by age and body of water: an analysis of 60 countries. Inj Prev. 2015;21(e1):e43–50.

Mollaoğlu M, Bolayir E. Injuries in patients with epilepsy and some factors associated with injury. Nöro Psikiyatri Arşivi. 2013;50(3):269.

PubMed   PubMed Central   Google Scholar  

Orhan İ. Sinking or Swimming: The Need for Water Safety and Swimming Education in the 21st Century. International Journal of Applied Exercise Physiology. 2020;9(6):263–9.

Petrucci O, et al. Flood fatalities in Europe, 1980–2018: Variability, features, and lessons to learn. Water. 2019;11(8):1682.

Şık N, et al. Early application of non-invasive ventilation for children with pulmonary edema after drowning. Pediatr Int. 2022;64(1): e14858.

Şık N, et al. A reappraisal of childhood drowning in a pediatric emergency department. Am J Emerg Med. 2021;41:90–5.

Şimşek Y, Satar S, Suda Boğulma ve Acil Servis. Turk J Emerg Med. 2013;13(2):81–5.

Söyüncü S, et al. Acil servise suda boğulma veya boğulayazma nedeniyle başvuran hastaların prognozunun belirlenmesinde GKS ve REMS skorlarının değerliliği. Turkish Journal of Emergency Medicine. 2008;8(2):067–72.

Taskesen M, Pirinççioglu AG, Yaramis A. Drowning and near-drowning in children in the southeast of Turkey. Eurasian Journal of Emergency Medicine. 2015;14(1):16.

Tunçez FT, et al. Evaluation of autopsied refugee deaths in Izmir, Turkey. Med Sci Law. 2022;62(3):199–205.

Turgut A. A study on multiple drowning syndromes. Int J Inj Contr Saf Promot. 2012;19(1):63–7.

Turgut A, Turgut T. A study on rescuer drowning and multiple drowning incidents. J Safety Res. 2012;43(2):129–32.

Turgut T, Yaman M, Turgut A. Educating children on water safety for drowning prevention. Soc Indic Res. 2016;129:787–801.

Uzun I, et al. Foreigners dying in Istanbul. J Forensic Sci. 2009;54(5):1101–4.

Yayci N, et al. The review of autopsy cases of accidental childhood deaths in Istanbul. J Forensic Leg Med. 2011;18(6):253–6.

Balcı Y, et al. Muğla’da Otopsisi Yapılan Gençlik Yaş Grubuna Ait Olguların Değerlendirilmesi. Adli Tıp Bülteni. 2018;23(3):156–61.

Yıldırım A, Polater E, Bütün C. 0–6 Yaşlar Arası Adli Nitelikli Çocuk Çağı Medikolegal Ölümlerinin Retrospektif Olarak Değerlendirilmesi. Adli Tıp Bülteni. 2020;25(2):65–71.

Küçük MP, et al. Yoğun Bakımda İzlenen Boğulma Olgularında Sonlanımı Etkileyen Parametreler: Sekiz Yıllık Retrospektif Hasta Verileri. Turk J Intensive Care. 2020;18(1):13–20.

Cömert SŞ, et al. Suda boğulma nedeniyle hastanede yatan olgularin özellikleri ve prognostik bulgular. İzmir Göğüs Hastanesi Dergisi. 2014;28(3):161–9.

Türkoğlu A, et al. Elazığ’da 2005–2012 yılları arasında meydana gelen suda boğulma olgularının retrospektif değerlendirilmesi. Fırat Tıp Derg. 2014;19(3):145–50.

Beydilli I, et al. Factors affecting drowning-related mortality of elderly foreigners according to autopsy results. Turk J Geriatr/Türk Geriatri Dergisi. 2016;19(2):81–6.

Demir U, et al. Beş yaşın altındaki çocukların kazayla ölümlerinin analizi. J Forensic Med. 2017;31:1.

MathSciNet   Google Scholar  

Tutanç M, et al. Çocuk yoğun bakım ünitesine yatan hastaların değerlendirilmesi. Duzce Medical Journal. 2011;13(3):18–22.

ADS   Google Scholar  

Tutanç M, et al. Çocuk yoğun bakım ünitesinde takip edilen suda boğulma ve boğulayazma olguları. Adli Tıp Dergisi. 2011;25(3):177–82.

Arslan MM, et al. Adana’da 1997–2001 yılları arasında otopsisi yapılan çocukluk çağı ölüm olgularının retrospektif olarak değerlendirilmesi. Adli Tıp Bülteni. 2004;9(2):37–42.

Turgut A, Işın A, Akdağ E. Yüzme havuzlarındaki suda boğulma vakalarının incelenmesi. İstanbul Üniversitesi Spor Bilimleri Dergisi. 2018;8(1):69–76.

Peden AE, Franklin RC, Willcox-Pidgeon S. Media reporting of summer drowning: A partial picture, useful for advocacy. Health Promot J Austr. 2020;31(3):491–6.

Queiroga AC, et al. Trends in drowning mortality in Portugal from 1992 to 2019: comparing Global Burden of Disease and national data. Inj Prev. 2022;28(4):318–24.

Franklin RC, et al. The burden of unintentional drowning: global, regional and national estimates of mortality from the Global Burden of Disease 2017 Study. Inj Prev. 2020;26(Supp 1):i83–95.

United Nations. Resolution on Global Drowning Prevention. 2021; Available from: https://www.undocs.org/en/A/75/L.76 . (Date cited: 24-05-2023)

Rahman A, et al. Drowning: Global Burden, Risk Factors, and Prevention Strategies. Oxford Research Encyclopedia of Global Public Health. 2021. Retrieved 18 Feb. 2024 from https://oxfordre.com/publichealth/view/10.1093/acrefore/9780190632366.001.0001/acrefore-9780190632366-e-307 .

Peden AE, et al. A systematic review of the evidence for effectiveness of interventions to address transport and other unintentional injuries among adolescents. J Saf Res. 2023;85:21–338.

Peden AE, et al. Adolescent transport and unintentional injuries: a systematic analysis using the Global Burden of Disease Study 2019. The Lancet Public Health. 2022;7(8):e657–69.

Howland J, et al. Why are most drowning victims men? Sex differences in aquatic skills and behaviors. Am J Public Health. 1996;86(1):93–6.

Koon W, et al. Burden of fatal drowning in California, 2005–2019. Inj Prev. 2023;29:371–7.

Peden AE, Franklin RC, Queiroga AC. Epidemiology, risk factors and strategies for the prevention of global unintentional fatal drowning in people aged 50 years and older: a systematic review. Inj Prev. 2018;24(3):240–7.

Download references

There was no funding associated with this research. Author AP is supported by an [Australian] National Health and Medical Research Council Emerging Leadership Fellowship (Grant ID: APP2009306).

Author information

Authors and affiliations.

Department of Coaching Education, Faculty of Sports Sciences, Akdeniz University, 07070, Antalya, Türkiye

School of Population Health, UNSW Sydney, Kensington, NSW, 2052, Australia

Amy E. Peden

College of Public Health, Medical and Veterinary Sciences, James Cook University, Townsville, QLD, 4811, Australia

You can also search for this author in PubMed   Google Scholar

Contributions

AI and AP designed and coordinated of the review. AI and AP screened literature and extracted data. AI led the data visualisation and writing of the study. AP reviewed the data visualisation and manuscript. AI and AP revised the manuscript. Both authors approve the submitted version.

Corresponding author

Correspondence to Ali Işın .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Işın, A., Peden, A.E. The burden, risk factors and prevention strategies for drowning in Türkiye: a systematic literature review. BMC Public Health 24 , 528 (2024). https://doi.org/10.1186/s12889-024-18032-9

Download citation

Received : 24 June 2023

Accepted : 07 February 2024

Published : 20 February 2024

DOI : https://doi.org/10.1186/s12889-024-18032-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Injury Prevention
  • Public Health
  • Water Safety

BMC Public Health

ISSN: 1471-2458

literature review source evaluation

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • Current issue
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Online First
  • Quality and reporting of large-scale improvement programmes: a review of maternity initiatives in the English NHS, 2010–2023
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • http://orcid.org/0000-0001-6963-4106 James McGowan 1 ,
  • http://orcid.org/0000-0002-3650-7415 Bothaina Attal 1 ,
  • http://orcid.org/0000-0002-2879-4020 Isla Kuhn 1 , 2 ,
  • http://orcid.org/0000-0002-6082-3151 Lisa Hinton 3 ,
  • http://orcid.org/0000-0002-1825-4864 Tim Draycott 4 ,
  • http://orcid.org/0000-0003-1979-7577 Graham P Martin 1 ,
  • http://orcid.org/0000-0002-5915-0041 Mary Dixon-Woods 1
  • 1 The Healthcare Improvement Studies Institute (THIS Institute), Department of Public Health and Primary Care , University of Cambridge , Cambridge , UK
  • 2 School of Clinical Medicine , University of Cambridge , Cambridge , UK
  • 3 Nuffield Department of Primary Care Health Sciences , University of Oxford , Oxford , UK
  • 4 Department of Women's Health , North Bristol NHS Trust , Westbury on Trym, Bristol , UK
  • Correspondence to Professor Mary Dixon-Woods, THIS Institute, University of Cambridge School of Clinical Medicine, Cambridge, UK; md753{at}medschl.cam.ac.uk

Background Large-scale improvement programmes are a frequent response to quality and safety problems in health systems globally, but have mixed impact. The extent to which they meet criteria for programme quality, particularly in relation to transparency of reporting and evaluation, is unclear.

Aim To identify large-scale improvement programmes focused on intrapartum care implemented in English National Health Service maternity services in the period 2010–2023, and to conduct a structured quality assessment.

Methods We drew on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidance to inform the design and reporting of our study. We identified relevant programmes using multiple search strategies of grey literature, research databases and other sources. Programmes that met a prespecified definition of improvement programme, that focused on intrapartum care and that had a retrievable evaluation report were subject to structured assessment using selected features of programme quality.

Results We identified 1434 records via databases and other sources. 14 major initiatives in English maternity services could not be quality assessed due to lack of a retrievable evaluation report. Quality assessment of the 15 improvement programmes meeting our criteria for assessment found highly variable quality and reporting. Programme specification was variable and mostly low quality. Only eight reported the evidence base for their interventions. Description of implementation support was poor and none reported customisation for challenged services. None reported reduction of inequalities as an explicit goal. Only seven made use of explicit patient and public involvement practices, and only six explicitly used published theories/models/frameworks to guide implementation. Programmes varied in their reporting of the planning, scope and design of evaluation, with weak designs evident.

Conclusions Poor transparency of reporting and weak or absent evaluation undermine large-scale improvement programmes by limiting learning and accountability. This review indicates important targets for improving quality in large-scale programmes.

  • health services research
  • healthcare quality improvement
  • health policy
  • obstetrics and gynecology
  • womens health

Data availability statement

Data are available upon reasonable request. Please contact the first author.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjqs-2023-016606

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Large-scale improvement programmes are a key strategy for addressing unwarranted variations in quality and safety of care, but their impact is mixed and often limited.

Previous research suggests a number of features of improvement programmes that need to be optimised, but how well these quality criteria are routinely met remains unknown.

WHAT THIS STUDY ADDS

Many large-scale maternity improvement initiatives in the English National Health Service—including some major national programmes of the last decade—lack an evaluation report.

Where an evaluation report was available, quality and design of programmes against prespecified criteria was highly variable, often demonstrating significant weaknesses.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Poor transparency of reporting and weak evaluation in large-scale improvement programmes undermine learning and accountability; explicit attention to features of quality is necessary to improve the design, conduct and impact of large-scale programmes.

Introduction

Variations in quality and safety of healthcare have remained troubling and persistent across health systems globally. Efforts to address these challenges often take the form of large-scale improvement programmes, 1–4 including, for example, multiorganisational collaborative approaches, major initiatives commissioned by policy and professional bodies, implementation programmes and research projects. These programmes are variably effective, with often disappointing results. 5–10 Clarity is, however, now emerging on some of the key features of ‘what good looks like’ for such programmes. 4 10–13 In this article, we report a study that both identifies large-scale improvement initiatives in a clinical area experiencing major patient safety challenges, and offers a structured quality assessment of improvement programmes where an evaluation report was retrievable.

The available literature suggests that a number of features are especially important in large-scale improvement programmes. First, such programmes should be well specified and reported 14–16 to ensure shared understanding of what the programme comprises and its mechanisms. 2 10 17 A second feature of high-quality improvement programmes is that the interventions they use and their delivery should be supported by best available clinical evidence. 11 18–21 Third, high-quality programmes should recognise and meet the requirements for implementation support in participating organisations. 2 22–25 Such support needs to be sensitive to the highly heterogenous nature of local capability, which has been implicated in variable responses to improvement programmes, 2 10 16 22–24 with lower performing organisations having distinctive support needs. 11 13 25–31 Fourth, consistent with published policy objectives, 32–34 programmes should explicitly address inequalities between socioeconomic and ethnic groups. 35 Fifth, patient and public involvement (PPI) has an important role in enhancing the impact of improvement efforts. 36–42 Sixth, improvement programmes benefit from use of formal published theories, models and frameworks from implementation science to guide their work. 43 Finally, an important feature of good improvement programmes is a commitment to sound evaluation, 11 20 21 including, where possible, assessment of effectiveness, process evaluation and economic evaluation. 44–46

The extent to which large-scale improvement programmes routinely meet these seven criteria is unclear. Many programmes, including those commissioned or delivered by national-level organisations, are conducted in a context where expectations of some features (eg, programme specification) may be insufficiently explicit, and incentives for high-quality evaluation and reporting may be lacking. 14 15

English National Health Service (NHS) maternity services are an important example of where quality problems are especially prominent in public discourse 47–49 arising from high-profile organisational failures, 50–53 evidence of persistent unwarranted variation in outcomes, 54–56 rising clinical negligence claims, 49 culture and workforce challenges 57 58 and inequalities linked to socioeconomic status and ethnicity. 33 35 59–63 These challenges have not yet proved tractable, despite multiple large-scale improvement initiatives. 64 65 Maternity services are therefore an important setting in which to assess quality of large-scale improvement programmes, particularly in relation to their reporting and evaluation.

We aimed to identify large-scale improvement programmes that had been implemented in English NHS maternity services between 2010 and 2023 and, for those with an available evaluation report, to conduct a structured quality assessment based on the selected features identified above.

Our design was a review with two components: a search for large-scale improvement initiatives implemented in English NHS maternity services in the period 2010–2023 and, for those that met definitional criteria as improvement programmes and had a retrievable evaluation report, a structured quality assessment. We focused specifically on programmes that primarily addressed quality or safety of intrapartum care, which has been consistently implicated in variations in adverse clinical outcomes in maternity care. 66–68 Initiatives primarily focused on antenatal or postnatal care were therefore not in scope.

Our initial exploratory work found that the majority of maternity improvement initiatives were not research projects and had not been reported in the academic literature; relevant information was mostly available in diverse sources such as websites, policy documents and programme reports. To ensure that both our search and our assessment of programmes was nonetheless structured and systematic, our review was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews guidance 69 ( online supplemental material A ). We also completed all five steps in the Arksey and O’Malley framework, 70 although not sequentially.

Supplemental material

As our review was not intended as a full scoping review of research, we did not register the review in an online database; we did, however, produce a protocol that was used to guide the conduct of the review ( online supplemental material B ).

Eligibility criteria

We developed prospective criteria to guide the identification of eligible improvement programmes and sources of evidence.

Identification of improvement programmes

Initial scoping identified that a large number of highly heterogeneous improvement efforts had taken place in English NHS maternity services in the period we were studying. The following types of initiatives, strategies and interventions were excluded from our review: incident investigation and inspection programmes; national clinical audits and confidential enquiries (eg, the Mothers and Babies: Reducing Risk through Audits and Confidential Enquiries - MBRRACE programme) 71 ; organisational restructuring, major system change and service transformation programmes; health technology assessments and trials of digital technologies without an existing evidence base; clinical guidelines or recommendations without an accompanying implementation programme; and single-site quality improvement projects. Additionally, programmes implemented in clinical specialties other than maternity, outside the English NHS, or implemented in full before 2010 were excluded.

Only initiatives that met our definitional criteria as improvement programmes, had an evaluation report available and focused on intrapartum care were eligible for quality assessment. For this purpose, we defined ‘improvement programmes’ as encompassing a set of planned activities applied at a scale larger than local quality improvement projects, and requiring participation of more than one organisation or clinical service (see box 1 for our full definition). 1–3 72

Definition of ‘improvement programme’

For the purpose of our study, we defined a healthcare improvement programme as a set of planned activities:

Seeking to address a known quality or safety deficit; or seeking to implement evidence-based recommendations or standards of care or practice.

Implemented at scale, that is, in two or more healthcare organisations or clinical services.

With the characteristics of an organised programme, for example, with a structured set of goals, resources, a programme team and report.

Primarily concerned with improving clinical care quality or safety including structures, processes or outcomes.

Only improvement programmes where an evaluation report could be retrieved were included in our quality assessment, since exploratory work indicated it would not be possible to make reliable judgements about programme quality and reporting without such a report. We defined an ‘evaluation report’ as a published assessment of programme design, implementation or outcomes, including formative and/or summative evaluation activities. 44 We classified reports as ‘retrievable’ where they were available for full-text review following their identification in search results, or were otherwise publicly available (eg, published on organisational websites). In determining eligibility for quality assessment, evaluation reports were included without regard to where they had been published (eg, in academic or grey literature) or to the design and quality of the evaluation.

To ensure comprehensive quality assessment of programmes where an evaluation report was available, we supplemented our analysis of evaluation reports with available information from policy reports, programme reports, website entries, peer-reviewed research articles, reviews and study protocols. We excluded editorials, viewpoints, commentaries and letters, non-English language articles and sources published before 2010. Consistent with our focus on intrapartum care, we also excluded sources that primarily addressed quality or safety of care in the antenatal or postnatal periods, or that had only limited focus on intrapartum care (eg, the Getting It Right First Time - GIRFT programme).

Information sources and search strategy

Information sources.

We searched two research databases, determining that this number was both proportionate to our aims and consistent with published guidance regarding the conduct of scoping studies. 73 Database searches were performed in MEDLINE via Ovid and CINAHL via Ebsco from 1 January 2010 to 8 February 2023. These databases were chosen because of their high subject relevance to maternity care. Subject headings (eg, Medical Subject Headings) and free text search terms and synonyms were included. We did not apply restrictions to publication type. Filters were applied for ‘England’ or ‘NHS’, adapted from Ayiku et al . 74 The database searches were designed and performed by a health librarian (IK) in collaboration with JM.

A series of structured searches were performed to identify literature or other information relevant to maternity improvement programmes in Google, Google Scholar and websites of national organisations active in UK maternity quality and safety, supplemented by purposive hand searches. Online searches were based on the search strings ‘maternity safety programme’, ‘maternity quality programme’ and ‘maternity improvement programme’, and were performed independently by two researchers (JM and BA).

Search strategy

Our search strategy was based on a modified ‘PICOS’ framework:

Patient population—women receiving NHS care during the intrapartum period.

Intervention—quality and safety improvement programmes.

Comparison—not applicable.

Outcome—clinical and other outcomes related to quality and safety of maternity care.

Setting—NHS maternity services in England.

Four studies identified during preliminary scoping searches were used as ‘golden bullets’ 75 to assess and improve the sensitivity and specificity of the search strings in identifying relevant literature. 76–79 We supplemented structured searches with forward and backward citation tracking of a purposively selected sample of included studies to improve the sensitivity of the search.

The full list of information sources, search strategies for database and grey literature searches and search record templates are provided in online supplemental material C .

Eligibility screening

Bibliographic database search results were deduplicated in EndNote, imported into Rayyan 80 and screened on the basis of title and abstract. Screening of search results from non-bibliographic sources was performed onscreen by JM and BA; for each search, the first 100 (Google Search and organisational websites) and 500 (Google Scholar) search results were screened and unique sources identified by consensus. Screening of all search results and full-text sources to identify improvement programmes eligible for assessment and relevant sources of evidence was performed independently by two researchers (JM and BA); disagreements regarding the eligibility of both programmes and sources were resolved by discussion.

Data categories and charting

Data categories.

For improvement programmes eligible for quality assessment, we charted data under the seven categories in table 1 . These categories correspond to selected features of quality we had identified from the wider improvement literature, as well as identifying basic programme characteristics.

  • View inline

Data charting framework

Multi-item checklist for assessment of programme specification (modified TIDieR checklist)

Multi-item checklist for assessment of programme evaluation

Our assessment of programmes was supported by published standards in two areas. First, we supported our assessment of programme specification (category 1 in the charting framework) and implementation support (category 3) by using a modified Template for Intervention Description and Replication (TIDieR) checklist, 81 which was adapted to apply to improvement programmes ( table 2 ).

Second, our assessment of programme evaluation (category 7 in the charting framework) was supported by a multi-item checklist informed by UK government guidance on programme evaluation (the ‘Magenta Book’) 44 ( table 3 ).

Charting process

The charting process was supported by a tool built in Microsoft Excel that mapped to the charting framework in table 1 ( online supplemental material D ). Consistent with scoping review methodology, 70 we developed the data items for extraction into the charting framework iteratively, modifying them as new data were identified and analysis progressed. The charting tool was piloted using a small sample (n=2) of sources and amended prior to formal charting. Two researchers (JM and BA) independently charted all data for six of the seven data categories; data relating to use of theories, models and frameworks were charted by JM. Disagreements in assessment gradings were resolved by discussion.

Appraisal of evidence and reporting of findings

Though conducting a quality assessment of eligible programmes (based on the seven features in table 1 ) was a key objective of our analysis, we did not seek to review evidence of effectiveness of programmes, nor did we aim to conduct an appraisal of the methodological quality of individual sources of evidence, as these were not goals of our study. Consistent with the norm in scoping reviews, 70 we also did not formally aggregate or synthesise evidence, instead developing summaries of the data organised by the charting framework. Key findings and themes are reported under the categories of this framework in the Results section 82 and are summarised in a series of supplemental tables.

Patient and public involvement

Patients and the public were not involved in the design or conduct of the review.

We identified 1434 records via bibliographic databases and searches of the grey literature and other sources. From these, 93 full-text sources were retrieved and assessed for eligibility, including one evaluation report that was not initially available, but was subsequently retrieved through personal correspondence. 83 The process by which sources of evidence were identified and screened to determine their eligibility is summarised in our PRISMA diagram ( figure 1 ).

  • Download figure
  • Open in new tab
  • Download powerpoint

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram (adapted from Page et al [ 109 ]). NHS, National Health Service.

Following full-text eligibility screening, we found 38 improvement initiatives (reported across 50 sources) that could not be included in the quality assessment. These initiatives are documented, together with exclusion reasons, in online supplemental material E . Of these, 14 initiatives—including most major initiatives in English maternity services that were implemented during the time period under study, such as the NHS England’s Maternity and Neonatal Safety Improvement Programme and the Maternity Safety Support Programme—lacked a retrievable evaluation report. A further 13 initiatives did not meet our definitional criteria as improvement programmes (eg, because they did not report implementation in two or more clinical services). We also excluded nine initiatives that focused primarily on antenatal or postnatal care from the quality assessment, since our scope was limited to intrapartum care. Two other initiatives that had not been implemented in the relevant setting or time period were also excluded at this stage.

We identified 15 initiatives implemented in maternity services in England between 2010 and 2023 that had a principal focus on intrapartum care, met our definitional criteria as improvement programmes and had a retrievable evaluation report. These 15 initiatives were included in the structured quality assessment and were reported across 43 sources of evidence. Of these sources, grey literature constituted the majority (24). Peer-reviewed academic journal articles constituted 19 sources, with nine taking the form of original research articles reporting findings of evaluations. 30 76–79 84–87 Other article types included study protocols (n=5), 88–92 quality improvement reports (n=3) 93–95 and review articles (n=2). 96 97

We report the findings of our quality assessment of the 15 programmes organised around the seven features of ‘what good looks like’ below, with additional data in online supplemental material F .

Feature 1: quality of programme specification

Reporting of the basic characteristics of the programmes included in our quality assessment was reasonable ( online supplemental material F: table 1 ). For example, it was possible to identify the clinical setting, target recipients, programme type and source of funding for all 15 programmes. Information on the scale of the programme was available for 14 of the 15 programmes, and on time period of implementation for 13 programmes.

Beyond these basic characteristics, completeness of programme description varied markedly between programmes and was generally of low quality ( online supplemental material F: table 2 ). No programme described all 10 items of the modified TIDieR checklist in full. Five of the included programmes provided no description for half of the checklist items. Two programmes (Saving Babies’ Lives Care Bundle and Prevention of cerebral palsy in pre-term Labour - PReCePT) provided full description of six checklist items, while three programmes (Maternity Incentive Scheme, Maternity Safety Training Fund and the Safer Births Project) did not describe any checklist item in full.

Completeness of description also varied between checklist items. Goals (described in full for 11 programmes) were generally well described, as were mechanisms of action and theories of change (described in full for eight programmes). In contrast, frequency, duration and time period of local implementation, local tailoring and modifications, processes for assessing or maintaining fidelity and outcome of fidelity assessment were described poorly. For example, no programmes offered a full description of frequency, duration and time period of local implementation or of local tailoring and modifications, and only three programmes offered any level of description for the latter item. Items relating to fidelity were particularly poorly described; only two programmes (Saving Babies’ Lives Care Bundle and PReCePT) offered any level of description of a fidelity assessment outcome. Only the Saving Babies’ Lives Care Bundle provided a full description of both the processes and outcome of fidelity assessment.

Feature 2: use of evidence-based interventions

Programmes varied in the extent to which they explicitly based their improvement interventions on evidence. Of the 15 programmes we assessed, only eight reported that their interventions were developed with explicit reference to published evidence ( online supplemental material F: tables 3 and 4 ). Of these, most (n=6) based their interventions on recommendations from national clinical guidance or quality standards, for example, those published by Royal Colleges and the National Institute for Health and Care Excellence. Two programmes (PReCePT and Perinatal Excellence to reduce injury in preterm birth - PERIPrem) cited a range of studies in support of their interventions, including systematic reviews and meta-analyses, randomised controlled trials and cohort studies. 79 95 98 99 The remaining seven programmes did not cite the evidence base for interventions used, if any.

Feature 3: description of implementation support for services

Completeness of description of the implementation support available to clinical services (final column, online supplemental material F: table 2 ) was poor. Only three programmes (Obstetric anal sphincter injury Care Bundle - OASI-CB, PReCePT and PERIPrem) were judged to have described implementation support and activities in full. Whether or not implementation support had been provided was unclear for seven programmes, as no level of description was offered.

We did not identify any programme report that described customisation of implementation support for challenged services. The PReCePT programme did offer enhanced implementation support to a subset of participating services, though the authors did not report whether this was targeted specifically at challenged services. 100 Recruitment to the Labour Ward Leadership programme was partially informed by Care Quality Commission (CQC) inspection reports, with ‘Trusts with identified problems’ given priority. 83 Two programmes 91 100 101 accounted for variation in local service contexts in their evaluation methods by using, among other criteria, CQC inspection ratings to inform sampling of study participants. The interim evaluation of the Maternity Incentive Scheme 102 stated that additional support was provided to Trusts that were not in compliance with all 10 incentivised actions to enable them to ‘achieve full compliance’, but we did not find evidence that this support was targeted to support challenged services. In contrast, NHS trusts in ‘special measures’ and those in receipt of support from national regulators were specifically excluded from participating in the Each Baby Counts Learn & Support programme. 103 An incidental finding was the potential for programmes to exacerbate inequalities of resourcing between services, as noted by a PERIPrem report (parentheses added):

In Trusts where there was a pre-existing QI (quality improvement) culture and a desire to embed new practice and change, implementation was easier. Those with active hospital QI teams were able to access additional support and training for the project. 104

Feature 4: commitment to reducing inequalities

Although we identified several evaluation reports that adjusted for socioeconomic status, ethnicity or levels of deprivation in their analysis, 78 79 84 95 100 101 105 we did not find any examples among the 15 programmes we assessed that identified the reduction of health or care inequalities as an explicit goal ( online supplemental material F: table 4 ).

Feature 5: patient and public involvement

Of the 15 programmes we assessed, only seven made explicit use of PPI practices. These included five that demonstrated comprehensive attention to involvement at all stages of the programme (including design, development, implementation, evaluation and dissemination). The remaining eight programmes did not mention of the role of patients or the public ( online supplemental material F: table 4 ).

Feature 6: use of formal published theories, models or frameworks

A minority (n=6) of assessed programmes made explicit use of formal published theories, models or frameworks from implementation science to guide programme implementation or evaluation ( online supplemental material F: tables 4 and 5 ). 106 We identified 12 documented examples of use of theories, models or frameworks from four of Nilsen’s proposed five categories, 107 including implementation theories (n=6), classic theories (n=2), determinant frameworks (n=2) and evaluation frameworks (n=2).

Feature 7: programme evaluation

Programmes varied in their reporting of the planning, scope, design and conduct of evaluation ( online supplemental material F: table 6 ). Evidence of a prospective evaluation plan (eg, in the form of a published protocol) was identified for seven of the 15 programmes, and four programmes conducted a pilot that informed programme development or implementation.

The evaluations we assessed were dominated by weak designs, often relying on post hoc methods of data collection, such as self-report questionnaires and staff surveys. Of the 19 peer-reviewed research articles identified (which collectively reported on nine programmes), five 76–79 84 reported evaluations of effectiveness, but only two studies reported on cost-effectiveness. 78 79 Only three evaluation reports 76 84 100 (including one preprint 100 ) employed a randomised design. Three evaluations employed quasiexperimental approaches. 78 79 92 Only four articles reported findings from process evaluations, implementation research or qualitative studies. 30 85–87

Large-scale improvement programmes have been a key strategy for addressing quality deficits in healthcare globally, but the programmes assessed in our review of one exemplar clinical area often fell short on key features of quality. Though a large number of improvement initiatives have been undertaken in a 13-year period in maternity services, a particularly challenged clinical specialty in the English NHS, many—including a large number of major national programmes of the last decade—did not meet a basic requirement of a retrievable evaluation report. This represents a major threat to learning and accountability. Among programmes with a focus on intrapartum care that offered some form of evaluation report and could be assessed, there was considerable variability and very often evident flaws in transparency and quality of programme specification, use of evidence-based interventions, implementation support, PPI, use of formal published theories, models and frameworks, and evaluation. Notably, no programme that we quality assessed had explicitly set reduction of inequality as a goal. Our findings are unlikely to be unique to maternity settings or to the English NHS. These findings, and the methods used to generate them, are likely to be of relevance to many other clinical areas targeted by large-scale improvement programmes in healthcare settings internationally.

A first step in improving the quality of improvement programmes is to ensure that they are sufficiently well specified to permit identification of their components and the mechanisms through which they work, not least so that they can be scaled with fidelity if shown to be effective, 10 14 108 and modified or abandoned if not effective. Programmes included in our quality assessment demonstrated highly variable completeness of programme description, with some demonstrating weaknesses across several items of a modified TIDieR checklist. 81 We also identified an important lack of transparency in reporting relating to the implementation support available to participating services, despite its recognised role in effective improvement, 25 29 including in maternity care. 30 100 Crucially, despite recurrent reported organisational degradations in NHS maternity services, we were unable to identify any examples of adaptation of implementation specifically to account for the context of challenged services, with one programme actively excluding challenged units. 103 The extent to which programmes explicitly grounded their interventions in evidence or drew on theories, models or frameworks from implementation science 43 to guide implementation and evaluation was also highly variable, with many examples of poor reporting practice.

Second, a commitment to evaluation and public reporting of all findings should be seen as fundamental to high-quality commissioning of large-scale improvement programmes in healthcare. Many (14) of the initiatives we identified—including some of the high-profile national maternity programmes of the last decade—lacked a retrievable evaluation report. In these cases, it is not possible to determine whether the programmes worked (made a difference to outcomes) and should be scaled, to assess whether these programmes represent a good use of resources, or to identify how programme design or implementation might have been improved. Even where evaluation reports were available, they often demonstrated substantial weaknesses. Future programme design and delivery should be organised to facilitate the use of rigorous evaluation designs that allow reliable assessments to be made about effectiveness and cost-effectiveness, as well as what works, what doesn’t and why across diverse clinical settings. 44

Third, despite repeated policy commitments to improve equity, 32 34 our findings add to concerns about inequalities in NHS maternity care. 33 35 59 We found no examples of improvement programmes included in our quality assessment that identified the reduction of health and care inequalities as an explicit goal. Indeed, there was some evidence of programme design having potential to contribute to widening inequalities between high-performing and low-performing services, which is likely to impede efforts aimed at improving equity. 33 Finally, despite the emphasis placed by national policy on including those who use maternity services in the design and delivery of improvement programmes, 41 42 PPI appeared to be lacking in over half of the programmes we assessed. Improving the impact of large-scale improvement efforts on the quality and safety of care in future will likely require these gaps between these enshrined policy objectives and programme design to be closed.

Strengths and limitations

Our adaptation of the principles of scoping review methodology was successful in addressing our aims, given that most programmes we identified were not research projects and therefore unsuitable for full scoping or systematic review designs. The design of our search strategy was improved by specialist librarian input, by pilot testing the search strings to test their capacity to identify relevant literature, by selecting databases with relevant scope and by supplementing our structured searches with extensive hand searches. We sought to lend rigour to the charting process by requiring that assessment decisions be agreed by two researchers, by pilot testing the data charting tool to improve its reliability and by basing our assessment criteria, where possible, on published standards relating to programme specification 81 and evaluation. 44

Our selection of the TIDieR checklist 81 —developed to improve published descriptions of healthcare interventions, including complex interventions—on which to base our assessments of reporting was appropriate given the sociotechnical nature of improvement programmes. Our modified checklist is likely to be valuable in future studies to enable better description of features specific to improvement programmes. Relatedly, our definition of ‘improvement programme’ offers clarity to those engaged in programme commissioning, design and evaluation regarding specific aspects of these interventions that distinguish them from other types of improvement intervention in healthcare.

We acknowledge several limitations. It is possible that relevant sources of evidence relevant to eligible programmes were missed by our search strategy. The list of quality features we identified was necessarily selective and may not be comprehensive. It remains possible that researcher-related factors may have biased our assessments in some non-transparent way. Practical considerations meant our search was limited to programmes implemented in England since 2010, meaning that potentially important learning from programmes implemented elsewhere in the UK and prior to 2010 was excluded. Restriction of scope to intrapartum care may have excluded some programmes with different characteristics of quality and reporting.

Conclusions

Transparent reporting and high-quality evaluation are critical to learning and accountability in healthcare systems facing persistent quality of care challenges. This review of large-scale maternity improvement programmes in the English NHS since 2010 has identified widespread poor practice in programme design, transparency of reporting and evaluation. These findings are both cause for concern and unlikely to be unique to this clinical setting. Our study suggests important targets for improving the design, delivery, evaluation and reporting of large-scale programmes in healthcare to maximise their impact on quality and safety, ensure accountability, including for how resources are used, and better aggregate learning to improve care for patients.

Ethics statements

Patient consent for publication.

Not applicable.

  • Ovretveit J ,
  • Gustafson D
  • Burnett S ,
  • Parand A , et al
  • Ovretveit J
  • Dixon-Woods M ,
  • McNicol S ,
  • Stephens T ,
  • Martin G , et al
  • Tyrrell PJ ,
  • Rudd AG , et al
  • Richardson A ,
  • Hibbert P , et al
  • Benning A ,
  • Suokas A , et al
  • Nwulu U , et al
  • Lenguerrand E ,
  • Siassakos D , et al
  • Dixon-Woods M
  • Brandrud AS ,
  • Hjortdahl P , et al
  • Vindrola-Padros C ,
  • Hill M , et al
  • Skivington K ,
  • Matthews L ,
  • Simpson SA , et al
  • Duncan MD , et al
  • Shojania KG ,
  • Grimshaw JM
  • Bolshakova M ,
  • Turner BJ , et al
  • Marshall M ,
  • Pronovost P ,
  • Ozieranski P ,
  • McNicol S , et al
  • Tarrant C , et al
  • Stephens TJ ,
  • Pearse RM , et al
  • Greenhalgh T ,
  • Macfarlane F , et al
  • Kaplan HC ,
  • Dritz MC , et al
  • Howarth E ,
  • Moore G , et al
  • Redwood S ,
  • Pithara-McKeown C ,
  • Stone T , et al
  • Barbosa EC , et al
  • NHS England
  • Care Quality Commission
  • Jardine J ,
  • Gurol-Urganci I , et al
  • Jackson C ,
  • Shaw S , et al
  • Whitaker GP , et al
  • Langley J ,
  • Wolstenholme D ,
  • Williams O , et al
  • Donetto S ,
  • Finlay T , et al
  • Mockford C ,
  • Staniszewska S ,
  • Griffiths F , et al
  • Eccles MP ,
  • HM Treasury
  • Portela MC ,
  • Pronovost PJ ,
  • Woodcock T , et al
  • Goldmann D , et al
  • Parliamentary and Health Service Ombudsman
  • House of Commons Health and Social Care Committee
  • Liberati E , et al
  • Proudlove NC ,
  • Walshe K , et al
  • Carroll F , et al
  • Patel R , et al
  • Birthrights
  • House of Commons Women and Equalities Committee
  • Department of Health and Social Care
  • Department of Health, Safer Maternity Care
  • NHS Litigation Authority
  • NHS Resolution
  • Tricco AC ,
  • Zarin W , et al
  • Felker BK ,
  • Patel A , et al
  • Batalden PB
  • Godfrey C ,
  • McInerney P , et al
  • Hudson T , et al
  • Zwakman M ,
  • Verberne LM ,
  • Kars MC , et al
  • Ismail KMK ,
  • Macdonald SE , et al
  • Vinayakarao L ,
  • Pathak S , et al
  • Widdows K ,
  • Roberts SA ,
  • Camacho EM , et al
  • Edwards HB ,
  • Redaniel MT ,
  • Sillero-Rejon C , et al
  • Ouzzani M ,
  • Hammady H ,
  • Fedorowicz Z , et al
  • Hoffmann TC ,
  • Glasziou PP ,
  • Boutron I , et al
  • NHS England (Avoiding Term Admissions into Neonatal units programme), Royal College of Midwives, Royal College of Obstetricians and Gynaecologists
  • Gurol-Urganci I ,
  • Bidwell P ,
  • Sevdalis N , et al
  • Roberts SA , et al
  • Sevdalis N ,
  • Silverton L , et al
  • Macdonald S , et al
  • Edwards H ,
  • Opmeer B , et al
  • O’Brien S ,
  • Jordan S , et al
  • Burhouse A ,
  • Ray S , et al
  • Edwards K ,
  • Glover Williams A ,
  • McBain H , et al
  • Robertson L ,
  • Prosser-Snelling E , et al
  • Jurczuk M ,
  • West of England Academic Health Science Network (AHSN)
  • Sillero-Rejon C
  • Grimwood T ,
  • Royal College of Obstetricians and Gynaecologists
  • Williams AG , et al
  • Roberts S ,
  • Camacho E , et al
  • Renwick S ,
  • Draycott T , et al
  • McKenzie JE ,
  • Bossuyt PM , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

  • Data supplement 1
  • Data supplement 2
  • Data supplement 3
  • Data supplement 4
  • Data supplement 5
  • Data supplement 6

Twitter @jgmcgowan, @ilk21, @LisaHinton4, @graham_p_martin, @MaryDixonWoods

Contributors MD-W is the guarantor of the study. Conceptualisation: JM, MD-W. Data curation: JM, BA. Formal analysis: JM, BA. Funding acquisition: JM, MD-W. Investigation: JM, BA, IK. Methodology: JM, BA, IK, LH, TD, GPM, MD-W. Project administration: JM, IK. Resources: JM, IK, MD-W. Software: JM, IK. Supervision: GPM, MD-W. Validation: LH, TD, GPM, MD-W. Visualisation: JM. Writing—original draft: JM, MDW. Writing—review and editing: all authors.

Funding The Health Foundation's grant to THIS Institute.

Competing interests TJD: Research and Innovation lead for PROMPT Maternity Foundation and has a part-time appointment at NHS Resolution where he leads on a Safety Action within the Maternity Incentive Scheme.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

U.S. flag

An official website of the United States government.

Here’s how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • American Job Centers
  • Apprenticeship
  • Demonstration Grants
  • Farmworkers
  • Federal Bonding Program
  • Foreign Labor Certification
  • Indians and Native Americans
  • Job Seekers
  • Layoffs and Rapid Response
  • National Dislocated Worker Grants
  • Older Workers
  • Skills Training Grants
  • Trade Adjustment Assistance
  • Unemployment Insurance
  • Workforce Innovation and Opportunity Act (WIOA)
  • WIOA Adult Program
  • Advisories and Directives
  • Regulations
  • Labor Surplus Area
  • Performance
  • Recovery-Ready Workplace Resource Hub
  • Research and Evaluation
  • ETA News Releases
  • Updates for Workforce Professionals
  • Regional Offices
  • Freedom of Information Act

The Role of the Workforce System in Addressing the Opioid Crisis: A Literature Review

Publication info, research methodology, description, other products.

This literature review describes findings from studies on various employment and training interventions to 1) assist individuals in recovery, 2) provide assistance to employers preventing opioid use disorder and creating a recovery-friendly workplace, and 3) develop the health care workforce to address the opioid crisis. The review was developed as part of an implementation evaluation of six Dislocated Worker Demonstration Grants to address the National Health Emergency (NHE) of the opioid crisis. Products form the study also include a resource guide, final report, and four short briefs on promising strategies.

As this review notes, the evidence base for employment interventions specifically aimed at or tested with people with opioid use disorder is limited and, that, while some of the approaches have been rigorously tested, others have not yet been evaluated but are seen as potentially promising practices. The research reviewed covers such approaches as:

  • Intensive case management, as found in various models, such as the individual placement and support (IPS) model, a counseling model based on the interpersonal cognitive problem solving (ICPS) method; and a strategy based on the customized employment support (CES) vocational model;
  • Use of "contingency management," a treatment approach that provides privileges or rewards to participants who exhibit desired behaviors;
  • "Lighter-touch" employment or vocational services for people receiving substance use disorder treatment;
  • Workplace prevention initiatives, employee assistance programs, recovery-friendly workplace initiatives, and modifications in workplace drug testing; and
  • Innovative methods to increase the reach and breadth of training for health care professionals, strategies to support provider training on using medication-assisted treatment, and use of nontraditional providers (such as peer recovery specialists).

The literature notes that, overall, the research on employment-related interventions for people with opioid use disorder is still in its infancy, and for that reason, opportunities for building evidence should be capitalized upon by any organization providing services to address it, and in so doing, lay the groundwork for more rigorous studies.

  • Newsletters

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

  • Will Douglas Heaven archive page

OpenAI has built a striking new generative video model called Sora that can take a short text description and turn it into a detailed, high-definition film clip up to a minute long.

Based on four sample videos that OpenAI shared with MIT Technology Review ahead of today’s announcement, the San Francisco–based firm has pushed the envelope of what’s possible with text-to-video generation (a hot new research direction that we flagged as a trend to watch in 2024 ).

“We think building models that can understand video, and understand all these very complex interactions of our world, is an important step for all future AI systems,” says Tim Brooks, a scientist at OpenAI.

But there’s a disclaimer. OpenAI gave us a preview of Sora (which means sky in Japanese) under conditions of strict secrecy. In an unusual move, the firm would only share information about Sora if we agreed to wait until after news of the model was made public to seek the opinions of outside experts. [Editor’s note: We’ve updated this story with outside comment below.] OpenAI has not yet released a technical report or demonstrated the model actually working. And it says it won’t be releasing Sora anytime soon. [ Update: OpenAI has now shared more technical details on its website.]

The first generative models that could produce video from snippets of text appeared in late 2022. But early examples from Meta , Google, and a startup called Runway were glitchy and grainy. Since then, the tech has been getting better fast. Runway’s gen-2 model, released last year, can produce short clips that come close to matching big-studio animation in their quality. But most of these examples are still only a few seconds long.  

The sample videos from OpenAI’s Sora are high-definition and full of detail. OpenAI also says it can generate videos up to a minute long. One video of a Tokyo street scene shows that Sora has learned how objects fit together in 3D: the camera swoops into the scene to follow a couple as they walk past a row of shops.

OpenAI also claims that Sora handles occlusion well. One problem with existing models is that they can fail to keep track of objects when they drop out of view. For example, if a truck passes in front of a street sign, the sign might not reappear afterward.  

In a video of a papercraft underwater scene, Sora has added what look like cuts between different pieces of footage, and the model has maintained a consistent style between them.

It’s not perfect. In the Tokyo video, cars to the left look smaller than the people walking beside them. They also pop in and out between the tree branches. “There’s definitely some work to be done in terms of long-term coherence,” says Brooks. “For example, if someone goes out of view for a long time, they won’t come back. The model kind of forgets that they were supposed to be there.”

Impressive as they are, the sample videos shown here were no doubt cherry-picked to show Sora at its best. Without more information, it is hard to know how representative they are of the model’s typical output.   

It may be some time before we find out. OpenAI’s announcement of Sora today is a tech tease, and the company says it has no current plans to release it to the public. Instead, OpenAI will today begin sharing the model with third-party safety testers for the first time.

In particular, the firm is worried about the potential misuses of fake but photorealistic video . “We’re being careful about deployment here and making sure we have all our bases covered before we put this in the hands of the general public,” says Aditya Ramesh, a scientist at OpenAI, who created the firm’s text-to-image model DALL-E .

But OpenAI is eyeing a product launch sometime in the future. As well as safety testers, the company is also sharing the model with a select group of video makers and artists to get feedback on how to make Sora as useful as possible to creative professionals. “The other goal is to show everyone what is on the horizon, to give a preview of what these models will be capable of,” says Ramesh.

To build Sora, the team adapted the tech behind DALL-E 3, the latest version of OpenAI’s flagship text-to-image model. Like most text-to-image models, DALL-E 3 uses what’s known as a diffusion model. These are trained to turn a fuzz of random pixels into a picture.

Sora takes this approach and applies it to videos rather than still images. But the researchers also added another technique to the mix. Unlike DALL-E or most other generative video models, Sora combines its diffusion model with a type of neural network called a transformer.

Transformers are great at processing long sequences of data, like words. That has made them the special sauce inside large language models like OpenAI’s GPT-4 and Google DeepMind’s Gemini . But videos are not made of words. Instead, the researchers had to find a way to cut videos into chunks that could be treated as if they were. The approach they came up with was to dice videos up across both space and time. “It’s like if you were to have a stack of all the video frames and you cut little cubes from it,” says Brooks.

The transformer inside Sora can then process these chunks of video data in much the same way that the transformer inside a large language model processes words in a block of text. The researchers say that this let them train Sora on many more types of video than other text-to-video models, varied in terms of resolution, duration, aspect ratio, and orientation. “It really helps the model,” says Brooks. “That is something that we’re not aware of any existing work on.”

“From a technical perspective it seems like a very significant leap forward,” says Sam Gregory, executive director at Witness, a human rights organization that specializes in the use and misuse of video technology. “But there are two sides to the coin,” he says. “The expressive capabilities offer the potential for many more people to be storytellers using video. And there are also real potential avenues for misuse.” 

OpenAI is well aware of the risks that come with a generative video model. We are already seeing the large-scale misuse of deepfake images . Photorealistic video takes this to another level.

Gregory notes that you could use technology like this to misinform people about conflict zones or protests. The range of styles is also interesting, he says. If you could generate shaky footage that looked like something shot with a phone, it would come across as more authentic.

The tech is not there yet, but generative video has gone from zero to Sora in just 18 months. “We’re going to be entering a universe where there will be fully synthetic content, human-generated content and a mix of the two,” says Gregory.

The OpenAI team plans to draw on the safety testing it did last year for DALL-E 3. Sora already includes a filter that runs on all prompts sent to the model that will block requests for violent, sexual, or hateful images, as well as images of known people. Another filter will look at frames of generated videos and block material that violates OpenAI’s safety policies.

OpenAI says it is also adapting a fake-image detector developed for DALL-E 3 to use with Sora. And the company will embed industry-standard C2PA tags , metadata that states how an image was generated, into all of Sora’s output. But these steps are far from foolproof. Fake-image detectors are hit-or-miss. Metadata is easy to remove, and most social media sites strip it from uploaded images by default.  

“We’ll definitely need to get more feedback and learn more about the types of risks that need to be addressed with video before it would make sense for us to release this,” says Ramesh.

Brooks agrees. “Part of the reason that we’re talking about this research now is so that we can start getting the input that we need to do the work necessary to figure out how it could be safely deployed,” he says.

Update 2/15: Comments from Sam Gregory were added .

Artificial intelligence

Ai for everything: 10 breakthrough technologies 2024.

Generative AI tools like ChatGPT reached mass adoption in record time, and reset the course of an entire industry.

What’s next for AI in 2024

Our writers look at the four hot trends to watch out for this year

  • Melissa Heikkilä archive page

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

Deploying high-performance, energy-efficient AI

Investments into downsized infrastructure can help enterprises reap the benefits of AI while mitigating energy consumption, says corporate VP and GM of data center platform engineering and architecture at Intel, Zane Ball.

  • MIT Technology Review Insights archive page

Stay connected

Get the latest updates from mit technology review.

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at [email protected] with a list of newsletters you’d like to receive.

IMAGES

  1. 39 Best Literature Review Examples (Guide & Samples)

    literature review source evaluation

  2. FREE 8+ Sample Literature Review Templates in PDF

    literature review source evaluation

  3. Literature Review Summary Table : Chapter 7: Synthesizing Sources

    literature review source evaluation

  4. 50 Smart Literature Review Templates (APA) ᐅ TemplateLab

    literature review source evaluation

  5. Sample literature review mla format

    literature review source evaluation

  6. School paper: Example of a literature review essay

    literature review source evaluation

VIDEO

  1. Review of literature

  2. Write Your Literature Review FAST

  3. Writing a Literature Review

  4. MCPC Literature Evaluation 1 Video

  5. Effective Review of Literature

  6. Literature review and its process

COMMENTS

  1. Evaluating Sources

    One of the best ways to evaluate source credibility is the CRAAP test. This stands for: Currency: Does the source reflect recent research? Relevance: Is the source related to your research topic? Authority: Is it a respected publication? Is the author an expert in their field?

  2. Evaluating Literature Reviews and Sources

    A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

  3. Evaluating Sources & Lit. Reviews

    Evaluating Literature Reviews and Sources A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

  4. Evaluating Sources

    The most important criteria for evaluating a potential resource are: Currency: When the source was published Coverage/Relevance: How closely related the source is to your topic and research question Authority: Who wrote the source and whether they are likely to be credible on the subject

  5. How to Write a Literature Review

    Step 2 - Evaluate and select sources Step 3 - Identify themes, debates, and gaps Step 4 - Outline your literature review's structure Step 5 - Write your literature review Free lecture slides Other interesting articles Frequently asked questions Introduction Quick Run-through Step 1 & 2 Step 3 Step 4 Step 5

  6. Evaluating Sources & Literature Reviews

    A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

  7. Chapter 5: Evaluating Sources

    When evaluating a work for inclusion in, or exclusion from, your literature review, ask yourself a series of questions about each source. 5.1.1 Evaluating books For primary and secondary sources you located in your search, use the ASAP mnemonic to evaluate inclusion in your literature review: 5.1.1.1 Age Is it outdated?

  8. Writing a literature review

    Introduction A formal literature review is an evidence-based, in-depth analysis of a subject. There are many reasons for writing one and these will influence the length and style of your review, but in essence a literature review is a critical appraisal of the current collective knowledge on a subject.

  9. Evaluating Sources

    Accuracy, authority, objectivity, currency and coverage are the five basic criteria for evaluating information from any sources. Scholarly journals vs. non-scholarly journals It has been mentioned on "The Literature" page of this guide that a literature review generally consists of scholarly works.

  10. PDF Conducting a Literature Review

    Literature Review A literature review is a survey of scholarly sources that provides an overview of a particular topic. Literature reviews are a collection of the most relevant and significant publications regarding that topic in order to provide a comprehensive look at what has been said on the topic and by whom.

  11. Writing a Literature Review

    Critically Evaluate: Mention the strengths and weaknesses of your sources Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts. Conclusion: Summarize the key findings you have taken from the literature and emphasize their significance

  12. Literature Reviews

    Literature Reviews - Source Evaluation - LibGuides at Kettering University Source Evaluation What is a Literature Review? A literature review is a comprehensive study and interpretation of literature that addresses a specific topic. Literature reviews are generally conducted in one of two ways:

  13. 5. The Literature Review

    Definition A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated.

  14. Chapter 9 Methods for Literature Reviews

    Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generat...

  15. Guidance on Conducting a Systematic Literature Review

    Literature review is an essential feature of academic research. Fundamentally, knowledge advancement must be built on prior existing work. To push the knowledge frontier, we must know where the frontier is. By reviewing relevant literature, we understand the breadth and depth of the existing body of work and identify gaps to explore.

  16. Literature reviews as independent studies: guidelines for academic

    A literature review - or a review article - is "a study that analyzes and synthesizes an existing body of literature by identifying, challenging, and advancing the building blocks of a theory through an examination of a body (or several bodies) of prior work (Post et al. 2020, p. 352).Literature reviews as standalone pieces of work may allow researchers to enhance their understanding of ...

  17. Literature review

    A literature review also includes a critical evaluation of the material; this is why it is called a literature review rather than a literature report. It is a process of reviewing the literature, as well as a form of writing. To illustrate the difference between reporting and reviewing, think about television or film review articles.

  18. Writing a literature review

    Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...

  19. PDF LITERATURE REVIEWS

    WRITING A TARGETED LITERATURE REVIEW a targeted literature review is NOT: ¡ a sophisticated evaluation of the entire literature or literatures related to your topic ¡ a set of thinly connected summaries of important related works haphazardly selected from many subfields a targeted literature review IS: ¡ a carefully curated set of sources from a small number of subfield literatures

  20. How to write a literature review

    What is a literature review The literature review is a written overview of major writings and other sources on a selected topic. Sources covered in the review may include scholarly journal articles, books, government reports, Web sites, etc. The literature review provides a description, summary and evaluation of each source.

  21. Literature Review Overview

    Select, analyze and annotate sources. Evaluate research articles and other documents. Structure and organize the literature review. Develop arguments and supporting claims. Synthesize and interpret the literature. Put it all together. Writing the Literature Review by Sara Efrat Efron; Ruth Ravid. Call Number: PE1478 .E37 2019. ISBN: 9781462536894.

  22. Ten Simple Rules for Writing a Literature Review

    Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications .For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively .Given such mountains of papers, scientists cannot be expected to examine in detail every ...

  23. Literature review as a research methodology: An ...

    As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.

  24. The burden, risk factors and prevention strategies for drowning in

    Drowning is a public health problem in Türkiye, as in the rest of the world. This study aims to systematically review the literature on drowning in Türkiye with a focus on data sources, epidemiology, risk factors and prevention strategies. Methods: Literature searches were conducted using PubMed, SPORTSDiscus, Scopus, Web of Science, Turk MEDLINE, Google Scholar and Google Akademik (Turkish ...

  25. Quality and reporting of large-scale improvement programmes: a review

    Results We identified 1434 records via databases and other sources. 14 major initiatives in English maternity services could not be quality assessed due to lack of a retrievable evaluation report. Quality assessment of the 15 improvement programmes meeting our criteria for assessment found highly variable quality and reporting. Programme specification was variable and mostly low quality.

  26. The Role of the Workforce System in Addressing the Opioid Crisis: A

    This literature review describes findings from studies on various employment and training interventions to 1) assist individuals in recovery, 2) provide assistance to employers preventing opioid use disorder and creating a recovery-friendly workplace, and 3) develop the health care workforce to address the opioid crisis. The review was developed as part of an implementation evaluation of six ...

  27. OpenAI teases an amazing new generative video model called Sora

    February 15, 2024. OpenAI. OpenAI has built a striking new generative video model called Sora that can take a short text description and turn it into a detailed, high-definition film clip up to a ...

  28. Mark Zuckerberg says Meta's Quest 3 is the clear winner compared to

    Someone get Mark Zuckerberg a gadget review YouTube channel, stat. The Meta CEO on Monday posted a video review of Apple's Vision Pro, the new rival mixed reality headset to Meta's Quest 3 ...