advantages of literature review to a researcher

• Become Involved |
• Give to the Library |
• Staff Directory |
• UNF Library
• Thomas G. Carpenter Library

Conducting a Literature Review

Benefits of conducting a literature review.

• Steps in Conducting a Literature Review
• Summary of the Process
• Additional Resources
• Literature Review Tutorial by American University Library
• The Literature Review: A Few Tips On Conducting It by University of Toronto
• Write a Literature Review by UC Santa Cruz University Library

While there might be many reasons for conducting a literature review, following are four key outcomes of doing the review.

Assessment of the current state of research on a topic . This is probably the most obvious value of the literature review. Once a researcher has determined an area to work with for a research project, a search of relevant information sources will help determine what is already known about the topic and how extensively the topic has already been researched.

Identification of the experts on a particular topic . One of the additional benefits derived from doing the literature review is that it will quickly reveal which researchers have written the most on a particular topic and are, therefore, probably the experts on the topic. Someone who has written twenty articles on a topic or on related topics is more than likely more knowledgeable than someone who has written a single article. This same writer will likely turn up as a reference in most of the other articles written on the same topic. From the number of articles written by the author and the number of times the writer has been cited by other authors, a researcher will be able to assume that the particular author is an expert in the area and, thus, a key resource for consultation in the current research to be undertaken.

Identification of key questions about a topic that need further research . In many cases a researcher may discover new angles that need further exploration by reviewing what has already been written on a topic. For example, research may suggest that listening to music while studying might lead to better retention of ideas, but the research might not have assessed whether a particular style of music is more beneficial than another. A researcher who is interested in pursuing this topic would then do well to follow up existing studies with a new study, based on previous research, that tries to identify which styles of music are most beneficial to retention.

Determination of methodologies used in past studies of the same or similar topics.  It is often useful to review the types of studies that previous researchers have launched as a means of determining what approaches might be of most benefit in further developing a topic. By the same token, a review of previously conducted studies might lend itself to researchers determining a new angle for approaching research.

Upon completion of the literature review, a researcher should have a solid foundation of knowledge in the area and a good feel for the direction any new research should take. Should any additional questions arise during the course of the research, the researcher will know which experts to consult in order to quickly clear up those questions.

• << Previous: Home
• Next: Steps in Conducting a Literature Review >>
• Last Updated: Aug 29, 2022 8:54 AM
• URL: https://libguides.unf.edu/litreview

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Fact check: Correcting the record about the UC Berkeley Library’s long-term space plan

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Conducting a literature review: why do a literature review, why do a literature review.

• How To Find "The Literature"
• Found it -- Now What?

Besides the obvious reason for students -- because it is assigned! -- a literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed.

You identify:

• core research in the field
• experts in the subject area
• methodology you may want to use (or avoid)
• gaps in knowledge -- or where your research would fit in

It Also Helps You:

• Publish and share your findings
• Justify requests for grants and other funding
• Identify best practices to inform practice
• Set wider context for a program evaluation
• Compile information to support community organizing

Great brief overview, from NCSU

Want To Know More?

• Next: How To Find "The Literature" >>
• Last Updated: Dec 7, 2022 12:19 PM
• URL: https://guides.lib.berkeley.edu/litreview

Dissertations - Skills Guide

• Where to start
• Research Proposal
• Ethics Form
• Primary Research

Literature Review

• Methodology
• Downloadable Resources
• Further Reading

What is it?

Literature reviews involve collecting information from literature that is already available, similar to a long essay. It is a written argument that builds a case from previous research (Machi and McEvoy, 2012). Every dissertation should include a literature review, but a dissertation as a whole can be a literature review. In this section we discuss literature reviews for the whole dissertation.

What are the benefits of a literature review?

There are advantages and disadvantages to any approach. The advantages of conducting a literature review include accessibility, deeper understanding of your chosen topic, identifying experts and current research within that area, and answering key questions about current research. The disadvantages might include not providing new information on the subject and, depending on the subject area, you may have to include information that is out of date.

How do I write it?

A literature review is often split into chapters, you can choose if these chapters have titles that represent the information within them, or call them chapter 1, chapter 2, ect. A regular format for a literature review is:

Introduction (including methodology)

This particular example is split into 6 sections, however it may be more or less depending on your topic.

Literature Reviews Further Reading

• << Previous: Primary Research
• Next: Methodology >>
• Last Updated: Oct 18, 2023 9:32 AM
• URL: https://libguides.derby.ac.uk/c.php?g=690330

Get Started

Take the first step and invest in your future.

Online Programs

Offering flexibility & convenience in 51 online degrees & programs.

Prairie Stars

Featuring 15 intercollegiate NCAA Div II athletic teams.

Find your Fit

UIS has over 85 student and 10 greek life organizations, and many volunteer opportunities.

Arts & Culture

Celebrating the arts to create rich cultural experiences on campus.

Give Like a Star

Your generosity helps fuel fundraising for scholarships, programs and new initiatives.

Bragging Rights

UIS was listed No. 1 in Illinois and No. 3 in the Midwest in 2023 rankings.

• Quick links Applicants & Students Important Apps & Links Alumni Faculty and Staff Community Admissions How to Apply Cost & Aid Tuition Calculator Registrar Orientation Visit Campus Academics Register for Class Programs of Study Online Degrees & Programs Graduate Education International Student Services Study Away Student Support UIS Life Dining Diversity & Inclusion Get Involved Health & Wellness COVID-19 United in Safety Residence Life Student Life Programs UIS Connection Important Apps UIS Mobile App Advise U Canvas myUIS i-card Balance Pay My Bill - UIS Bursar Self-Service Registration Email Resources Bookstore Box Information Technology Services Library Orbit Policies Webtools Get Connected Area Information Calendar Campus Recreation Departments & Programs (A-Z) Parking UIS Newsroom Connect & Get Involved Update your Info Alumni Events Alumni Networks & Groups Volunteer Opportunities Alumni Board News & Publications Featured Alumni Alumni News UIS Alumni Magazine Resources Order your Transcripts Give Back Alumni Programs Career Development Services & Support Accessibility Services Campus Services Campus Police Facilities & Services Registrar Faculty & Staff Resources Website Project Request Web Services Training & Tools Academic Impressions Career Connect CSA Reporting Cybersecurity Training Faculty Research FERPA Training Website Login Campus Resources Newsroom Campus Calendar Campus Maps i-Card Human Resources Public Relations Webtools Arts & Events UIS Performing Arts Center Visual Arts Gallery Event Calendar Sangamon Experience Center for Lincoln Studies ECCE Speaker Series Community Engagement Center for State Policy and Leadership Illinois Innocence Project Innovate Springfield Central IL Nonprofit Resource Center NPR Illinois Community Resources Child Protection Training Academy Office of Electronic Media University Archives/IRAD Institute for Illinois Public Finance

Request Info

Literature Review

• Request Info Request info for....     Undergraduate/Graduate     Online     Study Away     Continuing & Professional Education     International Student Services     General Inquiries

The purpose of a literature review is to collect relevant, timely research on your chosen topic, and synthesize it into a cohesive summary of existing knowledge in the field. This then prepares you for making your own argument on that topic, or for conducting your own original research.

Depending on your field of study, literature reviews can take different forms. Some disciplines require that you synthesize your sources topically, organizing your paragraphs according to how your different sources discuss similar topics. Other disciplines require that you discuss each source in individual paragraphs, covering various aspects in that single article, chapter, or book.

Within your review of a given source, you can cover many different aspects, including (if a research study) the purpose, scope, methods, results, any discussion points, limitations, and implications for future research. Make sure you know which model your professor expects you to follow when writing your own literature reviews.

Tip : Literature reviews may or may not be a graded component of your class or major assignment, but even if it is not, it is a good idea to draft one so that you know the current conversations taking place on your chosen topic. It can better prepare you to write your own, unique argument.

Benefits of Literature Reviews

• Literature reviews allow you to gain familiarity with the current knowledge in your chosen field, as well as the boundaries and limitations of that field.
• Literature reviews also help you to gain an understanding of the theory(ies) driving the field, allowing you to place your research question into context.
• Literature reviews provide an opportunity for you to see and even evaluate successful and unsuccessful assessment and research methods in your field.
• Literature reviews prevent you from duplicating the same information as others writing in your field, allowing you to find your own, unique approach to your topic.
• Literature reviews give you familiarity with the knowledge in your field, giving you the chance to analyze the significance of your additional research.

Choosing Your Sources

When selecting your sources to compile your literature review, make sure you follow these guidelines to ensure you are working with the strongest, most appropriate sources possible.

Topically Relevant

Find sources within the scope of your topic

Appropriately Aged

Find sources that are not too old for your assignment

Find sources whose authors have authority on your topic

Appropriately “Published”

Find sources that meet your instructor’s guidelines (academic, professional, print, etc.)

Tip:  Treat your professors and librarians as experts you can turn to for advice on how to locate sources. They are a valuable asset to you, so take advantage of them!

Organizing Your Literature Review

Synthesizing topically.

Some assignments require discussing your sources together, in paragraphs organized according to shared topics between them.

For example, in a literature review covering current conversations on Alison Bechdel’s  Fun Home , authors may discuss various topics including:

• her graphic style
• her allusions to various literary texts
• her story’s implications regarding LGBT experiences in 20 th  century America.

In this case, you would cluster your sources on these three topics. One paragraph would cover how the sources you collected dealt with Bechdel’s graphic style. Another, her allusions. A third, her implications.

Each of these paragraphs would discuss how the sources you found treated these topics in connection to one another. Basically, you compare and contrast how your sources discuss similar issues and points.

To determine these shared topics, examine aspects including:

• Definition of terms
• Common ground
• Issues that divide
• Rhetorical context

Summarizing Individually

Depending on the assignment, your professor may prefer that you discuss each source in your literature review individually (in their own, separate paragraphs or sections). Your professor may give you specific guidelines as far as what to cover in these paragraphs/sections.

If, for instance, your sources are all primary research studies, here are some aspects to consider covering:

• Participants
• Limitations
• Implications
• Significance

Each section of your literature review, in this case, will identify all of these elements for each individual article.

You may or may not need to separate your information into multiple paragraphs for each source. If you do, using proper headings in the appropriate citation style (APA, MLA, etc.) will help keep you organized.

If you are writing a literature review as part of a larger assignment, you generally do not need an introduction and/or conclusion, because it is embedded within the context of your larger paper.

If, however, your literature review is a standalone assignment, it is a good idea to include some sort of introduction and conclusion to provide your reader with context regarding your topic, purpose, and any relevant implications or further questions. Make sure you know what your professor is expecting for your literature review’s content.

Typically, a literature review concludes with a full bibliography of your included sources. Make sure you use the style guide required by your professor for this assignment.

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Why is it important to do a literature review in research?

The importance of scientific communication in the healthcare industry

The Importance and Role of Biostatistics in Clinical Research

 “A substantive, thorough, sophisticated literature review is a precondition for doing substantive, thorough, sophisticated research”. Boote and Baile 2005

Authors of manuscripts treat writing a literature review as a routine work or a mere formality. But a seasoned one knows the purpose and importance of a well-written literature review.  Since it is one of the basic needs for researches at any level, they have to be done vigilantly. Only then the reader will know that the basics of research have not been neglected.

The aim of any literature review is to summarize and synthesize the arguments and ideas of existing knowledge in a particular field without adding any new contributions.   Being built on existing knowledge they help the researcher to even turn the wheels of the topic of research.  It is possible only with profound knowledge of what is wrong in the existing findings in detail to overpower them.  For other researches, the literature review gives the direction to be headed for its success. 

The common perception of literature review and reality:

As per the common belief, literature reviews are only a summary of the sources related to the research. And many authors of scientific manuscripts believe that they are only surveys of what are the researches are done on the chosen topic.  But on the contrary, it uses published information from pertinent and relevant sources like

• Scholarly books
• Scientific papers
• Latest studies in the field
• Established school of thoughts
• Relevant articles from renowned scientific journals

and many more for a field of study or theory or a particular problem to do the following:

• Summarize into a brief account of all information
• Synthesize the information by restructuring and reorganizing
• Critical evaluation of a concept or a school of thought or ideas
• Familiarize the authors to the extent of knowledge in the particular field
• Encapsulate
• Compare & contrast

By doing the above on the relevant information, it provides the reader of the scientific manuscript with the following for a better understanding of it:

• It establishes the authors’  in-depth understanding and knowledge of their field subject
• It gives the background of the research
• Portrays the scientific manuscript plan of examining the research result
• Illuminates on how the knowledge has changed within the field
• Highlights what has already been done in a particular field
• Information of the generally accepted facts, emerging and current state of the topic of research
• Identifies the research gap that is still unexplored or under-researched fields
• Demonstrates how the research fits within a larger field of study
• Provides an overview of the sources explored during the research of a particular topic

Importance of literature review in research:

The importance of literature review in scientific manuscripts can be condensed into an analytical feature to enable the multifold reach of its significance.  It adds value to the legitimacy of the research in many ways:

• Provides the interpretation of existing literature in light of updated developments in the field to help in establishing the consistency in knowledge and relevancy of existing materials
• It helps in calculating the impact of the latest information in the field by mapping their progress of knowledge.
• It brings out the dialects of contradictions between various thoughts within the field to establish facts
• The research gaps scrutinized initially are further explored to establish the latest facts of theories to add value to the field
• Indicates the current research place in the schema of a particular field
• Provides information for relevancy and coherency to check the research
• Apart from elucidating the continuance of knowledge, it also points out areas that require further investigation and thus aid as a starting point of any future research
• Justifies the research and sets up the research question
• Sets up a theoretical framework comprising the concepts and theories of the research upon which its success can be judged
• Helps to adopt a more appropriate methodology for the research by examining the strengths and weaknesses of existing research in the same field
• Increases the significance of the results by comparing it with the existing literature
• Provides a point of reference by writing the findings in the scientific manuscript
• Helps to get the due credit from the audience for having done the fact-finding and fact-checking mission in the scientific manuscripts
• The more the reference of relevant sources of it could increase more of its trustworthiness with the readers
• Helps to prevent plagiarism by tailoring and uniquely tweaking the scientific manuscript not to repeat other’s original idea
• By preventing plagiarism , it saves the scientific manuscript from rejection and thus also saves a lot of time and money
• Helps to evaluate, condense and synthesize gist in the author’s own words to sharpen the research focus
• Helps to compare and contrast to  show the originality and uniqueness of the research than that of the existing other researches
• Rationalizes the need for conducting the particular research in a specified field
• Helps to collect data accurately for allowing any new methodology of research than the existing ones
• Enables the readers of the manuscript to answer the following questions of its readers for its better chances for publication
• What do the researchers know?
• What do they not know?
• Is the scientific manuscript reliable and trustworthy?
• What are the knowledge gaps of the researcher?

22. It helps the readers to identify the following for further reading of the scientific manuscript:

• What has been already established, discredited and accepted in the particular field of research
• Areas of controversy and conflicts among different schools of thought
• Unsolved problems and issues in the connected field of research
• The emerging trends and approaches
• How the research extends, builds upon and leaves behind from the previous research

A profound literature review with many relevant sources of reference will enhance the chances of the scientific manuscript publication in renowned and reputed scientific journals .

References:

http://www.math.montana.edu/jobo/phdprep/documents/phd6.pdf

journal Publishing services  |  Scientific Editing Services  |  Medical Writing Services  |  scientific research writing service  |  Scientific communication services

Related Topics:

Meta Analysis

Scientific Research Paper Writing

Medical Research Paper Writing

Scientific Communication in healthcare

pubrica academy

Related posts.

Statistical analyses of case-control studies

PUB - Selecting material (e.g. excipient, active pharmaceutical ingredient) for drug development

Selecting material (e.g. excipient, active pharmaceutical ingredient, packaging material) for drug development

PUB - Health Economics of Data Modeling

Health economics in clinical trials

Comments are closed.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Prevent plagiarism. Run a free check.

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

A faster, more affordable way to improve your paper

Scribbr’s new AI Proofreader checks your document and corrects spelling, grammar, and punctuation mistakes with near-human accuracy and the efficiency of AI!

Proofread my paper

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved November 3, 2023, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

• Research Process

Literature Review in Research Writing

• 4 minute read
• 408.7K views

Table of Contents

Research on research? If you find this idea rather peculiar, know that nowadays, with the huge amount of information produced daily all around the world, it is becoming more and more difficult to keep up to date with all of it. In addition to the sheer amount of research, there is also its origin. We are witnessing the economic and intellectual emergence of countries like China, Brazil, Turkey, and United Arab Emirates, for example, that are producing scholarly literature in their own languages. So, apart from the effort of gathering information, there must also be translators prepared to unify all of it in a single language to be the object of the literature survey. At Elsevier, our team of translators is ready to support researchers by delivering high-quality scientific translations , in several languages, to serve their research – no matter the topic.

What is a literature review?

A literature review is a study – or, more accurately, a survey – involving scholarly material, with the aim to discuss published information about a specific topic or research question. Therefore, to write a literature review, it is compulsory that you are a real expert in the object of study. The results and findings will be published and made available to the public, namely scientists working in the same area of research.

How to Write a Literature Review

First of all, don’t forget that writing a literature review is a great responsibility. It’s a document that is expected to be highly reliable, especially concerning its sources and findings. You have to feel intellectually comfortable in the area of study and highly proficient in the target language; misconceptions and errors do not have a place in a document as important as a literature review. In fact, you might want to consider text editing services, like those offered at Elsevier, to make sure your literature is following the highest standards of text quality. You want to make sure your literature review is memorable by its novelty and quality rather than language errors.

Writing a literature review requires expertise but also organization. We cannot teach you about your topic of research, but we can provide a few steps to guide you through conducting a literature review:

• Choose your topic or research question: It should not be too comprehensive or too limited. You have to complete your task within a feasible time frame.
• Set the scope: Define boundaries concerning the number of sources, time frame to be covered, geographical area, etc.
• Decide which databases you will use for your searches: In order to search the best viable sources for your literature review, use highly regarded, comprehensive databases to get a big picture of the literature related to your topic.
• Search, search, and search: Now you’ll start to investigate the research on your topic. It’s critical that you keep track of all the sources. Start by looking at research abstracts in detail to see if their respective studies relate to or are useful for your own work. Next, search for bibliographies and references that can help you broaden your list of resources. Choose the most relevant literature and remember to keep notes of their bibliographic references to be used later on.
• Review all the literature, appraising carefully it’s content: After reading the study’s abstract, pay attention to the rest of the content of the articles you deem the “most relevant.” Identify methodologies, the most important questions they address, if they are well-designed and executed, and if they are cited enough, etc.

If it’s the first time you’ve published a literature review, note that it is important to follow a special structure. Just like in a thesis, for example, it is expected that you have an introduction – giving the general idea of the central topic and organizational pattern – a body – which contains the actual discussion of the sources – and finally the conclusion or recommendations – where you bring forward whatever you have drawn from the reviewed literature. The conclusion may even suggest there are no agreeable findings and that the discussion should be continued.

Why are literature reviews important?

Literature reviews constantly feed new research, that constantly feeds literature reviews…and we could go on and on. The fact is, one acts like a force over the other and this is what makes science, as a global discipline, constantly develop and evolve. As a scientist, writing a literature review can be very beneficial to your career, and set you apart from the expert elite in your field of interest. But it also can be an overwhelming task, so don’t hesitate in contacting Elsevier for text editing services, either for profound edition or just a last revision. We guarantee the very highest standards. You can also save time by letting us suggest and make the necessary amendments to your manuscript, so that it fits the structural pattern of a literature review. Who knows how many worldwide researchers you will impact with your next perfectly written literature review.

Know more: How to Find a Gap in Research .

Language Editing Services by Elsevier Author Services:

What is a Research Gap

• Manuscript Preparation

Types of Scientific Articles

You may also like.

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Navigating the Reproducibility Crisis: A Guide to Analytical Method Validation

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Oct 31, 2023 3:00 PM
• URL: https://guides.library.yale.edu/YSNDoctoral

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Literature Reviews

Overview of literature reviews.

• Systematic Reviews
• Systematic Reviews for Social Sciences
• Checklists, Guides

What Type of Review is Right for You?

• Decision tree for review types from Cornell University Library

'Literature review' is a generic term that is often used to describe a range of different review types. For a class assignment, you may be required to review academic literature based on a topic of interest and write about the sources you selected. Or, if your are working on a research project, you may need to conduct a comprehensive search of the literature to write a literature review or a literature review chapter for a thesis or dissertation.

Listed below are common review types with brief descriptions for a quick comparison of characteristics. Citations are included for follow up and more details.

• Traditional Review (Integrative/Narrative) Grant & Booth (2009) describe 14 review types. They note the aim of this type of literature review is to examine the current/recent literature, so it may not include comprehensive searches and often it describes only a group of selected sources. 

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26 (2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

• Systematic Review  Aims to be comprehensive, adheres to transparent procedures, and provides evidence synthesis that can be used in intervention decisions and policy making.
• Systematized Review   Incorporates some systematic review procedures that can be included as part of a narrative (traditional) review.
• Meta-Analysis   Uses statistical methods to evaluate relevant research studies and may be part of a systematic review.
• Rapid Review   Applies systematic review methods but sets a time limit on locating and appraising sources for a shorten timeframe.
• Scoping Review   Explores research questions to map key concepts, evidence, and gaps in the literature and may take longer to complete than a systematic review,
• Umbrella Review  
• Compiles evidence from multiple reviews based on a broad problem for which there are competing interventions.
• Next: Systematic Reviews >>
• Last Updated: Nov 1, 2023 3:10 PM
• URL: https://guides.ucf.edu/literaturereviews

Advantages and disadvantages of literature review

This comprehensive article explores some of the advantages and disadvantages of literature review in research. Reviewing relevant literature is a key area in research, and indeed, it is a research activity in itself. It helps researchers investigate a particular topic in detail. However, it has some limitations as well.

What is literature review?

In order to understand the advantages and disadvantages of literature review, it is important to understand what a literature review is and how it differs from other methods of research. According to Jones and Gratton (2009) a literature review essentially consists of critically reading, evaluating, and organising existing literature on a topic to assess the state of knowledge in the area. It is sometimes called critical review.

A literature review is a select analysis of existing research which is relevant to a researcher’s selected topic, showing how it relates to their investigation. It explains and justifies how their investigation may help answer some of the questions or gaps in the chosen area of study (University of Reading, 2022).

A literature review is a term used in the field of research to describe a systematic and methodical investigation of the relevant literature on a particular topic. In other words, it is an analysis of existing research on a topic in order to identify any relevant studies and draw conclusions about the topic.

A literature review is not the same as a bibliography or a database search. Rather than simply listing references to sources of information, a literature review involves critically evaluating and summarizing existing research on a topic. As such, it is a much more detailed and complex process than simply searching databases and websites, and it requires a lot of effort and skills.

Advantages of literature review

Information synthesis

A literature review is a very thorough and methodical exercise. It can be used to synthesize information and draw conclusions about a particular topic. Through a careful evaluation and critical summarization, researchers can draw a clear and comprehensive picture of the chosen topic.

Familiarity with the current knowledge

According to the University of Illinois (2022), literature reviews allow researchers to gain familiarity with the existing knowledge in their selected field, as well as the boundaries and limitations of that field.

Creation of new body of knowledge

One of the key advantages of literature review is that it creates new body of knowledge. Through careful evaluation and critical summarisation, researchers can create a new body of knowledge and enrich the field of study.

Answers to a range of questions

Literature reviews help researchers analyse the existing body of knowledge to determine the answers to a range of questions concerning a particular subject.

Disadvantages of literature review

Time consuming

As a literature review involves collecting and evaluating research and summarizing the findings, it requires a significant amount of time. To conduct a comprehensive review, researchers need to read many different articles and analyse a lot of data. This means that their review will take a long time to complete.

Lack of quality sources  

Researchers are expected to use a wide variety of sources of information to present a comprehensive review. However, it may sometimes be challenging for them to identify the quality sources because of the availability of huge numbers in their chosen field. It may also happen because of the lack of past empirical work, particularly if the selected topic is an unpopular one.

Descriptive writing

One of the major disadvantages of literature review is that instead of critical appreciation, some researchers end up developing reviews that are mostly descriptive. Their reviews are often more like summaries of the work of other writers and lack in criticality. It is worth noting that they must go beyond describing the literature.

Key features of literature review

Clear organisation

A literature review is typically a very critical and thorough process. Universities usually recommend students a particular structure to develop their reviews. Like all other academic writings, a review starts with an introduction and ends with a conclusion. Between the beginning and the end, researchers present the main body of the review containing the critical discussion of sources.

No obvious bias

A key feature of a literature review is that it should be very unbiased and objective. However, it should be mentioned that researchers may sometimes be influenced by their own opinions of the world.

Proper citation

One of the key features of literature review is that it must be properly cited. Researchers should include all the sources that they have used for information. They must do citations and provide a reference list by the end in line with a recognized referencing system such as Harvard.

To conclude this article, it can be said that a literature review is a type of research that seeks to examine and summarise existing research on a particular topic. It is an essential part of a dissertation/thesis. However, it is not an easy thing to handle by an inexperienced person. It also requires a lot of time and patience.

Hope you like this ‘Advantages and disadvantages of literature review’. Please share this with others to support our research work.

Other useful articles:

How to evaluate website content

Advantages and disadvantages of primary and secondary research

Advantages and disadvantages of simple random sampling

Last update: 08 May 2022

References:

Jones, I., & Gratton, C. (2009) Research Methods for Sports Shttps://www.howandwhat.net/new/evaluate-website-content/tudies, 2 nd edition, London: Routledge

University of Illinois (2022) Literature review, available at: https://www.uis.edu/learning-hub/writing-resources/handouts/learning-hub/literature-review (accessed 08 May 2022)

University of Reading (2022) Literature reviews, available at: https://libguides.reading.ac.uk/literaturereview/starting (accessed 07 May 2022)

Author: M Rahman

M Rahman writes extensively online and offline with an emphasis on business management, marketing, and tourism. He is a lecturer in Management and Marketing. He holds an MSc in Tourism & Hospitality from the University of Sunderland. Also, graduated from Leeds Metropolitan University with a BA in Business & Management Studies and completed a DTLLS (Diploma in Teaching in the Life-Long Learning Sector) from London South Bank University.

Related Posts

How to be a good team player, what is halal tourism, pros and cons of cash cows.

• Open access
• Published: 04 September 2016

Improving the peer review of narrative literature reviews

• Jennifer A. Byrne 1 , 2  

Research Integrity and Peer Review volume  1 , Article number:  12 ( 2016 ) Cite this article

21k Accesses

58 Citations

29 Altmetric

Metrics details

As the size of the published scientific literature has increased exponentially over the past 30 years, review articles play an increasingly important role in helping researchers to make sense of original research results. Literature reviews can be broadly classified as either “systematic” or “narrative”. Narrative reviews may be broader in scope than systematic reviews, but have been criticised for lacking synthesis and rigour. The submission of more scientific manuscripts requires more researchers acting as peer reviewers, which requires adding greater numbers of new reviewers to the reviewing population over time. However, whereas there are many easily accessible guides for reviewers of primary research manuscripts, there are few similar resources to assist reviewers of narrative reviews. Here, I summarise why literature reviews are valued by their diverse readership and how peer reviewers with different levels of content expertise can improve the reliability and accessibility of narrative review articles. I then provide a number of recommendations for peer reviewers of narrative literature reviews, to improve the integrity of the scientific literature, while also ensuring that narrative review articles meet the needs of both expert and non-expert readers.

Peer Review reports

Over the past 30 years, the size of the published scientific literature has expanded exponentially [ 1 ]. While it has been argued that this rate of expansion is unsustainable [ 2 ], underlying factors such as greater numbers of scientists and scientific journals [ 3 ] are unlikely to change in the short term. The submission of more manuscripts for publication requires more peer reviewers, yet the current demand for capable, available manuscript reviewers is not being met [ 3 ]. This has serious adverse consequences for the validity of published research and overall trust in science [ 3 ].

Review articles help both experts and non-experts to make sense of the increasing volume of original publications [ 4 , 5 ]. Busy clinicians have a particular reliance upon review articles, because of their constant need for reliable, up-to-date information, yet limited available time [ 6 ]. Literature reviews can also help other content experts such as researchers and policymakers to identify gaps in their own reading and knowledge. However, literature reviews are also sought by readers with little or no prior understanding of the reviewed topic, such as researchers seeking to rapidly triage results from high-throughput analyses and students for whom literature reviews can represent entry points into a new field. For the benefit of both expert and non-expert readers, it is essential that review articles accurately synthesise the relevant literature in a comprehensive, transparent and objective manner [ 7 , 8 ].

Numbers of review articles are increasing in fields where this has been measured [ 4 ], as is the diversity of review types published [ 9 , 10 ]. Although there are now many review sub-types that can be distinguished based upon the literature search, appraisal, synthesis and analysis methods used [ 9 , 10 ], review articles can be broadly classified as either “systematic” or “narrative” [ 5 , 11 ]. Systematic reviews take defined approaches to the identification and synthesis of study findings and include other review sub-types such as evidence maps [ 12 ]. The systematic review is considered to be the gold standard of evidence synthesis, but also carries the potential disadvantages of narrow scope [ 11 ], and requiring more time and resources to prepare and update [ 7 ]. Narrative reviews, also referred to as “traditional reviews” [ 5 ] and “literature reviews” [ 9 ], constitute the majority of review articles published in some fields [ 7 ]. Other review sub-types, such as rapid and scoping reviews also present information in a narrative format [ 9 ]. Narrative reviews have been criticised for rarely employing peer-reviewed methodologies, or duplicate curation of evidence [ 5 ], and for often failing to disclose study inclusion criteria [ 11 ]. Despite these limitations, narrative reviews remain frequent within the literature, as they offer breadth of literature coverage and flexibility to deal with evolving knowledge and concepts [ 11 ]. In this article, I will provide advice regarding the peer review of narrative reviews, and the advice presented aims to be broadly applicable. I will not attempt to provide advice regarding the peer review of systematic reviews [ 13 , 14 ].

Given the broad readership of literature reviews, content and methodology experts as well as reviewers with less directly relevant expertise can play important roles in the peer-review process [ 15 ]. Peer reviewers with related content expertise are best placed to assess the reliability of the information presented, while other reviewers can ensure that this information remains accessible to readers with different levels of prior knowledge. However, whereas there are easily accessible guides for reviewers of primary research manuscripts [ 16 , 17 ], there are few similar resources available for reviewers of literature reviews [ 15 , 18 ]. This article therefore proposes a number of recommendations for peer reviewers (Table  1 ) to ensure that narrative literature review articles make the best possible contributions to their fields, while also meeting their readers’ often diverse needs.

Ask whether the literature review justifies its place in the literature

Lower than expected ratios between numbers of original publications and review articles suggest excessive numbers of reviews in some fields, which may contribute to the very problem that review articles aim to solve [ 4 ]. With rapidly rising publication rates in many fields [ 2 ], even content-expert peer reviewers should check publication databases for similar and/or overlapping review articles as part of the peer-review process. Pre-empting such scrutiny, authors should clearly define the review’s scope and what it intends to achieve [ 8 ]. If there have been other recent reviews of the same or similar topics, the authors should explain how their manuscript is unique. This could be through combining literature from related fields, by updating existing reviews in light of new research evidence [ 8 ], or because published reviews may have been subject to bias. A clear definition of a review’s scope is a recognised tool to reduce evidence selection bias [ 19 ]. Review authors can also define their subject by referring to literature reviews of related topics that will not be explored in depth. These definitions and statements should form part of an overall narrative structure that helps readers to anticipate and understand the information presented [ 20 ].

Ask whether the literature searches conducted were clearly defined

A criticism frequently levelled at traditional or narrative reviews is that they do not always state or follow rules regarding literature searches [ 5 , 7 , 11 ]. Providing evidence that comprehensive literature searches have been conducted, preferably according to pre-defined eligibility criteria [ 19 ], increases confidence that the review’s findings and conclusions are reliable, and have not been subject to selection bias. Ideally, any literature search choices made by the authors should be clearly stated, transparent and reproducible [ 11 ].

Check for citation breadth and balance

Consider whether the authors have cited a comprehensive range of literature or whether they have tended to cite papers that support their own point of view. If there are important papers that have not been cited, suggest to the authors that these be added, and explain why. If only a limited number of articles can be cited due to the journal’s requirements, check that these studies are representative of those available.

Where possible, verify that information has been summarised correctly

Many different types of citation errors can be identified in the research literature [ 21 , 22 ], and these may occur regardless of the journal impact factor [ 22 ]. The increasing size and complexity of primary reports [ 3 ] also render data extraction and summary more challenging. Realistically, it is unlikely that individual peer reviewers will have detailed knowledge of any full review topic [ 19 ]. Nonetheless, if you are a content expert, take time to cross-reference at least some individual statements to citations, for the particular benefit of non-expert readers. If your level of expertise means that you are unable to verify the accuracy of particular sections of the review, you should indicate this to your editor. Peer reviewers can also ask about data extraction methods, if these were not described in the manuscript. Adopting systematic review practices, such as duplicate independent data extraction, or independent data extraction and validation, can reduce content errors and increase reliability [ 19 ].

Check that original references have been cited

Authors sometimes incorrectly cite original studies, both in original manuscripts and reviews [ 23 , 24 ]. While checking the content, ask whether descriptions of original findings were referenced accordingly, as opposed to being incorrectly attributed to reviews [ 23 ].

Consider how studies were critically evaluated

Beyond correct data summary, narrative literature reviews should include critical data appraisal and some level of data synthesis. How this should be done varies according to the review scope and methodology [ 9 , 10 , 19 ]. While some narrative reviews reasonably focus on breadth as opposed to depth of literature coverage [ 10 ], limited or poor data appraisal risks placing undue emphasis on poor quality research [ 9 ]. Evaluating at least some aspects of the methods used by individual studies can improve reliability [ 7 ]. Similarly, ask how the authors have interpreted conflicting findings or studies with apparently outlying results [ 9 , 11 ].

Evaluate whether tables/figures/diagrams support the text

While not all literature reviews need to include figures or tables, these can help to summarise findings and make key messages clearer. Some detailed information may be best presented in tables, with a shorter summary within the text. Tables can improve the availability of quantitative data for cross-checking, better demonstrate the results of qualitative or quantitative data synthesis, and reassure both peer reviewers and readers that comprehensive, objective analyses have been performed. If figures or tables are included, these need to be original; otherwise, the authors need to have obtained permission to reproduce these from an original source.

Consider whether the review will help someone entering the field

Literature reviews are not always read by subject experts, and it is important that the peer-review process considers this. Reviewers who are not direct content experts may valuably request clarification of nomenclature and/or historical issues that may have seemed too obvious for the authors to have explained. Summary diagrams suggested by peer reviewers may help make a literature review more accessible to a broader audience.

Ask whether the review expands the body of knowledge

Ultimately, the goal of a literature review should be to further the body of knowledge [ 18 ]. Extending or developing ideas is clearly a difficult task, and is often the weakest section of a review [ 25 ]. Consider therefore whether the authors have derived and clearly presented new ideas and/or new research directions from any identified knowledge gaps. Having read the manuscript with fresh eyes, peer reviewers may have valuable ideas to contribute.

Do not forget the rules for reviewing manuscripts in general

The review of literature reviews has some particular considerations, but all the usual manuscript review rules also apply, such as managing conflicts of interest and allocating appropriate time [ 16 , 17 ]. Try to separate the assessment of language and grammar from the more important assessment of scientific quality and remain aware that expert reviewers risk bringing their own biases to the peer-review process [ 15 ].

Conclusions

More quality peer reviewers are needed within the scientific community [ 3 ], including those with the capacity and confidence to review narrative literature reviews. Although it has been difficult to identify predictors of peer-reviewer performance and effective training methods, younger reviewer age has been reproducibly associated with better quality manuscript reviews [ 26 , 27 ]. This association suggests that peer reviewers should be recruited relatively early in their careers, and encouraged to participate widely in manuscript review. Associations between younger peer-reviewer age and better manuscript reviews may also highlight the need for regular training, to ensure that the peer-review community remains up-to-date regarding new approaches to editing or reviewing manuscripts. Indeed, a recent industry survey reported that over three quarters of researchers were interested in further reviewer training [ 28 ]. I therefore hope that this article will add to existing resources [ 29 ] to encourage less experienced peer reviewers to extend their efforts towards narrative literature reviews.

Bornmann L, Mutz R. Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J Assoc Inform Sci Tech. 2015;66(11):2215–22.

Google Scholar  

Pautasso M. Publication growth in biological sub-fields: patterns, predictability and sustainability. Sustainability. 2012;4(12):3234–47.

Article   Google Scholar  

Siebert S, Machesky LM, Insall RH. Overflow in science and its implications for trust. Elife. 2015;4: doi: 10.7554/eLife.10825 .

Ketcham CM, Crawford JM. The impact of review articles. Lab Invest. 2007;87(12):1174–85.

Dijkers MP. Task Force on Systematic Reviews and Guidelines. The value of traditional reviews in the era of systematic reviewing. Am J Phys Med Rehabil. 2009;88(5):423–30.

McAlister FA, Clark HD, van Walraven C, Straus SE, Lawson FM, Moher D, et al. The medical review article revisited: has the science improved? Ann Intern Med. 1999;131(12):947–51.

Haddaway NR, Woodcock P, Macura B, Collins A. Making literature reviews more reliable through application of lessons from systematic reviews. Conserv Biol. 2015;29(6):1596–605.

Pautasso M. Ten simple rules for writing a literature review. PLoS Comput Biol. 2013;9(7):e1003149.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inform Lib J. 2009;26(2):91–108.

Paré G, Trudel M-C, Jaana M, Kitsiou S. Synthesizing information systems knowledge: a typology of literature reviews. Inform Management. 2015;52(2):183–99.

Collins JA, Fauser BCJM. Balancing the strengths of systematic and narrative reviews. Hum Reprod Update. 2005;11(2):103–4.

Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016;5:28.

Higgins JPT, Green S. Handbook for systematic reviews of interventions. The Cochrane Collaboration, John Wiley & Sons Ltd; 2011.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700.

Oxman AJ. Checklists for review articles. BMJ. 1994;309(6955):648–51.

Bourne PE, Korngreen A. Ten simple rules for reviewers. PLoS Comput Biol. 2006;2(9):e110.

Nicholas KA, Gordon W. A quick guide to writing a solid peer review. Eos. 2011;92(28):233–4.

Jennex ME. Literature reviews and the review process: an editor-in-chief’s perspective. CAIS. 2015;36:8.

O’Connor A, Sargeant J. Research synthesis in veterinary science: narrative reviews, systematic reviews and meta-analysis. Vet J. 2015;206(3):261–7.

Docherty M, Smith R. The case for structuring the discussion of scientific papers. BMJ. 1999;318(7193):1224–5.

Davids JR, Weigl DM, Edmonds JP, Blackhurst DW. Reference accuracy in peer-reviewed pediatric orthopaedic literature. J Bone Joint Surg Am. 2010;92(5):1155–61.

Awrey J, Inaba K, Barmparas G, Recinos G, Teixeira PG, Chan LS, et al. Reference accuracy in the general surgery literature. World J Surg. 2011;35(3):475–9.

Gavras H. Inappropriate attribution: the “lazy author syndrome”. Am J Hypertens. 2002;15(9):831.

Katz TJ. Propagation of errors in review articles. Science. 2006;313(5791):1236.

Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. MIS Q. 2002;26:2.

Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA. 1998;280(3):231–3.

Callaham ML, Tercier J. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Med. 2007;4(1):e40.

Warne V. Rewarding reviewers- sense or sensibility? A Wiley study explained. Learned Pub. 2016;29(1):41–50.

COPE Ethical Guidelines for Peer Reviewers. Available: http://publicationethics.org/resources/guidelines-new/cope-ethical-guidelines-peer-reviewers . Accessed 10 Aug, 2016.

Download references

Acknowledgements

I thank Dr Mona Shehata (Princess Margaret Cancer Centre, Toronto, Canada) for discussions, Ms Sarah Frost for critical reading, reviewers of this manuscript for many constructive comments, and reviewers of past publications for feedback which also contributed towards the development of this manuscript.

Not applicable.

Availability of data and materials

Authors’ contributions.

JAB drafted, wrote and approved the final manuscript.

Competing interests

The author declares that she has no competing interests.

Consent for publication

Ethics approval and consent to participate, author information, authors and affiliations.

Molecular Oncology Laboratory, Children’s Cancer Research Unit, Kids Research Institute, The Children’s Hospital at Westmead, Locked Bag 4001, Westmead, 2145, NSW, Australia

Jennifer A. Byrne

The University of Sydney Discipline of Child and Adolescent Health, The Children’s Hospital at Westmead, Locked Bag 4001, Westmead, 2145, NSW, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jennifer A. Byrne .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Cite this article.

Byrne, J.A. Improving the peer review of narrative literature reviews. Res Integr Peer Rev 1 , 12 (2016). https://doi.org/10.1186/s41073-016-0019-2

Download citation

Received : 17 June 2016

Accepted : 05 August 2016

Published : 04 September 2016

DOI : https://doi.org/10.1186/s41073-016-0019-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Peer review
• Narrative literature review

Research Integrity and Peer Review

ISSN: 2058-8615

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Email: [email protected]

• Instagram /

Home / How Can a Researcher Take Advantage of a Systematic Literature Review When Conducting Research ?

• How Can a Researcher Take Advantage of a Systematic Literature Review When Conducting Research ?

A systematic literature review is a valuable tool that researchers can utilize to gather and analyze existing knowledge and evidence related to a specific research topic. It involves a comprehensive and rigorous approach to reviewing published studies, academic papers, and other relevant sources of information. By conducting a systematic literature review, researchers can gain a deeper understanding of the existing body of knowledge, identify research gaps, and inform their own research design and methodology. This blog explores the various ways in which researchers can take advantage of a systematic literature review when conducting their research, highlighting its benefits and significance in the research process. Before diving into the topic, we need to know the examples of systematic literature review topics to fully know how we can take advantage of it. So, let’s dive into it.

Examples of systematic literature review

Here are a few examples of systematic literature review topics across different disciplines;

• The impact of mindfulness-based interventions on stress reduction: A systematic literature review and meta-analysis.
• Effectiveness of cognitive-behavioral therapy in treating depression: A systematic review and meta-analysis of randomized controlled trials.
• The role of nutrition in preventing cardiovascular diseases: A systematic literature review of observational studies and clinical trials.
• The effects of exercise on cognitive function in older adults: A systematic review and meta-analysis of randomized controlled trials.
• The effectiveness of school-based bullying prevention programs: A systematic review of intervention studies.
• The impact of social media on body image dissatisfaction and disordered eating behaviors: A systematic literature review.
• The role of artificial intelligence in healthcare diagnosis and treatment: A systematic review of current applications and challenges.
• The effectiveness of virtual reality-based interventions in pain management: A systematic literature review and meta-analysis.
• The impact of parental involvement on academic achievement: A systematic review of longitudinal studies.
• The effectiveness of mindfulness-based stress reduction programs in workplace settings: A systematic review of controlled trials.                                                                                                                                                                                    These examples highlight how systematic literature reviews can be conducted across various research domains , including psychology, healthcare, education, technology, and social sciences. To know the ways in which a researcher can take advantage of a systematic literature review, we need to go to the root of it by covering some in-depth questions on it.

So the first question is:

 How does the integration of a systematic literature review into the research process enhance the identification and selection of relevant research gaps and research questions?

The integration of a systematic literature review into the research process can significantly enhance the identification and selection of relevant research gaps and research questions in several ways:

A comprehensive review of existing knowledge:  A systematic literature review involves a rigorous and systematic search and analysis of existing research studies within a specific field or topic. By conducting such a review, researchers gain a comprehensive understanding of the current state of knowledge, identifying what has already been explored and established. This enables them to identify gaps in the existing literature and areas where further research is needed.

Identification of research trends and emerging areas:  A systematic literature review allows researchers to identify research trends, emerging concepts, and innovative approaches within their field. By analyzing a wide range of studies, researchers can observe patterns, recurring themes, or novel ideas that can inform the development of relevant research questions. This process helps them identify gaps in knowledge that need to be addressed or areas where new research directions can be pursued.

The second question is

  What specific methodological frameworks or tools can researchers employ to effectively conduct a systematic literature review and extract valuable insights for their research  ?

Researchers can employ several methodological frameworks and tools to effectively conduct a systematic literature review and extract valuable insights for their research. Here are some specific frameworks and tools commonly used:

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses): PRISMA is a widely recognized framework for conducting systematic literature reviews. It provides a structured approach for transparently reporting the review process, including study selection, data extraction, and synthesis. Adhering to the PRISMA guidelines ensures the rigor and transparency of the review.

PICOS (Population, Intervention, Comparison, Outcome, Study Design): PICOS is a framework used to define the key components of a research question for a systematic literature review. It helps researchers specify the relevant population, intervention or exposure, comparison group, outcome measures, and study design characteristics to guide the search and selection of studies.

Number three in this list is,

How can researchers leverage the findings and conclusions from a systematic literature review to inform the development of a comprehensive theoretical framework or conceptual model for their research study ?

Researchers can leverage the findings and conclusions from a systematic literature review to inform the development of a comprehensive theoretical framework or conceptual model for their research study in the following ways: Identify gaps and limitations: Through a systematic literature review, researchers gain insights into existing theories, models, and frameworks in their field. They can identify gaps, limitations, or inconsistencies in the current body of knowledge. This knowledge allows researchers to position their own study within the context of existing theories and identify areas where their research can contribute to filling these gaps.

Synthesize key concepts and variables: A systematic literature review helps researchers identify the key concepts and variables that are commonly used or emphasized in previous studies. By analyzing the findings from multiple studies, researchers can extract and synthesize these concepts to form a foundation for their own theoretical framework or conceptual model. This synthesis provides a solid basis for developing hypotheses and research questions.

The fourth question is

What strategies can researchers employ to overcome potential biases or limitations associated with the inclusion and exclusion criteria used during the systematic literature review process ?

Researchers can employ several strategies to overcome potential biases or limitations associated with the inclusion and exclusion criteria used during the systematic literature review process. Here are some key strategies: Clearly define inclusion and exclusion criteria: It is crucial to establish clear and explicit criteria for including or excluding studies from the review. These criteria should be well-documented and based on the research objectives and research questions. Clearly defining the criteria helps minimize subjectivity and potential bias in the selection process.

Use multiple reviewers and inter-rater reliability checks: Involving multiple reviewers in the study selection process can help mitigate individual biases. Each reviewer independently evaluates the relevance of studies against the inclusion and exclusion criteria. To ensure consistency and minimize discrepancies, reviewers can conduct inter-rater reliability checks, where a subset of studies is assessed by multiple reviewers, and agreement rates are calculated. Discrepancies can be resolved through discussion and consensus.

The last question is

How does the utilization of advanced data mining and text analysis techniques within a systematic literature review enhance the identification of emerging research trends, theoretical frameworks, or practical applications that can shape the direction of future research?

The utilization of advanced data mining and text analysis techniques within a systematic literature review can significantly enhance the identification of emerging research trends, theoretical frameworks, and practical applications, ultimately shaping the direction of future research. Here’s how:

Efficient identification and extraction of relevant information:  Advanced data mining and text analysis techniques can automate the process of identifying and extracting relevant information from a large volume of scholarly articles. These techniques can efficiently analyze titles, abstracts, keywords, and full-text documents to identify key concepts, relationships, and patterns that may not be immediately apparent through manual review. This enables researchers to quickly and comprehensively assess the landscape of existing literature.

Identification of emerging research trends:  By applying data mining and text analysis techniques, researchers can detect emerging research trends that may not be easily identifiable through traditional methods. These techniques can identify keywords, topic clusters, co-occurrence patterns, or citation networks that highlight areas of active research or emerging topics within a field. Identifying these trends allows researchers to focus on novel and timely areas for further investigation.

In conclusion, researchers can take full advantage of a systematic literature review by using it as a foundation for their own research, enhancing their critical thinking skills, informing evidence-based decision-making, and fostering collaboration and knowledge exchange. By harnessing the power of existing knowledge, researchers can make meaningful contributions to their respective fields and drive positive change.

If you want us to cover more questions on this topic, then you can comment below. Also, don’t forget to like and share if you have gained some new insight from this blog. Thank you for reading this blog, it made my day .

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Our Advantages

• Affordable Packages
• Quality Guaranteed
• US & UK Native Writers
• 30 Days Free Revisions
• Urgent Delivery Available
• Place & Receive Order Online
• Plagiarism Report (additional costs)

We Also Offer Discounts

For more than $500, for more than $1000, for more than $2000.

• WHAT PEOPLE SAY ?
•   RECENT POSTS
•   FREQUENTLY ASKED QUESTIONS
• What is the meaning of the term “Literature Review”?
• Understanding the Importance of PRISMA in Systematic Reviews: A Deep Dive into Flow Diagram
• How to pen down literature review chapter for an empirical paper?
• Literature Review Sources – How to do literature review for your thesis or dissertation?

Will I receive personalized guidance for literature review writing?

How do you select the references for a project, who will write my literature review chapter, how long do you take to deliver a literature review, what are the citations styles you use, what are the packages you provide, how can i be sure about the security of the documents i share, get started with literature review writing.

Don't let the time slip off your hands. You may not get sufficient time for developing the chapter of literature review, let it get completed by the experts at Lit-Review. Simply define us your research topic, objectives, required length & style for writing literature review chapter. Give us your 5 minutes and place your order online to get literature review help.

© lit-review 2023. All rights reserved.

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

A Guide to Using AI Tools to Summarize Literature Reviews

Table of Contents

Needless to say, millions of scientific articles are getting published every year making it difficult for a researcher to read and comprehend all the relevant publications.

Back then, researchers used to manually conduct literature reviews by sifting through hundreds of research papers to get the significant information required for the research.

Fast forward to 2023 — things have turned out quite distinct and favorable. With the inception of AI tools, the literature review process is streamlined and researchers can summarize hundreds of research articles in mere moments. They can save time and effort by using AI tools to summarize literature reviews.

This article articulates the role of the top AI tools used to summarize literature reviews. You can also learn how AI is used as a powerful tool for summarizing scientific articles and understanding the impact of AI on academic research.

Understanding the Role of AI Tools in Literature Reviews

Before we talk about the benefits of AI tools to summarize literature reviews, let’s understand the concept of AI and how it streamlines the literature review process.

Artificial intelligence tools are trained on large language models and they are programmed to mimic human tasks like problem-solving, making decisions, understanding patterns, and more. When Artificial Intelligence and machine learning algorithms are implemented in literature reviews, they help in processing vast amounts of information, identifying highly relevant studies, and generating quick and concise summaries — TL;DR summaries.

AI has revolutionized the process of literature review by assisting researchers with powerful AI-based tools to read, analyze, compare, contrast, and extract relevant information from research articles.

By using natural language processing algorithms, AI tools can effectively identify key concepts, main arguments, and relevant findings from multiple research articles at once. This assists researchers in quickly understanding the overview of the existing literature on a respective topic, saving their valuable time and effort.

Key Benefits of Using AI Tools to Summarize Literature Review

1. best alternative to traditional literature review.

Traditional literature reviews or manual literature reviews can be incredibly time-consuming and often require weeks or even months to complete. Researchers have to sift through myriad articles manually, read them in detail, and highlight or extract relevant information. This process can be overwhelming, especially when dealing with a large number of studies.

However, with the help of AI tools, researchers can greatly save time and effort required to discover, analyze, and summarize relevant studies. AI tools with their NLP and machine learning algorithms can quickly analyze multiple research articles and generate succinct summaries. This not only improves efficiency but also allows researchers to focus on the core analysis and interpretation of the compiled insights.

2. AI tools aid in swift research discovery!

AI tools also help researchers save time in the discovery phase of literature reviews. These AI-powered tools use semantic search analysis to identify relevant studies that might go unnoticed in traditional literature review methods. Also, AI tools can analyze keywords, citations , and other metadata to prompt or suggest pertinent articles that align and correlate well with the researcher’s search query.

3. AI Tools ensure to stay up to date with the most research ideas!

Another advantage of using AI-powered tools in literature reviews is their ability to handle the ever-increasing volume of published scientific research. With the exponential growth of scientific literature, it has become increasingly challenging for researchers to keep up with the latest scientific research and biomedical innovations.

However, AI tools can automatically scan and discover new publications, ensuring that researchers stay up-to-date with the most recent developments in their field of study.

4. Improves efficiency and accuracy of Literature Reviews

The use of AI tools in literature review reduces the occurrences of human errors that may occur during traditional literature review or manual document summarization. So, literature review AI tools improve the overall efficiency and accuracy of literature reviews, ensuring that researchers can access relevant information promptly by minimizing human errors.

List of AI Tools to Streamline Literature Reviews

We have several AI-powered tools to summarize literature reviews. They utilize advanced algorithms and natural language processing techniques to analyze and summarize lengthy scientific articles.

Let's take a look at some of the most popular AI tools to summarize literature reviews.

SciSpace Literature Review

Semantic scholar, paper digest.

SciSpace Literature Review is the best AI tool for summarizing literature review. It is the go-to tool that summarizes articles in seconds. It uses natural language processing models GPT 3.5 and GPT 4.0 to generate concise summaries. It is an effective and efficient AI-powered tool to streamline the literature review process and summarize multiple research articles at once. Once you enter a keyword, research topic, or question, it initiates your literature review process by providing instant insights from the top 5 highly relevant papers at the top.

These insights are backed by citations that allow you to refer to the source. All the resultant relevant papers appear in an easy-to-digest tabular format explaining each of the sections used in the paper in different columns. You can also customize the table by adding or removing the columns according to your research needs. This is the unique feature of this literature review AI tool.

SciSpace Literature review stands out as the best AI tool to summarize literature review by providing concise TL;DR text and summaries for all the sections used in the research paper. This way, it makes the review process easier for any researcher, and could comprehend more research papers in less time.

Try SciSpace Literature Review now!

Semantic Scholar is an AI-powered search engine that helps researchers find relevant research papers based on the keyword or research topic. It works similar to Google Scholar.It helps you discover and understand scientific research by providing suitable research papers. The database has over 200 million research articles, you can filter out the results based on the field of study, author, date of publication, and journals or conferences.

They have recently released the Semantic Reader — an AI-powered tool for scientific readers that enhances the reading process. This is available in the beta version.

Try Semantic Scholar here

Paper Digest — another valuable text summarizer tool (AI-powered tool) that summarizes the literature review and helps you get to the core insights of the research paper in a few minutes! This powerful tool works pretty straightforwardly and generates summaries of research papers. You just need to input the article URL or DOI and click on “Digest” to get the summaries. It comes for free and is currently in the beta version.

You can access Paper Digest here !

SciSummary is another AI tool that summarizes scientific articles and literature review. It uses natural language processing algorithm to generate concise summaries. You need to upload the document on the dashboard or send the article link via email and your summaries will be generated and delivered to your inbox. This is the best AI-powered tool that helps you read and understand lengthy and complicated research papers. It has different pricing plans (both free and premium) which start at $4.99/month, you can choose the plans according to your needs.

You can access SciSummary here

Step-by-Step Guide to Using AI Tools to Summarize Literature Reviews

Here’s a short step-by-step guide that clearly articulates how to use AI tools for summary generation!

• Select the AI-powered tool that best suits your research needs.
• Once you've chosen a tool, you must provide input, such as an article link, DOI, or PDF, to the tool.
• The AI tool will then process the input using its algorithms and techniques, generating a summary of the literature.
• The generated summary will contain the most important information, including key points, methodologies, and conclusions in a succinct format.
• Review and assess the generated summaries to ensure accuracy and relevance.

Challenges of using AI tools for summarization

AI tools are designed to generate precise summaries, however, they may sometimes miss out on important facts or misinterpret specific information.

Here are the potential challenges and risks researchers should be wary of when using AI tools to summarize literature reviews!

1. Lack of contextual intelligence

AI-powered tools cannot ensure that they completely understand the context of the research papers. This leads to inappropriate or misleading summaries of similar academic papers.

To combat this, researchers should feed additional context to the AI prompt or use AI tools with more advanced training models that can better understand the complexities of the research papers.

2. AI tools cannot ensure foolproof summaries

While AI tools can immensely speed up the summarization process, but, they may not be able to capture the complete essence of a research paper or accurately decrypt complex concepts.

Therefore, AI tools are just to be considered as technology aids rather than replacements for human analysis or understanding of key information.

3. Potential bias in the generated summaries

AI-powered tools are largely trained on the existing data, and if the training data is biased, it can eventually lead to biased summaries.

Researchers should be cautious and ensure that the training data is diverse and representative of various sources, different perspectives, and research domains.

4. Quality of the input article affects the summary output

The quality of the research article that we upload or input data also has a direct effect on the accuracy of the generated summaries.

If the input article is poorly written or contains errors, the AI tool might not be able to generate coherent and accurate summaries. Researchers should select high-quality academic papers and articles to obtain reliable and informative summaries.

Concluding!

AI summarization tools have a substantial impact on academic research. By leveraging AI tools, researchers can streamline the literature review process, enabling them to stay up-to-date with the latest advancements in their field of study and make informed decisions based on a comprehensive understanding of current knowledge.

By understanding the role of AI tool to summarize literature review, exploring different AI tools for summarization, following a systematic review process, and assessing the impact of these tools on their academic research, researchers can harness AI tools in enhancing their literature review processes.

If you are also keen to explore the best AI-powered tool for summarizing the literature review process, head over to SciSpace Literature Review and start analyzing the research papers right away — SciSpace Literature Review

Few More Insightful Articles — Just for you!

10 best ai for essay writing, role of ai in systematic literature review, how to use ai tools for conducting a literature review, research paper summarizer | an overview of the best ai summarizers, you might also like.

AI for Essay Writing — Exploring Top 10 Essay Writers

This paper is in the following e-collection/theme issue:

Published on 31.10.2023 in Vol 25 (2023)

Data Quality in Health Research: Integrative Literature Review

Authors of this article:

Original Paper

• Filipe Andrade Bernardi 1 * , MSc   ; 
• Domingos Alves 1 * , Prof Dr   ; 
• Nathalia Crepaldi 1 , PhD   ; 
• Diego Bettiol Yamada 1 * , PhD   ; 
• Vinícius Costa Lima 1 , PhD   ; 
• Rui Rijo 1, 2, 3, 4 * , Prof Dr  

1 Ribeirão Preto School of Medicine, University of Sao Paulo, Ribeirão Preto, Brazil

2 Polytechnic Institute of Leiria, Leiria, Portugal

3 Institute for Systems and Computers Engineering, Coimbra, Portugal

4 Center for Research in Health Technologies and Services, Porto, Portugal

*these authors contributed equally

Corresponding Author:

Filipe Andrade Bernardi, MSc

Ribeirão Preto School of Medicine

University of Sao Paulo

Av Bandeirantes, 3900

Ribeirão Preto, 14040-900

Phone: 55 16997880795

Email: [email protected]

Background: Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance.

Objective: Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality.

Methods: A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases.

Results: After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches.

Conclusions: The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques.

International Registered Report Identifier (IRRID): RR2-10.1101/2022.05.31.22275804

Introduction

In health care settings, the priceless value of data must be emphasized, and the relevance and performance of digital media are evidenced by the efforts of governments worldwide to develop infrastructure and technology, aiming to expand their ability to take advantage of generated data. It is important to emphasize that technology, by itself, cannot transform data into information, and the participation of health care professionals is essential for knowledge production from a set of data. Through research that optimizes health interventions and contributes to aligning more effective policies, knowledge combines concrete experiences, values, contexts, and insights, which may enable a framework for evaluation and decision-making [ 1 ].

The low quality, nonavailability, and lack of integration (fragmentation) of health data can be highlighted among the main factors that negatively influence research and health decision-making. In addition, it is worth noting the existence of a large number of remote databases accessible only in a particular context. Such factors cause data quality problems and, consequently, information loss. Despite the intense volume, information remains decentralized, but it needs to help the decision-making process [ 2 ], making its coordination and evaluation challenging.

The crucial role of data spans a wide range of areas and sectors, ranging from health care data to financial data, social media, transportation, scientific research, and e-commerce. Each data type presents its own challenges and requirements regarding quality, standardization, and privacy. Ensuring the quality and reliability of these data is essential to support the combination of different sources and types of data that can lead to even more powerful discoveries [ 3 ].

For example, using poor-quality data in developing artificial intelligence (AI) models can lead to decision-making processes with erroneous conclusions. AI systems, which are increasingly used to aid decision-making, have used labeled big data sets to build their models. Data are often collected and marked by poorly trained algorithms, and research often demonstrates this method’s problems. Algorithms can present biases in judgments about a person’s profession, nationality, or character and basic errors hidden in the data used to train and test their models. Consequently, prediction can be masked, making it difficult to distinguish between right and wrong models [ 4 ].

Principles are also established in the semantic web domain to ensure adequate data quality for use in linked data environments. Such recommendations are divided into 4 dimensions: quality of data sources, quality of raw data, quality of the semantic conversion, and quality of the linking process. The first principle is related to the availability, accessibility, and reliability of the data source, as well as technical issues, such as performance and verifiability [ 5 ]. The second dimension refers to the absence of noise, inconsistencies, and duplicates in the raw data from these data sources. In addition, it also addresses issues regarding the completeness, accuracy, cleanness, and formatting of the data to be helpful and easily converted into other models, if necessary. The last 2 dimensions refer to the use of high-quality validated vocabularies, flexible for semantic conversion, and the ability of these data to be combined with other semantic data, thus generating sophisticated informational intelligence. Such factors depend on correctness, granularity, consistency, connectedness, isomorphism, and directionality [ 6 ].

The heterogeneity of data in this area is intrinsically connected to the type of information generated by health services and research, which are considered diverse and complex. The highly heterogeneous and sometimes ambiguous nature of medical language and its constant evolution, the enormous amount of data constantly generated by process automation and the emergence of new technologies, and the need to process and analyze data for decision-making constitute the foundation for the inevitable computerization of health systems and research and to promote the production and management of knowledge [ 7 ].

There are different concepts of data quality [ 8 ]. According to the World Health Organization, quality data portray what was determined by their official source and must encompass the following characteristics: accuracy and validity, reliability, completeness, readability, timeliness and punctuality, accessibility, meaning or usefulness, confidentiality, and security [ 9 ]. Data quality can be affected at different stages, such as the collection process, coding, and nonstandardization of terms. It can be interfered with by technical, organizational, behavioral, and environmental aspects [ 10 ].

Even when data exist, some aspects make their use unfeasible by researchers, managers, and health care professionals, such as the noncomputerization of processes, heterogeneity, duplicity, and errors in collecting and processing data in health information systems [ 11 ]. Reliable health data must support decision-making and strategies to improve service delivery to generate consistent evidence on health status, so the data quality management process must ensure the reliability of the data collected [ 12 ].

Some health institutions have action protocols that require their departments to adopt quality improvement and resource-saving initiatives. Consequently, various methodologies to improve the quality of services have been applied in the health field. Mulgund et al [ 13 ] demonstrated, for example, how data quality from physician-rating sites can empower patients’ voices and increase the transparency of health care processes.

Research in scientific communities about new strategies constantly evolves to improve research quality through better reproducibility and empowerment of researchers and provides patient groups with tools for secure data sharing and privacy compliance [ 14 ]. Raising a hypothesis and defining a methodology are a standard scientific approach in health research, which will lead to the acquisition of specific data. In contrast, data production in the big data era is often completely independent of the possible use of the data. One of the hallmarks of the big data era is that the data are often used for a purpose other than the one for which they were acquired. In this sense, influencing the modification of acquisition processes in clinical contexts requires more structured approaches [ 13 ].

The health sector is increasingly using advanced technologies, such as sophisticated information systems, knowledge-based platforms, machine learning algorithms, semantic web applications, and AI software [ 15 ]. These mechanisms use structured data sets to identify patterns, resolve complex problems, assist with managerial and strategic decision-making, and predict future events. However, it is crucial to ensure that the data used for these analyses adhere to the best practices and metrics for evaluating data quality to avoid biases in the conclusions generated by these technologies. Failure to do so can make it challenging to elucidate previously unknown health phenomena and events [ 16 ].

To use the best practices, institutions use the results of literature reviews due to the significant time savings and high reliability of their studies. Thus, through an integrative literature review, the main objective of this work is to identify and evaluate digital health technology interventions designed to support the conduct of health research based on data quality.

Study Design

The Population, Concept, and Context (PCC) strategy was applied to define the research question. The PCC strategy guides the question of the study and its elaboration, helping in the process of bibliographic search for evidence. The adequate definition of the research question indicates the information necessary to answer it and avoids the error of unnecessary searches [ 17 ].

“Population” refers to the population or problem to be investigated in the study. “Content” refers to all the detailed elements relevant to what would be considered in a formal integrative review, such as interventions and phenomena of interest and outcomes. “Context” is defined according to the objective and the review question. It can be determined by cultural factors, such as geographic location, gender, or ethnicity [ 18 ]. For this study, the following were defined: P=digital technology, C=data accuracy, and C=health research.

In this sense, the following research questions were defined:

• What is the definition of health research data quality?
• What are the health research data quality techniques and tools?
• What are the indicators of the data confidence level in health research?

Health Research

Numerous classifications characterize scientific research, depending on its objective, type of approach, and nature. Regardless of the purpose of how surveys can be classified, levels of confidence in data quality must be ubiquitous at all stages of the survey. Detailed cost-effectiveness analysis may inform decisions to adopt technology methods and tools that support electronic data collection of such interventions as an alternative to traditional methods.

Health research systems have invested heavily in research and development to support sound decisions. In this sense, all types of studies were observed that presented results of recent opportunities to apply the value of digital technology to the quality of the information in the direct or indirect evaluation of the promotion of health research. Therefore, in a transversal way, we considered all types of studies dealing with such aspects.

Types of Approaches

Various methods for setting priorities in health technology research and development have been proposed, and some have been used to identify priority areas for research. They include surveys and measurements of epidemiological estimates, clinical research, and cost-effectiveness assessments of devices and drugs. The technical challenges and estimation of losses due to variations in clinical practice and deviations from protocols have been supported by recommendation manuals and good practice guidelines. However, each of these proposed methods has specific severe methodological problems.

First, all these approaches see research simply as a method of changing clinical practice. However, there are many ways to change clinical practice, and conducting research may not be the most effective or cost-effective way. Research’s real value is generating information about what clinical practice should be. The question of how to implement survey results is a separate but related issue. Therefore, these methods implicitly assume no uncertainty surrounding the decision that the proposed research should inform.

Types of Interventions and Evaluated Results

Technology-based interventions that affect and aggregate concepts, designs, methods, processes, and outcomes promote data quality from all health research.

Measures demonstrate how results can address political, ethical, and legal issues, including the need to support and use technological mechanisms that bring added value regardless of the type and stage at which they are applied to research. We looked at how the results can be evaluated to address other questions, such as which subgroups of domains should be prioritized, which comparators and outcomes should be included, and which follow-up duration and moments would be most valuable for improving interventions on the reliability of health research data.

Eligibility Criteria

Research carried out in English and Portuguese, with quantitative and qualitative approaches, primary studies, systematic reviews, meta-analyses, meta-synthesis, books, and guidelines, published from 2016 onward was included. This choice is justified because we sought scientific indications that were minimally evaluated by our community. In this sense, websites, white papers, reports, abstracts only, letters, and commentaries were not considered. The year limitation is justified because knowledge is considered an adequate degree of being up to date.

In addition to the methodological design, we included any studies that described the definition, techniques, or tools that have the essential functions of synthesis, integration, and verification of existing data from different research sources to guarantee acceptable levels of data quality. In this way, we expected to monitor trends in health research, highlight areas for action on this topic, and, finally, identify gaps in health data arising from quality control applications.

Although the primary objective of this review was to seek evidence of data quality from health research, we also independently included studies on health data quality and research data quality. The exclusion criteria were applied to studies with a lack of information (eg, the paper was not found), studies whose primary focus was not health and research, and papers not relevant to the objective of the research, papers not available as full text in the final search, and papers not written in English or Portuguese. In addition, the titles and respective authors were checked to verify possible database repetitions. All criteria are presented in Table 1 .

a Not applicable.

Databases and Search Strategies

A search was carried out in 6 electronic scientific databases in January 2022 because of their quality parameters and broad scope: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers (IEEE) Digital Library, Cumulative Index of Nursing and Allied Health Literature (CINAHL), and Latin American and Caribbean Health Sciences Literature (LILACS). For the search, descriptors and their synonyms were combined according to the Health Sciences Descriptors (DeCS) [ 19 ] and Medical Subject Headings (MeSH) [ 20 ]. The following descriptors and keywords were selected, combined with the Boolean connectors AND and OR: “Data Accuracy,” “Data Gathering,” and “Health Research.” These descriptors and keywords come from an iterative and tuning process after an exploratory phase. The same search strategy was used in all databases.

Google Scholar was used for manual searching, searching for other references, and searching for dissertations. These documents are considered gray literature because they are not published in commercial media. However, they may thus reduce publication bias, increase reviews’ comprehensiveness and timeliness, and foster a balanced picture of available evidence [ 21 ].

We created a list of all the studies we found and removed duplicates. A manual search was performed for possible studies/reports not found in the databases. The references of each analyzed study were also reviewed for inclusion in the search. The search was carried out in January 2022, and based on the inclusion and exclusion criteria described, the final number of papers included in the proposed integrative review was reached. The search procedure in the databases and data platforms is described in Table 2 , according to the combination of descriptors.

a IEEE: Institute of Electrical and Electronics Engineers.

b LILACS: Latin American and Caribbean Health Sciences Literature.

c CINAHL: Cumulative Index of Nursing and Allied Health Literature.

Data Collection

First, 2 independent reviewers with expertise in information and data science performed a careful reading of the title of each paper. The selected papers were filtered after reading the abstract and selected according to the presence of keywords and descriptors of interest. The reviewers were not blinded to the journal’s title, study authors, or associated institutions. The established inclusion and exclusion criteria adequacy was verified for all screened publications. Any disagreements between the 2 reviewers were resolved by a senior third independent evaluator. The Mendeley reference manager [ 22 ] was used to organize the papers. Subsequently, the extracted findings were shared and discussed with the other team members.

Data synthesis aims to gather findings into themes/topics that represent, describe, and explain the phenomena under study. The extracted data were analyzed to identify themes arising from the data and facilitate the integration and development of the theory. Two reviewers performed data analysis and shared it with other team members to ensure the synthesis adequately reflected the original data.

Data Extraction

Data extraction involved first-order (participants’ citations) or second-order (researchers’ interpretation, statements, assumptions, and ideas) concepts in qualitative research. Second-order concepts were extracted to answer the questions of this study [ 17 ].

We looked at data quality characteristics in the studies examined, the assessment methods used, and basic descriptive information, including the type of data under study. Before starting this analysis, we looked for preexisting data quality and governance models specific to health research but needed help finding them. Thus, 2 reviewers were responsible for extracting the following data from each paper:

• Bibliographic information (title, publication date and journal, and authors)
• Study objectives
• Methods (study design, data collection, and analysis)
• Results (researchers’ interpretation, statements, assumptions, and ideas)

Result Presentation

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist ( Multimedia Appendix 1 ) and flowchart were used to visualize the search strategy results in the databases. PRISMA follows a minimum set of items to improve reviews and meta-analyses [ 23 ]. Based on the PRISMA flowchart, a narrative synthesis was prepared, in which we described the objectives and purposes of the selected and reviewed papers, the concepts adopted, and the results related to the theme of this review.

Data Synthesis

The data synthesis process involved several steps to ensure a systematic and comprehensive analysis of the findings. After a rigorous study selection process, the extracted data were analyzed using a coding and categorization approach.

Initially, a coding framework was developed based on the research objectives and key themes identified in the literature. This framework served as a guide for organizing and categorizing the extracted data. At least 2 independent reviewers performed this coding process to ensure consistency and minimize bias. Any discrepancies or disagreements were resolved through consensus discussions. Relevant data points from each study were coded and assigned to specific categories or themes ( Multimedia Appendix 2 ), capturing the main aspects related to data quality in health research, as shown in Table 3 .

Once the data were coded and categorized, a thorough analysis was conducted to identify patterns, trends, and commonalities across the studies. Quantitative data, such as frequencies or percentages of reported data quality issues, were analyzed using descriptive statistics. Qualitative data, such as themes or explanations provided by the authors, were analyzed using thematic analysis techniques to identify recurring concepts or narratives related to data quality.

The synthesized findings were then summarized and organized into coherent themes or subtopics. This involved integrating the coded data from different studies to identify overarching patterns and relationships. Similar results were grouped, and relationships between different themes or categories were explored to derive meaningful insights and generate a comprehensive picture of data quality in health research.

As part of the data synthesis process, the quality of the included studies was also assessed. This involved evaluating the studies’ methodological rigor, reliability, and validity using established quality assessment tools or frameworks. The quality assessment results were considered when interpreting and discussing the synthesized findings, providing a context for understanding the strength and limitations of the evidence.

Study Characteristics

In this review, 27,709 occurrences were returned from the search procedure, with 789 (2.84%) records from the SCOPUS database, 2 (0.01%) from LILACS, 1989 (7.18%) from the IEEE Digital Library, 5589 (20.17%) from the Web of Science, and 19,340 (69.80%) from PubMed. Searches were also performed in the World Health Organization Library and Information Networks for Knowledge (WHOLIS) and CINAHL databases, but no results were found. Of these, 25,202 (90.95%) records were flagged as ineligible by the automation tools and filters available in the databases, because they were mainly reports, editorial papers, letters or comments, book chapters, dissertations, and theses or because they did not specifically address the topic of interest according to the use of descriptors. Furthermore, 204 (0.74%) records were duplicated between databases and were removed.

After carefully evaluating the titles and abstracts (first screening step), 1221 (80.22%) of 1522 search results were excluded. For inclusion of papers after reading the abstracts, 81 () of 301 (26.9%) papers were listed for a full reading. After analyzing and extracting the desired results, 33 (40.7%) papers were included in the review because they answered the research questions. The entire selection, sorting, extraction, and synthesis process is described through the PRISMA flowchart [ 23 ], represented in Figure 1 .

The 33 studies covered the period of 2017-2021 and were conducted in 22 countries. Most studies were concentrated in Europe (n=11, 33.3%) and North America (the United States and Canada; n=10, 30.3%). Others were carried out in Oceania (Australia; n=4, 12.1%), Asia (China and Taiwan; n=3, 9.1%), and the Middle East (Iran and Saudi Arabia; n=2, 6.1%). In addition, studies were carried out collaboratively or in a network (the United States and India; the United States and African countries; the US Consortium, the United Kingdom, South Africa, Costa Rica, Canada, Sweden, Switzerland, and Bahrain; n=3, 9.1%).

In their entirety, the studies were carried out in high-income countries, and most of the assessments were based on the evidence available in English. The United States (n=11, 33.3%) and Australia (n=4, 12.1%) led in studies involving the investigated topic. No studies conducted or coordinated by middle-income countries were reported. In addition to the low economic diversity of countries where the research was conducted, all papers were evaluated in a single language. The involvement and collaboration of emerging countries took place exclusively through partnerships and participation in consortia.

Regarding the domains described in the studies, there was tremendous variability and inconsistency between the terms presented (n=38 terms). Note that no consensus existed between critical and noncritical variables for data quality assessment. The lack of consensus reflected that the definitions of concepts vary and their relationships are not homogeneous across studies. The discrepancy between domains and evaluated concepts did not allow an evaluation of parity between metrics and was present during all phases of the studies found. The subtopic distribution into the defined categories also evidenced the high-variability factors and strategies in the literature to lead with data quality. The distribution of the improvement strategies for data quality is shown in Table 4 and that of the related influencing factors for data quality in Table 5 .

Data Quality Issues and Challenges

The metrics extracted from the studies comprised domains related to the methodology adopted by them, that is, concepts that supported the definition of data quality and their respective individual or combined categorizations regarding the adjusted use for the purpose (n=8, 24.2%) of frameworks (n=6, 18.2%), ontologies (n=2, 6.1%), good practice guides (n=15, 45.5%), or combinations of methodologies (n=2, 6.1%).

Among the studies that used the concept of purpose-adjusted use, terms such as “gold standard according to experts” [ 24 ], “intrinsic quality” [ 25 ], “ideal record” [ 26 ], “data fitness” [ 27 , 28 ], and “data culture” [ 29 , 30 ] were addressed. In general, the use of frameworks and ontologies was based on previously published studies and available in development libraries as modules for mapping-adapted entities, proprietary or embedded systems, and data-based strategies for process improvement [ 31 - 34 ].

The central guides and guidelines adopted in data quality studies refer to the adoption of national protocols and policies, agreements signed between research networks and consortia, guides to good clinical practices (International Conference on Harmonization—Good Clinical Practice, ICHGCP [ 35 - 38 ]; Food and Drug Administration, FDA [ 35 , 38 ]; Health Insurance Portability and Accountability Act, HIPPA [ 39 ]), or information governance principles, models, and strategies (International Organization for Standardization, ISO [ 40 , 41 ]; Joint Action Cross-Border Patient Registries Initiative, PARENT [ 41 ]; Findability, Accessibility, Interoperability, and Reuse, FAIR [ 25 , 40 , 42 ]).

Regarding data quality, dimensions were interposed in all research stages, thus being a fundamental factor in being incorporated with good practices and recommendations, giving light to health research, regardless of their methodological designs. The distribution of dimensions evaluated in our findings showed significant heterogeneity, as shown in Table 6 .

Factors Affecting Data Quality

The study considered factors such as the environment, application time, and development steps, all influencing data quality. Controlled environments were reported in research-only scenarios with planning and proof-of-concept development [ 34 , 35 , 37 , 38 , 43 - 45 ]. Transition and validation environments were identified where research and service were combined [ 25 , 27 , 31 , 40 , 46 - 49 ]. Most studies were conducted in restricted environments specific to health services. Most studies also used their own research repositories, while others relied on external sources, such as preexisting data models [ 25 , 26 , 33 , 40 ] or public databases [ 38 , 50 ]. The research applications spanned diverse health areas, including electronic health records, cancer, intensive care units, rare diseases, maternal health, and more. However, the research areas were more concentrated in specialties such as clinical research [ 27 , 31 , 35 , 37 , 48 ], health informatics [ 43 , 45 ], and research networks [ 25 , 34 , 40 , 44 , 49 ]. Collaborative research networks and clinical trials played a prominent role in the application areas.

Data sources used in the research included literature papers, institutional records, clinical documents, expert perceptions, data models, simulation models, and government databases. Technical limitations were related to performance concerns, infrastructure differences, security measures, visualization methods, and access to data sources.

Other aspects mentioned included the disparity in professionals’ knowledge, the inability to process large volumes of information, and the lack of human and material resources. Legal limitations were attributed to organizational policies that restricted extensive analysis.

The main challenge reported in the studies was related to methodological approaches, particularly the inability to evaluate solutions across multiple scopes, inadequate sample sizes, limited evaluation periods, the lack of a gold standard, and the need for validation and evaluation in different study designs.

Overall, the integrated findings highlight the importance of considering the environment, application time, and methodological approaches in ensuring data quality in health research. The identified challenges and limitations provide valuable insights for future research and the development of strategies to enhance data quality assurance in various health domains.

Strategies for Improving Data Quality

In the analyzed studies, various strategies and interventions were used to plan, manage, and analyze the impact of implementing procedures on data quality assurance. Business intelligence models guided some studies, using extraction, transform, and load (ETL) [ 32 , 40 , 41 , 47 , 51 ]; preprocessing [ 28 , 45 , 52 - 54 ]; Six Sigma practices [ 32 , 48 ]; and the business process management (BPM) model [ 33 ]. Data monitoring strategies included risk-based approaches [ 36 , 37 ], data source verification [ 35 , 37 , 38 ], central monitoring [ 37 , 38 ], remote monitoring (eg, telephone contact) [ 31 , 38 ], and training [ 29 ]. Benchmarking strategies were applied across systems or projects in some cases [ 26 , 50 , 51 ].

Quantitative analyses primarily involved combined strategies, with data triangulation often paired with statistical analyses. Data mining techniques [ 24 ], deep learning, and natural language processing [ 45 ] were also used in combination or individually in different studies. Statistics alone was the most commonly used quantitative technique. The qualitative analysis encompassed diverse approaches, with consultation with specialists [ 30 , 34 , 43 , 44 , 54 , 55 ], structured instruments [ 29 , 38 , 44 , 46 ], data set validation [ 41 , 42 , 56 ], and visual analysis [ 33 , 40 , 48 ] being prominent. Various qualitative techniques, such as interviews [ 27 ], the Delphi technique [ 24 ], feedback audit [ 35 ], grammatical rules [ 39 ], and compliance enforcement [ 49 ], were reported.

Different computational resources were used for analysis and processes. The R language (R Core Team and the R Foundation for Statistical Computing) was commonly used for planning and defining data sets, while Python and Java were mentioned in specific cases for auditing databases and error detection. Clinical and administrative software, web portals, and electronic data capture platforms (eg, Research Electronic Data Capture [REDCap], CommonCarecom, MalariaCare, Assistance Publique–Hôpitaux de Paris–Clinical Data Repository [AP-HP-CDR], Intensive Care Unit DaMa–Clinical information System [ICU-DaMa-CIS]) were used for support, decision-making, data set planning, collection, and auditing. Additional tools, such as dictionaries, data plans, quality indicators, data monitoring plans, electronic measurements (e-measures), and Microsoft Excel spreadsheets, were also used.

It is evident that a range of strategies, interventions, and computational resources were used to ensure data quality in the studies. Business intelligence models, statistical analyses, data mining techniques, and qualitative approaches played significant roles in analyzing and managing data quality. Various programming languages and software tools were used for different tasks, while electronic data capture platforms facilitated data collection and auditing. The integration of these findings highlights the diverse approaches and resources used to address data quality in the analyzed studies.

Synthesis of Findings

The main barriers reported related to the theme of research in the area of health data quality cite circumstances regarding use, systems, and health services. Such barriers are influenced by technical, organizational, behavioral, and environmental factors that cover significant contexts of information systems, specific knowledge, and multidisciplinary techniques [ 43 ]. The quality of each data element in the 9 categories can be assessed by checking its adherence to institutional norms or by comparing and validating it with external sources [ 41 ]. Table 7 summarizes the main types of obstacles reported in the studies.

Although many electronic records provide a dictionary of data from their sources, units of measurement were often neglected and adopted outside of established standards. Such “human errors” are inevitable, reinforcing the need for continuous quality assessment from the beginning of collection. However, some studies have tried to develop ontologies to allow the automated and reproducible calculation of data quality measures, although this strategy did not have great acceptance. For Feder [ 55 ], “The harmonized data quality assessment terminology, although not comprehensive, covers common and important aspects of the quality assessment practice.” Therefore, generating a data dictionary with its determined types and creating a data management plan are fundamental in the planning of research [ 28 ].

Both the way of collecting and the way of inputting data impact the expected result from a data set. Therefore, with a focus on minimizing data entry errors as an essential control strategy for clinical research studies, implementing intervention modes of technical barriers was presented as pre- and postanalysis [ 56 ]. The problems were caused by errors in the data source, extraction, transform, and load process or by limitations of the data entry tool. Extracting information to identify actionable insights by mining clinical documents can help answer quantitative questions derived from structured health quality research data sources [ 39 ].

Given the time and effort involved in the iterative error detection process, typical manual curation was considered insufficient. The primary sources of error included human and technological errors [ 35 ]. However, outliers identified by automated algorithms should be considered potential outliers, leaving the field specialists in charge [ 51 ]. In contrast, different and ambiguous definitions of data quality and related characteristics in emergency medical services were presented [ 55 ]. Such divergences were based on intuition, previous experiences, and evaluation purposes. Using definitions based on ontology or standardization is suggested to compare research methods and their results. The definitions and relationships between the different data quality dimensions were unclear, making the quality of comparative assessment difficult [ 52 ].

In terms of evaluation methods, similar definitions overlapped. The difference lay in the distribution comparison and validity verification, where the definition of distribution comparison was based on comparing a data element with an official external reference [ 54 ]. Meanwhile, the validity check was concerned with whether a particular value wass an outlier, a value outside the normal range. The reasons for the existence of multiple evaluation practices were the heterogeneity of data sources about syntax (file format), schema (data structure models), and semantics (meaning and varied interpretations) [ 50 ]. There should be a standard set of data to deal with such inconsistencies and allow data transformation into a structure capable of interoperating with its electronic records [ 40 ].

Data standardization transforms databases from disparate sources into a standard format with shared specifications and structures. It also allows users from different institutions to share digital resources and can facilitate the merging of multicenter data and the development of federated research networks [ 34 ]. For this, 2 processes are necessary: (1) standardization of individual data elements, adhering to terminology specifications [ 49 ], and (2) standardization of the database structure through a minimum data set, which specifies where data values are located and stored in the database [ 50 ]. Improvements in electronic collection software functionality and its coding structures have also been reported to result in lower error rates [ 36 ].

In addition, it is recommended to know the study platform and access secondary data sources that can be used. In this way, transparency in the systemic dissemination of data quality with clear communication, well-defined processes, and instruments can improve the multidisciplinary cooperation that the area requires [ 44 ].

Awareness campaigns on the topic at the organizational level contributed to improving aspects of data governance. The most reported error prevention activities were the continuing education of professionals with regular training of data collectors during their studies [ 50 ]. In this sense, in-service education should promote the correct use of names formulated by structured systems to improve the consistency and accuracy of records and favor their regular auditing. Health systems that received financial incentives for their research obtained more satisfactory results regarding the degree of reliability of their data [ 53 ].

Figure 2 depicts the great diversity of elements involved in the data quality process in health research, representing the planning (precollection), development (data acquisition and monitoring), and analysis (postcollection) stages. In our findings, each phase presented a set of strategies and tools implemented to provide resources that helped the interaction between phases.

For the success of research, the processes and techniques must be fluid and applied in a direction based on good guides and recommendations. The research must go through phases, with well-established bases and tools suitable for its purpose, using sources and instruments available through digital strategies and systems, models, guides and feedback, and audit mechanisms.

In addition, every beginning of a new phase must be supported by well-defined pillars that encompass the exhaustive use of validations and pretests; plans for monitoring, management, and data analysis; precautions for ethical and legal issues; training of the team; and channels for effective communication.

In the broadest sense, incorporating data quality techniques and tools is analogous to going on a trip, that is, going from point A to point B. The starting point refers to good planning of issues, such as the year’s season, the quantity and type of items that will be transported, the most appropriate means of transport, the budget available, and tips and guidance available in the different means of communication. Even if the path is already known, an important step that precedes the beginning of its execution is always the definition of the best route. Consulting maps and updated conditions are always recommended since they can change over time.

However, the execution phase of a trip is not limited to reaching the final destination. During the journey, we should always be attentive to signs and directions, without obviously failing to enjoy the landscape and all its opportunities. Finally, when we arrive at our destination, we must bear in mind that to obtain the best results, it is necessary to know the best guides and tourist attractions. A wrong choice or decision can provide us with a low-quality photograph, an unexpected experience, and, as an effect, an epilogue of bad memories.

Principal Findings

This study presented contributions to aid the ultimate goal of good data quality focused on findings that used some digital technology (ie, to develop a disciplined process of identifying data sources, preparing the data for use, and evaluating the value of those sources for their intended use). Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research.

The routine of health services that deal with demands for collecting and consuming data and information can benefit from the set of evidence on tools, processes, and evaluation techniques presented here. Increasingly ubiquitous in the daily lives of professionals, managers, and patients, technology should not be adopted without a specific purpose, as doing so can generate misinterpreted information obtained from unreliable digital health devices and systems. The resources presented can help guide medical decisions that not only involve medical professionals but also indirectly contribute to avoiding decisions based on low-quality information that can put patients’ lives at risk.

With the promotion of the data culture increasingly present in a transversal way, research and researchers can offer increasingly more reliable evidence and, in this way, benefit the promotion and approach to the health area. This mutual cycle must be transparent so that there is awareness that adherence to such a practice can favor the potential strengthening of a collaborative network based on results and promote fluidity and methodological transparency. In addition, it encourages data sharing and, consequently, the reuse of data into reliable information silos, enhancing the development and credibility of health research. At the international level, platforms with a centralized structure of reliable data repositories of patient records that offer data sharing have reduced duplication of efforts and costs. This collaboration can further decrease disparate inequities between middle- and high-income, giving celerity and minimizing risks in the development and integrity of studies.

Reliable data can play a crucial role in enlightening health institutions that prioritize cultivating a data-centric culture and are well equipped to deliver high-quality information. This, in turn, facilitates improved conditions for patient care. In addition to mapping concepts between different sources and application scenarios, it is essential to understand how initial data quality approaches are anchored in previous concepts and domains, with significant attention to suitability for use, following guidelines or using frameworks in a given context [ 41 ]. Since the concept in the same data source can change over time, it is still necessary to carry out mapping with an emphasis on its dimensions in a sensible way and on how the evolution of concepts, processes, and tools impacts the quality assessment of research and health services [ 47 ].

The realization of mapping with emphasis on domains or concepts must coexist in health information systems. The outcome favors maximizing processes, increasing productivity, reducing costs, and meeting research needs [ 26 ]. Consequently, within legal and ethical limits, it is increasingly necessary to use data comprehensively and efficiently to benefit patients [ 57 ]. For example, recent clinical and health service research has adopted the “fit for use” concept proposed in the information science literature. This concept implies that data quality dimensions do not have objective definitions but depend on tasks characterized by research methods and processes [ 48 ]. Increasingly, data quality research has borrowed concepts from various referencing disciplines. More importantly, with many different referencing disciplines using data quality as a context within their discipline, the identity of the field of research has become increasingly less distinct [ 33 ].

Comparison With Prior Work

The large dissonance between domain definitions has increasingly motivated the search for a gold standard to be followed [ 30 ]. The area has received particular attention, especially after the term “big data” gained increasing strength [ 58 ]. The human inability to act with a large volume of information in research and the need to control this high data volume are increasingly driving the emergence of digital solutions. Although the definition of these digital data quality tools occurs from the end user’s perspective, their implementation occurs from the researcher’s perspective; a data set is highly context specific [ 33 ]. So, a generic assessment framework is unlikely to provide a comprehensive data quality analysis for a specific study, making its selection dependent on the study’s analysis plan [ 40 ].

The use of ontologies, for example, can help quantify the impact of likely problems, promote the validity of an effective electronic measure, and allow a generalization of the assessment approach to other data analysis tasks in more specific domains [ 55 ]. This benefit allows the decision-making process and planning of corrective actions and resource allocation faster [ 47 ]. However, the complex coding process can generate inconsistencies and incompleteness due to the characterization of clinically significant conditions, insufficient clinical documentation, and variability in interpretation [ 30 ]. Therefore, it is critical to use specific rules that capture relevant associations in their corresponding information groups. Administrative health data can also capture valuable information about such difficulties using standardized terminologies and monitor and compare coded data between institutions [ 24 ].

Nevertheless, as a consequence of this lack of standard, the use of integrated quality assurance methods combined with standard operating procedures (SOPs) [ 58 ], the use of rapid data feedback [ 38 ], and supportive supervision during the implementation of surveys are feasible, effective, and necessary to ensure high-quality data [ 31 ]. Adopting such well-defined interventions still plays an essential role in data quality management. It is possible to perform these activities through process control and monitoring methods, data manipulation and visualization tools, techniques, and analysis to discover patterns and perspectives on the target information subset [ 27 ]. Regardless of the model adopted, these tools should aim to discover abnormalities and provide the ability to stop and correct them in an acceptable time, also allowing for the investigation of the cause of the problem [ 56 ].

Technology is an excellent ally in these processes, and in parallel with the tools of the Lean Six Sigma philosophy, it can partially replace human work [ 31 ]. To maximize the potential of this combination, the value derived from using analytics must dictate data quality requirements. Computer vision/deep learning, a technology to visualize multidimensional data, has demonstrated data quality checks with a systematic approach to guarantee a reliable and viable developed asset for health care organizations for the holistic implementation of machine learning processes [ 53 ]. However, most of these analytical tools still assume that the analyzed data have high intrinsic quality, which can thus allow possible failures in the process, in addition to the final experiments’ lack of optimization, safety, and reliability [ 37 ].

In this way, the reuse of information has a tremendous negative impact [ 48 ]. The centralized storage of variables without excellent mapping to changes in system paradigms (metadata) and with a mechanism to trace the effects of changes in concepts that are frequent in the health area can also affect the reliability of research [ 37 ]. For example, the severity classification of a given condition can change over time and, consequently, mitigate the comparability power of a study or even prevent it from being used as a basis for planning or evaluating a new one [ 52 ]. In addition, the cultural background and experience of researchers can influence the interpretation of data [ 44 ]. Therefore, a combination of integrated tools located centrally and at each partner site for decentralized research networks can increase the quality of research data [ 40 ].

A central metadata repository contains common data elements and value definitions used to validate the content of data warehouses operated at each location [ 34 ]. So, the consortium can work with standardized reports on data quality, preserving the autonomy of each partner site and allowing individual centers to improve data in their locally sourced systems [ 29 ]. It is, therefore, essential to consider the quality of a record’s content, the data quality usability, and what mechanisms can make data available for broader use [ 41 ]. As outlined by Kodra et al [ 42 ], managing data at the source and applying the FAIR guiding principles for data management are recognized as fundamental strategies in interdisciplinary research network collaboration.

Data production and quality information dissemination depend on establishing a record governance model; identifying the correct data sources; specifying data elements, case report forms, and standardization; and building an IT infrastructure per agreed principles [ 29 ]. Developing adequate documentation, training staff, and providing audit data quality are also essential and can serve as a reference for teaching material for health service education [ 25 ]. This can facilitate more quality studies in low- and middle-income countries.

The lack of such studies implies that health systems and research performance in these countries still face significant challenges at strategic stages, such as planning and managing complete data, leading to errors in population health management and clinical care [ 43 ]. In turn, the low use of health information and poor management of health information systems in these countries make evidence-based decisions and planning at the community level difficult [ 2 ]. The results also demonstrate that, despite existing, such individual training efforts focus mainly on transmitting data analysis skills [ 33 ].

Identifying systematic and persistent defects in advance and correctly directing human, technical, and financial resources are essential to promote better management and increase the quality of information and results achieved in research [ 42 ]. This step can provide improvements and benefits to health managers, allowing greater efficiency in services and better allocation of resources. Promoting such benefits to society through relevant data impacts the performance and effectiveness of public health services [ 39 ] and boosts areas of research, innovation, and enterprise development [ 59 ].

Creative approaches to decision-making in data quality and usability require good use of transdisciplinary collaboration among experts from various fields regardless of study design planning or application area [ 59 ]. This use may be reaching the threshold of significant growth and thus forcing the need for a metamorphosis from the measurement and evaluation of data quality, today focused on content, to a direction focused on use and context [ 57 ].

Without a standard definition, the use of the “fit for purpose” concept for performance monitoring, program management, and data quality decision-making is growing. As a large part of this quality depends on the collection stage, interventions must target the local level where it occurs and must encompass professionals at the operational level and forms at the technical level. Identifying and addressing behavioral and organizational challenges and building technical capacity are critical [ 60 ], increasingly fostering a data-driven culture [ 29 , 30 ].

Limitations

Among the limitations of our review, we first highlight the search for works written in English and Portuguese, since the interpretation of concepts and even the literal translations of terms referring to the dimensions and adaptations to different cultural realities can vary, and thus influenced part of our evaluation [ 31 ]. The limitation may impact the results by excluding relevant research published in other languages and overlooking diverse cultural perspectives. To mitigate this, we suggest expanding collaboration with multilingual experts and including studies in various languages to ensure a comprehensive and unbiased evaluation of data quality.

Second, the absence of evidence in middle-income countries prevented the authors from conducting an adequate synthesis regarding the performance and application of the evidence found in these countries [ 2 ]. Limited representation from middle-income countries hinders the generalizability and applicability of findings, risking a biased understanding of intervention effectiveness. Inclusion of more studies from middle-income countries is vital for comprehensive evidence synthesis, enabling better comprehension of intervention performance in worldwide contexts and avoiding oversight of critical perspectives and outcome variations.

Third, due to the rapid growth of technologies applied to the area, we conducted a search focused on the past 5 years, which may draw attention away from other fundamentals and relevant procedures. The limited time span may lead to incomplete findings and conclusions, hindering a comprehensive understanding of the field’s knowledge and advancements. To address this limitation, future research should consider a broader time frame to include older studies, allowing for a more thorough examination of fundamentals and relevant procedures impacted by the rapid evolution of technologies in the area.

Future Directions

Once the technical and organizational barriers have been overcome, with data managed, reused, stored, extracted, and appropriately distributed [ 46 ], health care must also pay attention to behavior focused on interactions between human, artificial, and hybrid actors. This interaction reflects the importance of adhering to social, ethical, and professional norms, including demands related to justice, responsibility, and transparency [ 60 ]. In short, increasing dependence on quality information increases its possibilities [ 61 ], but it also presents regulators and policy makers with considerable challenges related to their governance in health.

For future work, developing a toolkit based on process indicators is desirable to verify the quality of existing records and provide a score and feedback on the aspects of the registry that require improvements. There is a need for coordination between undergoing initiatives at national and international levels. At the national level, we recommend developing a centralized, public, national “registration as a service” platform, which will guarantee access to highly trained personnel on all topics mentioned in this paper, promoting the standardization of registries. In addition to allowing cost and time savings in creating new registries, the strategy should allow for linking essential data sources on different diseases and increase the capacity to develop cooperation at the regional level.

We also suggest using the data models found in this study to serve as a structured information base for decision support information system development and health observatories, which are increasingly relevant to public health. Furthermore, concerning the health context, it may allow the execution of implementation research projects and the combination with frameworks that relate to health behavior interventions, for example, the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework [ 62 ], among others.

This study will help researchers, data managers, auditors, and systems engineers think about the conception, monitoring, tools, and methodologies used to design, execute, and evaluate their research and proposals concerned with data quality. With a well-established and validated data quality workflow for health care, it is expected to assist in mapping the management processes of health care research and promote the identification of gaps in the collection flow where any necessary data quality intervention can be accordingly evaluated with the best tools described here. In conclusion, the results provide evidence of the best practices using data quality approaches involving many other stakeholders, not just researchers and research networks. Although there are some well-known data quality guidelines, they are context specific and not found in the identified scientific publications. So, the information collected in this study can support better decision-making in the area and provide insights that are distinct from the context-specific information typically found in scientific publications.

Data Availability

The data sets generated and analyzed during this study are available as Multimedia Appendix 2 or can be obtained from the corresponding author upon reasonable request.

Conflicts of Interest

None declared.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist.

Individual studies.

• Hekler E, Tiro JA, Hunter CM, Nebeker C. Precision health: the role of the social and behavioral sciences in advancing the vision. Ann Behav Med 2020 Nov 01;54(11):805-826 [ https://europepmc.org/abstract/MED/32338719 ] [ CrossRef ] [ Medline ]
• Harrison K, Rahimi N, Danovaro-Holliday MC. Factors limiting data quality in the expanded programme on immunization in low and middle-income countries: a scoping review. Vaccine 2020 Jun 19;38(30):4652-4663 [ https://linkinghub.elsevier.com/retrieve/pii/S0264-410X(20)30401-1 ] [ CrossRef ] [ Medline ]
• Halevy A, Norvig P, Pereira F. The unreasonable effectiveness of data. IEEE Intell Syst 2009 Mar;24(2):8-12 [ CrossRef ]
• Knight W. The foundations of AI are riddled with errors. Wired. 2021 Mar 31. URL: https://www.wired.com/story/foundations-ai-riddled-errors/ [accessed 2023-10-18]
• Assaf A, Senart A. Data quality principles in the semantic web. 2012 Presented at: ICSC2012: 6th IEEE International Conference on Semantic Computing; September 19-21, 2012; Palermo, Italy [ CrossRef ]
• Zaveri A, Rula A, Maurino A, Pietrobon R, Lehmann J, Auer S. Quality assessment for linked data: a survey. Maastricht University. 2017 May 5. URL: https://cris.maastrichtuniversity.nl/en/publications/quality-assessment-for-linked-data-a-survey [accessed 2023-06-09]
• Peng C, Goswami P. Meaningful integration of data from heterogeneous health services and home environment based on ontology. Sensors (Basel) 2019 Apr 12;19(8):1747 [ https://www.mdpi.com/resolver?pii=s19081747 ] [ CrossRef ] [ Medline ]
• Kahn MG, Callahan TJ, Barnard J, Bauck AE, Brown J, Davidson BN, et al. A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data. EGEMS (Wash DC) 2016 Sep 11;4(1):1244 [ https://europepmc.org/abstract/MED/27713905 ] [ CrossRef ] [ Medline ]
• World Health Organization. Improving Data Quality: A Guide for Developing Countries. Manila: World Health Organization. Regional Office for the Western Pacific; 2003.
• Lucyk K, Tang K, Quan H. Barriers to data quality resulting from the process of coding health information to administrative data: a qualitative study. BMC Health Serv Res 2017 Nov 22;17(1):766 [ https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-017-2697-y ] [ CrossRef ] [ Medline ]
• A experiência brasileira em sistemas de informação em saúde Internet;1. Ministério da Saúde do Brasil, Organização Pan-Americana da Saúde, Fundação Oswaldo Cruz. 2009. URL: http://bvsms.saude.gov.br/bvs/publicacoes/experiencia_brasileira_sistemas_saude_volume1.pdf [accessed 2023-10-18]
• Ndabarora E, Chipps JA, Uys L. Systematic review of health data quality management and best practices at community and district levels in LMIC. Inf Dev 2013 Jun 27;30(2):103-120 [ CrossRef ]
• Mulgund P, Sharman R, Anand P, Shekhar S, Karadi P. Data quality issues with physician-rating websites: systematic review. J Med Internet Res 2020 Sep 28;22(9):e15916 [ https://www.jmir.org/2020/9/e15916/ ] [ CrossRef ] [ Medline ]
• Benchoufi M, Ravaud P. Blockchain technology for improving clinical research quality. Trials 2017 Jul 19;18(1):335 [ https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-017-2035-z ] [ CrossRef ] [ Medline ]
• Lovis C. Unlocking the power of artificial intelligence and big data in medicine. J Med Internet Res 2019 Nov 08;21(11):e16607 [ https://www.jmir.org/2019/11/e16607/ ] [ CrossRef ] [ Medline ]
• Pellison FC, Rijo RPCL, Lima VC, Crepaldi NY, Bernardi FA, Galliez RM, et al. Data integration in the Brazilian Public Health System for tuberculosis: use of the semantic web to establish interoperability. JMIR Med Inform 2020 Jul 06;8(7):e17176 [ https://medinform.jmir.org/2020/7/e17176/ ] [ CrossRef ] [ Medline ]
• Stern C, Lizarondo L, Carrier J, Godfrey C, Rieger K, Salmond S, et al. Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synth 2020 Oct;18(10):2108-2118 [ CrossRef ] [ Medline ]
• Aromataris E, Munn Z. JBI manual for evidence synthesis. JBI. 2020. URL: https://jbi-global-wiki.refined.site/space/MANUAL [accessed 2023-10-18]
• Pellizzon RDF. Pesquisa na área da saúde: 1. Base de dados DeCS (Descritores em Ciências da Saúde). Acta Cir Bras 2004 Apr;19(2):153-163 [ CrossRef ]
• Romano L. Using Medical Subject Headings (MeSH) in cataloging. Techn Serv Q 2018 Jan 29;35(2):217-219 [ CrossRef ]
• Paez A. Gray literature: an important resource in systematic reviews. J Evid Based Med 2017 Aug;10(3):233-240 [ https://pubmed.ncbi.nlm.nih.gov/29266844/ ] [ CrossRef ] [ Medline ]
• Mendeley. Mendeley Reference Manager [disk]. Version 2.40.0. London: Mendeley Ltd; 2022.
• Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 2021 Apr;88:105906 [ https://linkinghub.elsevier.com/retrieve/pii/S1743-9191(21)00040-6 ] [ CrossRef ] [ Medline ]
• Peng M, Lee S, D'Souza AG, Doktorchik CTA, Quan H. Development and validation of data quality rules in administrative health data using association rule mining. BMC Med Inform Decis Mak 2020 Apr 25;20(1):75 [ https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-020-1089-0 ] [ CrossRef ] [ Medline ]
• Schmidt CO, Struckmann S, Enzenbach C, Reineke A, Stausberg J, Damerow S, et al. Facilitating harmonized data quality assessments. A data quality framework for observational health research data collections with software implementations in R. BMC Med Res Methodol 2021 Apr 02;21(1):63 [ https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-021-01252-7 ] [ CrossRef ] [ Medline ]
• Harkener S, Stausberg J, Hagel C, Siddiqui R. Towards a core set of indicators for data quality of registries. Stud Health Technol Inform 2019 Sep 03;267:39-45 [ CrossRef ] [ Medline ]
• Ni K, Chu H, Zeng L, Li N, Zhao Y. Barriers and facilitators to data quality of electronic health records used for clinical research in China: a qualitative study. BMJ Open 2019 Jul 02;9(7):e029314 [ https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=31270120 ] [ CrossRef ] [ Medline ]
• Huser V, Li X, Zhang Z, Jung S, Park RW, Banda J, et al. Extending Achilles Heel data quality tool with new rules informed by multi-site data quality comparison. Stud Health Technol Inform 2019 Aug 21;264(5):1488-1489 [ CrossRef ] [ Medline ]
• Burnett SM, Wun J, Evance I, Davis KM, Smith G, Lussiana C, et al. Introduction and evaluation of an electronic tool for improved data quality and data use during malaria case management supportive supervision. Am J Trop Med Hyg 2019 Apr;100(4):889-898 [ https://europepmc.org/abstract/MED/30793695 ] [ CrossRef ] [ Medline ]
• Scobie HM, Edelstein M, Nicol E, Morice A, Rahimi N, MacDonald NE, et al. SAGE Working Group on Immunization and Surveillance Data Quality and Use. Improving the quality and use of immunization and surveillance data: summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization. Vaccine 2020 Oct 27;38(46):7183-7197 [ https://linkinghub.elsevier.com/retrieve/pii/S0264-410X(20)31159-2 ] [ CrossRef ] [ Medline ]
• Gass JD, Misra A, Yadav MNS, Sana F, Singh C, Mankar A, et al. Implementation and results of an integrated data quality assurance protocol in a randomized controlled trial in Uttar Pradesh, India. Trials 2017 Sep 07;18(1):418 [ https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-017-2159-1 ] [ CrossRef ] [ Medline ]
• Daniel C, Serre P, Orlova N, Bréant S, Paris N, Griffon N. Initializing a hospital-wide data quality program. The AP-HP experience. Comput Methods Programs Biomed 2019 Nov;181:104804 [ CrossRef ] [ Medline ]
• Andrews R, Wynn M, Vallmuur K, Ter Hofstede AHM, Bosley E, Elcock M, et al. Leveraging data quality to better prepare for process mining: an approach illustrated through analysing road trauma pre-hospital retrieval and transport processes in Queensland. Int J Environ Res Public Health 2019 Mar 29;16(7):1138 [ https://www.mdpi.com/resolver?pii=ijerph16071138 ] [ CrossRef ] [ Medline ]
• Bian J, Lyu T, Loiacono A, Viramontes TM, Lipori G, Guo Y, et al. Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data. J Am Med Inform Assoc 2020 Dec 09;27(12):1999-2010 [ https://europepmc.org/abstract/MED/33166397 ] [ CrossRef ] [ Medline ]
• Houston L, Probst Y, Martin A. Assessing data quality and the variability of source data verification auditing methods in clinical research settings. J Biomed Inform 2018 Jul;83:25-32 [ https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(18)30093-5 ] [ CrossRef ] [ Medline ]
• Tian Q, Liu M, Min L, An J, Lu X, Duan H. An automated data verification approach for improving data quality in a clinical registry. Comput Methods Programs Biomed 2019 Nov;181:104840 [ CrossRef ] [ Medline ]
• Houston L, Martin A, Yu P, Probst Y. Time-consuming and expensive data quality monitoring procedures persist in clinical trials: a national survey. Contemp Clin Trials 2021 Apr;103:106290 [ CrossRef ] [ Medline ]
• Houston L, Probst Y, Yu P, Martin A. Exploring data quality management within clinical trials. Appl Clin Inform 2018 Jan 31;9(1):72-81 [ https://europepmc.org/abstract/MED/29388180 ] [ CrossRef ] [ Medline ]
• Malmasi S, Hosomura N, Chang LS, Brown CJ, Skentzos S, Turchin A. Extracting healthcare quality information from unstructured data. AMIA Annu Symp Proc 2017 Apr 16;2017:1243-1252 [ https://europepmc.org/abstract/MED/29854193 ] [ Medline ]
• Juárez D, Schmidt E, Stahl-Toyota S, Ückert F, Lablans M. A generic method and implementation to evaluate and improve data quality in distributed research networks. Methods Inf Med 2019 Sep 12;58(2-03):86-93 [ http://www.thieme-connect.com/DOI/DOI?10.1055/s-0039-1693685 ] [ CrossRef ] [ Medline ]
• Kodra Y, Posada de la Paz M, Coi A, Santoro M, Bianchi F, Ahmed F, et al. Data quality in rare diseases registries. Adv Exp Med Biol 2017;1031:149-164 [ CrossRef ] [ Medline ]
• Kodra Y, Weinbach J, Posada-de-la-Paz M, Coi A, Lemonnier S, van Enckevort D, et al. Recommendations for improving the quality of rare disease registries. Int J Environ Res Public Health 2018 Aug 03;15(8):1644 [ https://www.mdpi.com/resolver?pii=ijerph15081644 ] [ CrossRef ] [ Medline ]
• Kumar M, Gotz D, Nutley T, Smith JB. Research gaps in routine health information system design barriers to data quality and use in low- and middle-income countries: a literature review. Int J Health Plann Manag 2018 Jan 02;33(1):e1-e9 [ CrossRef ] [ Medline ]
• van der Bij S, Khan N, Ten Veen P, de Bakker DH, Verheij R. Improving the quality of EHR recording in primary care: a data quality feedback tool. J Am Med Inform Assoc 2017 Jan;24(1):81-87 [ https://europepmc.org/abstract/MED/27274019 ] [ CrossRef ] [ Medline ]
• Juddoo S, George C. Discovering most important data quality dimensions using latent semantic analysis. 2018 Presented at: 2018 International Conference on Advances in Big Data, Computing and Data Communication Systems (icABCD 2018); August 6-7, 2018; Durban, South Africa [ CrossRef ]
• Pozzar R, Hammer MJ, Underhill-Blazey M, Wright AA, Tulsky JA, Hong F, et al. Threats of bots and other bad actors to data quality following research participant recruitment through social media: cross-sectional questionnaire. J Med Internet Res 2020 Oct 07;22(10):e23021 [ https://www.jmir.org/2020/10/e23021/ ] [ CrossRef ] [ Medline ]
• Sarafidis M, Tarousi M, Anastasiou A, Pitoglou S, Lampoukas E, Spetsarias A, et al. Data quality challenges in a learning health system. Stud Health Technol Inform 2020 Jun 16;270:143-147 [ https://pubmed.ncbi.nlm.nih.gov/32570363/ ] [ CrossRef ] [ Medline ]
• Shaheen NA, Manezhi B, Thomas A, AlKelya M. Reducing defects in the datasets of clinical research studies: conformance with data quality metrics. BMC Med Res Methodol 2019 May 10;19(1):98 [ https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-019-0735-7 ] [ CrossRef ] [ Medline ]
• D'Amore JD, Li C, McCrary L, Niloff JM, Sittig DF, McCoy AB, et al. Using clinical data standards to measure quality: a new approach. Appl Clin Inform 2018 Apr;9(2):422-431 [ http://www.thieme-connect.com/DOI/DOI?10.1055/s-0038-1656548 ] [ CrossRef ] [ Medline ]
• Sirgo G, Esteban F, Gómez J, Moreno G, Rodríguez A, Blanch L, et al. Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: the importance of data quality assessment. Int J Med Inform 2018 Apr;112:166-172 [ CrossRef ] [ Medline ]
• Sunderland KM, Beaton D, Fraser J, Kwan D, McLaughlin PM, Montero-Odasso M, ONDRI Investigators; et al. The utility of multivariate outlier detection techniques for data quality evaluation in large studies: an application within the ONDRI project. BMC Med Res Methodol 2019 May 15;19(1):102 [ https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-019-0737-5 ] [ CrossRef ] [ Medline ]
• Pezoulas V, Kourou K, Kalatzis F, Exarchos T, Venetsanopoulou A, Zampeli E, et al. Medical data quality assessment: on the development of an automated framework for medical data curation. Comput Biol Med 2019 Apr;107:270-283 [ https://www.sciencedirect.com/science/article/abs/pii/S0010482519300733 ] [ CrossRef ] [ Medline ]
• Chang H, Huang E, Hou I, Liu H, Li F, Chiou S. Using a text mining approach to explore the recording quality of a nursing record system. J Nurs Res 2019 Jun;27(3):e27 [ https://europepmc.org/abstract/MED/30694223 ] [ CrossRef ] [ Medline ]
• Mashoufi M, Ayatollahi H, Khorasani-Zavareh D. A review of data quality assessment in emergency medical services. Open Med Inform J 2018 May 31;12(1):19-32 [ https://europepmc.org/abstract/MED/29997708 ] [ CrossRef ] [ Medline ]
• Feder SL. Data quality in electronic health records research: quality domains and assessment methods. West J Nurs Res 2018 May 24;40(5):753-766 [ CrossRef ] [ Medline ]
• Johnson S, Speedie S, Simon G, Kumar V, Westra B. Quantifying the effect of data quality on the validity of an eMeasure. Appl Clin Inform 2017 Dec 14;08(04):1012-1021 [ CrossRef ]
• Mittelstadt B. Principles alone cannot guarantee ethical AI. Nat Mach Intell 2019 Nov 04;1(11):501-507 [ CrossRef ]
• Shankaranarayanan G, Blake R. From content to context: the evolution and growth of data quality research. J Data Inf Qual 2017 Jan 04;8(2):1-28 [ CrossRef ]
• Keller S, Korkmaz G, Orr M, Schroeder A, Shipp S. The evolution of data quality: understanding the transdisciplinary origins of data quality concepts and approaches. Annu Rev Stat Appl 2017 Mar 07;4(1):85-108 [ CrossRef ]
• Morley J, Luciano F. How to design a governable digital health ecosystem. In: The 2020 Yearbook of the Digital Ethics Lab. Cham: Springer; 2021:69-88
• Lepri B, Oliver N, Letouzé E, Pentland A, Vinck P. Fair, transparent, and accountable algorithmic decision-making processes. Philos Technol 2017 Aug 15;31(4):611-627 [ CrossRef ]
• Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health 2019;7:64 [ https://europepmc.org/abstract/MED/30984733 ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung; submitted 18.08.22; peer-reviewed by M Kapsetaki, I Adeleke, H Veldandi, WD Dotson, TFA Ang; comments to author 15.02.23; revised version received 18.04.23; accepted 14.07.23; published 31.10.23

©Filipe Andrade Bernardi, Domingos Alves, Nathalia Crepaldi, Diego Bettiol Yamada, Vinícius Costa Lima, Rui Rijo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 31.10.2023.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Social Security

Beyond benefits study.

Beyond Benefits Study Description

This is a one-page overview of the Beyond Benefits Study. The Social Security Administration (SSA) is conducting the Beyond Benefits Study to collect information about the service, medical, and employment needs of working-age adults exiting Social Security disability programs due to medical improvement. This overview describes the data collection activities planned and the goals of the study.

Motivational Interviewing Literature Review Memo

Recently, there has been an increasing emphasis on the use of “Motivational Interviewing” (MI) to identify and address the employment challenges among people with disabilities. This memo provides a literature review on the effectiveness of MI in various populations and discusses its utilization to foster employment and job advancement among people with disabilities.

The Relationship Between Myocardial Infarction and Estrogen Use: A Literature Review

Affiliations.

• 1 Gynecology, Heart's International Hospital, Rawalpindi, PAK.
• 2 Medicine and Surgery, Sri Venkata Sai Medical College, Telangana, IND.
• 3 Internal Medicine, Vydehi Institute of Medical Sciences and Research Centre, Bangalore, IND.
• 4 Gynecology, Karachi Medical and Dental College, Karachi, PAK.
• 5 Endocrinology, Alfredo Van Grieken University Hospital, Coro, VEN.
• 6 Internal Medicine, Katuri Medical College and Hospital, Guntur, IND.
• 7 Medicine, University College Dublin, Dublin, IRL.
• 8 Medicine, European University Faculty of Medicine, Tbilisi, GEO.
• 9 Internal Medicine, Non Resident Indian (NRI) Medical College, Vijayawada, IND.
• 10 Internal Medicine, Albert Einstein Healthcare Network, Philadelphia, USA.
• PMID: 37900417
• PMCID: PMC10612533
• DOI: 10.7759/cureus.46134

This thorough literature evaluation was prompted by significant research into the complex interactions between estrogen use and myocardial infarction (MI). Estrogen has fascinated researchers because of its possible cardioprotective benefits and its impact on cardiovascular health. In order to clarify the connection between estrogen use and the risk of MI, this review critically examines the body of prior evidence. This review focuses on estrogen and its pivotal role in cardiovascular health, concentrating on lipid metabolism, vasodilation, inflammation, and endothelial function. It examines contentious data about estrogen therapy's heart-protective effects, taking into account age, initiation timing, dosage, and dosage of administration. Genetic and epigenetic influences on MI risk among estrogen users highlight intricate, personalized estrogen effects. The conclusion summarizes the main findings and emphasizes the need for an all-encompassing strategy for initiating and managing estrogen medication. It is crucial to consider patient-specific traits and risk factors to successfully customize treatment regimens. This review sheds vital light on the potential directions for better cardiovascular treatment for postmenopausal women by shedding light on the complex link between estrogen use and myocardial infarction. The review also identifies research gaps and future objectives in this area, highlighting the demand for novel medicines and individualized strategies to improve cardiovascular outcomes.

Keywords: cvd; estradiol; estrone; myocardial infarction; transdermal estrogen.

Copyright © 2023, Javed et al.

Publication types