Critical Appraisal of the Journal Literature
Return to contents:
| Medicine & Health Sciences guide | Psychology
guide | Tizard guide |
Headings:
| General resources | Teaching
resources | Critically appraising qualitative research
|
| Critically appraising quantitative research | Appraising web pages |
High quality / recommended resources have been marked with this icon : 
The ability to critically appraise research papers is very useful. If you have done a literature search, and found what look to be some useful papers, then it is often a good idea to sit down and assess the quality and validity of the material - particularly if you plan to refer to some of the results in your own research..
What is critical appraisal?
The culture and concept of "critical appraisal" has largely grown out of the evidence-based health care movement. It fits into the cycle of "getting evidence into practice." In theory, this means improving the quality and cost effectiveness of health care by finding the best available research evidence on the outcomes of health care interventions, and basing decisions on health care upon it. In practice this translates as, first of all, "finding the evidence" (doing a literature search in the most appropriate databases), then carefully checking the validity of the research ("critical appraisal"), before applying the lessons learnt from the evidence in the care of patients/clients ("getting research into practice").
The idea has now spread to other disciplines in the social sciences (such as psychology), and tools are available to help people evaluate qualitative research.
All critical appraisal means is being able to look at a piece of research in a very objective and structured way in order to decide how valid it is compared to other research. This page provides you with a number of resources (including appraisal checklists for various study designs) to help you do this.
What relevance does it have to my work?
Critical appraisal is most aptly applied to quantitative studies that look at the effectiveness of different health and medical interventions (e.g. "randomised" or "blinded" controlled trials, crossover trials, meta-analyses or systematic reviews).
However, the skills can also be applied to the assessment of qualitative studies of psychosocial and behavioural interventions: e.g. observational or interview data obtained from case or cohort studies 1.
Furthermore, studies on the effectiveness of cognitive, behavioural and other psychosocial interventions are also being conducted using quantitative research methodologies (e.g. randomised controlled trials); especially where these interventions are used as part of a combination therapy which includes medication.
Applying critical appraisal methods may not be suitable for all of you. However, if you will be working in the area of community or social care, and particularly in a context where you will be making decisions which affect your clients' health, then you may find these resources of use.
Critical appraisal resources:
The aim of this page is not to teach you how to ctitically appraise a journal article, but to point you towards resources which can help you learn for yourself, or assist you after you have been taught to do it more formally.
General resources:
CASP (Critical Appraisal
Skills Programme) - part of the Public Health Resource Unit
based at Oxford, CASP runs training workshops on critical appraisal skills.
This site also contains some of their checklists
for appraising research (below).
Evidence-Based Medicine Toolkit - hosted by the University of Alberta, this is an online "box" of handy tools to help you find, appraise, and apply in practice, evidence-based research
How to read a paper
- a set of ten guides from the BMJ (individual links given in the relevant
section below). General topics include:
- Getting your bearings (deciding what a paper is about)
- Assessing the methodological quality of published papers
- Statistics for the non-statistician
- Statistics for the non-statistician II
JAMA user guides to the medical literature (subscription only)
Levels of Evidence - a ranking system used to rank various study designs in order of evidence-based merit: systematic reviews/meta-analyses and well conducted randomised controlled trials (RCTs) are usually seen as the best form of "evidence", with research based on the outcome of a case series placed somewhere near the bottom
Netting the Evidence - search for the keyword "appraisal" to find a quality assessed list of appraisal resources
User's guides to evidence
based practice - based on a series of articles published in
JAMA, these guides give comprehensive advice on how to find, appraise
and apply research in practice
Teaching resources:
Teaching materials from the Centre for Evidence Based Medicine (University of Toronto) - includes teaching scenarios for use with problem based learning
Critically appraising qualitative research:
10 questions
to help you make sense of qualitative research (pdf) - a checklist
of things to look for produced by CASP
How to read a
paper: papers that go beyond numbers (qualitative research)
- more detailed background on appraising qualitative research
Evaluation of Qualitative Research Studies - article from the EBN user's guide series (Russell, C.K. and Gregory, D.M. Evid Based Nurs 2003;6:36-40 doi:10.1136/ebn.6.2.36)
Integrative Approaches to Qualitative and Quantitative Evidence (.pdf) - originally published by the Health Development Agency, this document covers the problems of appraising qualitative research, heirachies of evidence within the different type of qualitative studies; and how the findings of qualitative and quantitative research can be synthesised
Critically appraising quantitative health research:
Appraising Quality of Life (QOL) measures
Issues in the measurement of health-related Quality of Life - some tips and background on appraising papers which measure quality of life
Appraising case or cohort studies
11 questions
to help you make sense of case control studies (pdf) - a checklist of
things to look for in a paper produced by CASP
12 questions
to help you make sense of a cohort study (pdf) - a checklist of things
to look for in a paper produced by CASP
Appraising economic evaluations (papers about the cost of health care interventions)
10 questions
to help you make sense of economic evaluations (pdf) - a checklist
of things to look for in a paper produced by CASP
How to read a
paper: papers that tell you what things cost (economic analyses)
- BMJ article explaining how to appraise a paper that evaluates the economic
cost of a healthcare intervention
Appraising controlled trials
This includes clinical trials, randomised controlled trials, crossover trials, and blinded trial designs.
10 questions to
help you make sense of randomised controlled trials (pdf) - a checklist
of things to look for when reading a paper based upon the results of trial
research, produced by CASP
How to read a
paper: papers that report drug trials - BMJ article explaining
how to appraise a report of trial results
Appraising meta-analyses and systematic reviews
10 questions
to help you make sense of reviews (pdf) - a checklist of things to
look for when reading a systematic review or meta-analysis, prepared by
CASP
How to read a
paper: papers that summarise other papers (systematic reviews and meta-analyses)
- BMJ article explaining how to appraise a review or meta-analysis
Systematic Review (of Therapy) Worksheet - produced by the Centre for Evidence Based Medicine at the University of Toronto
Appraising papers assessing the accuracy of diagnostic tests
These include things like the Hospital Anxiety and Depression Scale, or DSM IV
12
questions to help you make sense of a diagnostic test study (pdf)
- a checklist of things to look for when assessing the sensitivity, specificity
& validity of a diagnostic test, prepared by CASP
Diagnostic Worksheet - produced by the Centre for Evidence Based Medicine at the University of Toronto
How to read a
paper: papers that report diagnostic or screening tests - BMJ
article explaining how to appraise a paper about a diagnostic test
How good is that test? and How good is that test II? - two resources from Bandolier
Appraising papers about therapy
Therapy Worksheet - produced by the Centre for Evidence Based Medicine at the University of Toronto
Appraising papers about prognosis (long term outcome)
Prognosis Worksheet - produced by the Centre for Evidence Based Medicine at the University of Toronto
Appraising articles about aetiology (risk of harm)
Harm Worksheet - produced by the Centre for Evidence Based Medicine at the University of Toronto
Appraising clinical guidelines
AGREE Collaboration - project which created an appraisal tool for assessing clinical guidelines: see the AGREE Instrument which outlines the framework for developing, reporting, and assessing clinical guidelines
1. See glossary of study designs - (now go back to text)
To report errors or make suggestions for this page:
Please e-mail Jason Harper