Quality appraisal of qualitative research

179 Pages·2015·5.09 MB·English

Quality appraisal of qualitative
research
Karin Hannes
Methodology of Educational Sciences Research Group
KU Leuven
[email protected]
Agenda: Quality appraisal
• What is it?
• Should we appraise quality?
• Are there any criteria that compare to the basic quality
criteria for quantitative studies?
• Are there any techniques to diminish quality threats?
• What are the different stages in a critical appraisal
exercise?
• How do different appraisal instruments compare to
each other?
• How to use and report a critical appraisal outcome?
Quality appraisal: what is it!
• “the process of systematically examining
research evidence to assess its validity, results
and relevance before using it to inform a
decision”
• Hill, A., Spittlehouse, C, 2003. What is critical appraisal? Evidence Based
Medicine, 3 (2), 1-8. Available from:
• http://www.evidence-based-
medicine.co.uk/ebmfiles/WhatisCriticalApprai
sal.pdf
Quality appraisal: Should we?
A review of published QES
• 21 papers did not describe appraisal of candidate
studies
• 6 explicitely mentioned not conducting formal
appraisal of studies
• 5 papers did a critical appraisal, but did not use a
formal checklist
• 7 described modifying existing instruments
• 1 used an existing instrument without modification
Dixon-Woods M, Booth A, Sutton AJ.
Synthesizing qualitative research: a review of
published reports. Qual Res 2007; 7:375
Quality appraisal: Should we?
• Qualitative research is
subject to the same
criteria as
quantitative research.
Validity, reliability and
generalisibility should
be addressed in critical
appraisal.
Quality appraisal: Should we?
•Adjust the tools.
Qualitative research
is in need of a set of
criteria specifically
designed for it.
Quality appraisal: Should we?
• Put an end to
criteriology.
Do not try to fit
something that in the
end stiffles the
creative aspects of
qualitative research.
Quality appraisal: Should we?
• Use criteria as guides
to good practice
They should not be used
as rigid requirements in
appraising papers.
Quality appraisal: Should we?
• Interpretivists (idealism) Realists-pragmatics
– Bias: Subjectivity is uses – Bias: Researcher bias affects
actively/creatively through trustworthiness, or validity
the research process
– Validity:the emphasis is on
– Validity: there are multiple striving for truth in being
ways of understanding adequate, accurate, credible
reality
– Reliability: Steps to establish
– Reliability: Researchers it should be build into the
report information and research process to affirm
readers discern the patterns researchers’ observations
identified and verify
interpretations
The more you appraise, the
The more you appraise, the
lesser the chance to end
more it stifles creativity.
up with flawed results. The
The main inclusion
main inclusion criterion is
criterion is relevance!
quality!
Quality appraisal: Should we?
I appraise!
Note: I might substantially have been brainwashed in the ‘risk of bias’ discourse, beyond my personal control.