As physiotherapists we are required by the Code of Ethical Conduct and the Standards of Practice to commit to lifelong learning to maintain our competence and incorporate evidence into practice.1,2 A common strategy for meeting these expectations and staying up-to-date with emerging areas of practice is to search for and review current research related to a specific subject.
However, if you are anything like me, this can be a daunting process as searches may turn up copious numbers of articles or, if I have used the wrong search terms, turn up no usable resources. How can I find good, trustworthy research? How do I know which articles I should review? And, when reading an article, how do I determine if what I am reading is reliable, valid and relevant to my practice?
The purpose of this article is to provide members with some simple tips about what to look for when searching for and reading articles. We will also look at some red flags that should make you suspicious about the findings reported by the author(s).
The first step when starting your search is to identify your question. “Systematic, explicit, and reproducible are three key words” to keep in mind when initiating a search.4 It’s recommended that you:4
Start with a specific, answerable and relevant question or series of questions.
Too specific a question may limit your results and too general a question will provide you with too many results.
By developing a well-thought out, clinically focused and specific question, the search becomes easier and more relevant to your practice needs.
The question should define:4
What specific client group, situation or population are you targeting? (e.g. community dwelling seniors).
What specific assessment, intervention or other clinical issue are you addressing? (e.g., risk factors for falls).
What outcomes are you wanting to achieve or what adverse effects do you want to avoid? (e.g., the incidence of falls.)
Perform the search in a systematic and methodological manner4
Determine where you will start your search. A quick search on Google or Google Scholar may give you an overview of the topic and what might be available. A trip to your local academic library for current books and access to electronic databases would be a good next step.
How far back in the literature do you wish to search? Narrow or broaden your search dates as required. Although recent evidence is generally preferable, do not ignore good research done in the past. It may still be useful and relevant.
You can limit your search by adding filters to select articles by language of publication or availability of an online abstract.
There are many resources available that will guide you in selecting search terms, using thesaurus/subject search methods, using free text or MeSH (Medical Subject Headings) terms and/or combining terms using “and.” “or” or other techniques to assist with focusing your database search.
Keep a record of your search terms, process and databases searched to allow you, or another individual, to reproduce the search at a later date.
Consider the source
Where you search for the answer to your question is important. In today’s world, the first place many of us start is an internet search. The greatest advantage of the Internet is the convenience and ease of access to large quantities of information. While a Google search may provide some general ideas about what is available on a topic, the major disadvantage is that there is simply no quality control of the information provided.4 One way to weed out more questionable information is to use a search engine such as Google Scholar.
If you use an internet search, you must have strict criteria for selecting sites and be critical when determining if they are credible, reliable or even real sources of information. Starting with government, university and professional association sites may be appropriate.
Here is a checklist of questions to ask yourself as you evaluate a webpage.4
Authority: Who is responsible for publishing and managing the page?
Authors and creators should be clearly identified and you should be able to contact them. Any affiliations with commercial or other organizations should be clearly identified.
What are the goals/values of the author?
What makes the author an authority on the subject? Are they accredited or endorsed by a reputable organization?
Does the site contain articles, and if so, do they contain footnotes or citations for other sources used?
Are the facts verifiable?
Are other sources cited?
Are individual articles signed and attributed?
Is the information presented as opinion, propaganda or fact? If fact, is it accurate? Are claims supported by evidence?
Consider the purpose of the site. Look at the different types of websites identified above.
If there is a bias (cultural, political, religious, marketing, academic) is it clearly stated?
Is the site kept current? Is the date of the last revision posted? How frequently is the information updated?
Scope, coverage, relevance
Does the information meet your needs, who is the intended audience?
What is the level of the material presented? (basic, academic, advanced)
What is the time period or geographical area covered?
Is the author/owner/presenter selling something?
Does the site have a corporate sponsor?
Are there hidden costs or do you need to enter personal information to proceed?
If you are suspicious, can’t answer the questions or the answers are negative, move on to another source.
Electronic bibliographic databases
While the internet may be a good place to start to get an idea of what’s available on a given topic, using a recognized electronic bibliographic database is a better option to collect current, relevant and reliable articles. These databases are focused compilations of published research, scholarly articles, books etc.4 Before you start your search you need to know what topics/subjects the database focuses on. The University of Alberta library suggests four “best bets” for physiotherapists wanting to do a literature search.5 These four include: CINAHL Plus with Full Text, MEDLINE via EBSCOhost, MEDLINE via Ovid, and PEDro. Members will also be familiar with the Cochrane Database of Systematic Reviews which is a database for systematic reviews of any topic relevant to health care.
Hopefully your search will turn up a number of articles. The next step is to look at the question the researchers asked and consider if their question is similar to the question you want an answer to. You can often find this in the abstract. If you cannot determine what question the authors were trying to answer, you may want to move on to the next article on the list.
You should also ask yourself “Are the subjects in the study similar enough to my situation that I can apply the study’s findings?”4 If the article is looking at treatment options available to you, with a population that is similar to yours, you may be able to apply the information to your situation. If not, then the study results, whether valid or not, will not be of use to your practice. Move on to the next article.
Below are a series of questions you can ask yourself as you review the content.7 If any of the answers are no or not really, then a red flag should appear in your mind. This does not mean you cannot use the information, but you will need to proceed with caution and will want to verify the information from another source.
Is the size of the study group appropriate for the identified purpose and type of the research? The appropriate size of a study group will vary depending on the type of study. For example, a case study will only look at a few cases and no control group. However, you would expect a randomized control/clinical trial to have a large number of participants that are randomly separated into treatment and control groups.4 The participants should be representative of the population being studied and large enough to allow for some generalization of the results.4 Conclusions that generalize findings to a large group or the entire population based on a case study of 10 individuals are not likely to be valid.
Was the study well designed?
Is the study design described and followed? Does it fit with the research question being asked?
Is there a specific hypothesis and is it clearly stated?
Is it clear how participants were recruited and what the criteria were?
Are researchers and/or participants blinded?
Are any conflicts of interest disclosed? If not, should there have been?
Do the author’s conclusions make sense based on the results of the study or could there be other reasons to explain the findings?
Are the conclusions supported by or similar to other published research? If not, why? The authors should offer a plausible explanation for the differences between their findings and those of other researchers, including any limitations to or aspects of their study that may have negatively impacted its accuracy.
Are the findings and conclusions completely different or more extreme than those of other studies? If yes, a red flag should appear. It’s not uncommon for small studies to report extreme results (both positive and negative) related to a treatment. This relates to the study’s sample size and precision. Larger studies typically report more modest effects and are thought to more accurately reflect the true effect of a treatment.10 While it is exciting when a study demonstrates a significant positive effect, readers should be alert to the sample size and be cautious about adopting a treatment based on the findings of a small study, regardless of how positive the findings are.
Can/has the study be replicated by other researchers?
Has the article been peer reviewed?
Keep in mind that even journal articles that appear credible may not be. Recently the academic world has been challenged by predatory publishing practices that see work of low quality and questionable merit published in open access journals.6 While open source publishing sounds like a great thing to a clinician searching for evidence, especially one who doesn’t have access to a university library, it can lead to problems if the articles retrieved were published by a predatory journal. These journals often publish articles without the oversight and rigorous peer review used by traditional publishers to ensure the quality, accuracy and scientific merit of the research and its reported findings.6 This means that readers need to review the methods used by researchers and be aware of flags that may indicate a journal is not using rigorous peer review.
Although there are many ways to identify questionable journals, one way to spot a predatory publisher is to look at the speed with which an article moves its way through the submission and review process.9 It is common for journals to publish the date an article was submitted and the date it was accepted for publication. The interval between the two is an indication of the time spent having the article peer-reviewed and revised. It is typical for this to take several weeks or months. It is uncommon for it to be a matter of days. If you are unfamiliar with a journal and its reputation, and note that the review process is very brief, this should raise your suspicions about the nature and quality of the journal’s peer review process and the quality and validity of the research in question.
Although it does take more time, if you want evidence that is reliable and valid to inform, enhance or support your practice you must do your due diligence and appraise the quality of the research. It is not sufficient to simply read the abstract and/or rely on the authors’ conclusions. There are many tools available to help guide your appraisal of the evidence if this is something new to you. Using systematic reviews, where experts have already done a lot of the work for you, may also be a good option for finding the answer to the question you are asking.
If you have questions that the authors have not answered fully or if you feel uncomfortable with the conclusions the authors have drawn based on the results, you should look further. The cardinal principle for anyone reading scientific articles should be “Trust but Verify” everything.6
Physiotherapy Alberta College + Association (2017) Code of Ethical Conduct. Available at https://www.physiotherapyalberta.ca/files/code_of_ethical_conduct.pdf
Physiotherapy Alberta College + Association (2017) Standard of Practice: Competence. Available at https://www.physiotherapyalberta.ca/files/practice_standard_competence.pdf
Government of Alberta. Health Professions Act. Available at http://www.qp.alberta.ca/1266.cfm?page=H07.cfm&leg_type=Acts&isbncln=9780779793440&display=html
Lou, JQ. Searching for the evidence. In: Law, M, editor. Evidence-based rehabilitation: a guide to practice. Thorofare, NJ: SLACK Incorporated; 2002. p. 71-91.
University of Alberta Libraries. A-Z databases: Physical Therapy. Available at https://guides.library.ualberta.ca/az.php?s=60713
Califf RM, McCall J, Harrington RA. Assessing Research Results in the Medical Literature: Trust but Verify. JAMA Intern Med. 2013;173(12):1053–1055. doi:10.1001/jamainternmed.2013.829
University of Saskatchewan. University Library: Predatory Publishers: Home. Available at https://libguides.usask.ca/predatorypublishers
Hill, K. What is a good study?: Guidelines for evaluating scientific studies. Available at https://sciencebasedlife.wordpress.com/resources-2/what-is-a-good-study-guidelines-for-evaluating-scientific-studies/
Glasson, V. 6 Ways to Spot a Predatory Journal. Available at: https://www.rxcomms.com/blog/6-ways-spot-predatory-journal/
Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rucker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JP. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomized controlled trials. BMJ 2011; 343:d4002. Available at https://www.bmj.com/content/343/bmj.d4002 Accessed June 27, 2019.
Patient Safety. Every Person. Every Time.
Regulating Alberta's physiotherapy profession and acting as an association by providing member services.