Assessments 2 &3 – Critically Appraised Topic (CAT)
Format: Assessment 2: OSCE and Assessment 3: Written report using instructions below
Due: Assessment 2 OSCE: Week 6, During scheduled class time, time to be allocated (between 9am and 3pm)
Assessment 3 Written report: Week 7, Monday 9am
Weighting: 70% of overall grade for subject (30% OSCE/ 40% report)
Time limit: 7-10mins (Assessment 2: OSCE)
Word count: 2000 words (Assessment 3: Written report)
A Critically Appraised Topic (CAT) is a summary of the evidence to a focused clinical question.
You are required to find and critically appraise the research evidence to answer a question of treatment/intervention effectiveness.
- ASK: Develop a research question. The treatment question may be drawn from a specific clinical scenario or family/personal health experience and must be relevant to occupational therapy.
Please see document in iLearn (assessment folder: “Potential CAT topics for 223”) with example topics for you to choose from if you do not have a topic in mind
- ACQUIRE: Conduct a review of the literature to answer your question of treatment/intervention effectiveness. Identify two studies to critically evaluate.
- One must be related to treatment/intervention effectiveness (acquired from your search above), in the form of a systematic review relevant to the question.
- The remaining paper is required to be a different study design from your first paper, which may include a diagnostic, prognostic, or qualitative study that might provide additional information related to the condition or to the intervention.
- Choose two databases to search for relevant research articles to answer your PICO question.
- Your literature search should be designed to locate the research articles that represent, as far as possible, the best/most appropriate research available to answer the chosen clinical question. An attempt to locate the highest quality evidence appropriate for the question is reflected in the marking criteria.
- If, upon searching for the answer to a clinical question, you can’t find these types of studies you will need to broaden your question (i.e. be less specific) or choose another topic.
- In a few cases there may be no research evidence of any kind or a very small amount of low-quality evidence available to answer your question – if this occurs, you may need to broaden your question or choose another question.
- APPRAISE: Critically appraise the two studies deemed to be the highest quality evidence from your search. Critical appraisal checklists that are referred to in the subject will assist you when critically appraising the research evidence. One example is CASP, found at https://casp-uk.net/casp-tools-checklists/ . The textbook also includes critical appraisal checklists (Hoffman, R., Bennett, S., & Del Mar, C. (2017). Evidence-based practice across the health professions (3rd ed.). Chatswood: Elsevier Australia).
- For many study designs, there are more than 1 type of checklist available – choose which appraisal checklist you would like to use. You do not have to submit a completed critical appraisal checklist, please present a summary of your critical appraisal as per the below format guidelines/ template
- APPLY: You will apply your knowledge through two methods:
- ASSESSMENT TASK 2: You will apply the results through an OSCE where you will discuss the evidence on your topic with a patient
- ASSESSMENT TASK 3: You will also write a report using the template below creating a cohesive evidence-based summary for other health professionals working in the field
Format Guidelines for the CAT
| OSCE COMPONENT |
You are an occupational therapy student who has been asked to discuss the research evidence for your clinical question with your adult client (Mr/Mrs Jane Smith). You will demonstrate what you (as a therapist) would actually say to your client about the clinical question you have researched, and how you would actually communicate this information. You can assume that your client is interested in the findings of your clinical question as it relates to their own health condition/intervention or that of the person they are caring for (child or elderly family member). Your session with the client should provide a general summary of the research evidence and outline how it applies to the client’s health condition/intervention.
You should present the information professionally and in language that a client will understand e.g. focus on what would be considered important for the client to know and how to best describe the evidence without using clinical/statistical jargon.
Your session will run for 7-10 minutes. Please remember to wear your Bond Occupational Therapy uniform including fully enclosed shoes.
Your session should include the following components:
- A brief introduction
- A brief overview of the topic (health condition and intervention) that is relevant to your client scenario (explaining the problem) and your clinical question (in language relevant to a client)
- An invitation to your client to engage in a discussion with you about the topic
- A summary of the key findings of relevant research related to the clinical question. Results should be presented generally to your client in a natural conversation.
- Outline any key considerations/options for this treatment relevant to your client e.g. explain what the options are, what is involved in the intervention
- Include a basic summary of the research results (e.g. probability, or likely size of the benefit or harm, where this is known from the SR; as well as prognostic or qualitative summary of other results from your CAT)
- Outline any other information to help your client make an informed decision such as benefits or harms or resources required. These may include time, costs, inconvenience, interference with daily roles, social supports, quality of life issues etc. You should consider how the proposed intervention option might be accessed locally and what, if any, funding options may be available (only brief details need to be provided).
- If your evidence was inconclusive from your CAT, you may wish to describe some general pros and cons about the treatment intervention
- If your evidence was against the effectiveness of the treatment intervention, discuss the implications for your client and consider an alternative intervention
- Explore your client’s values, preferences, and circumstances
- Check your client’s understanding throughout the session / at the end of the session and answer any questions
- Make or defer the decision
- Brief summary / close of the session
In your preparation, consider what questions/comments the client may respond with. Think about what you would want to know if you were in the client’s position.
OCTY71-107 Rubric – Critically Appraised Topic OSCE:
| Criteria | High Distinction (85-100%) | Distinction (75-84%) | Credit (65-74%) | Pass (50-64%) | Poor (0-49%) | Grade |
| Work provides evidence of an exceptional level of achievement of subject learning outcomes. | Work provides evidence of a very high level of achievement of subject learning outcomes. | Work provides evidence of a high level of achievement of subject learning outcomes. | Work provides evidence of an acceptable level of achievement of subject learning outcomes. | Work provides evidence of minimal learning in relation to subject learning outcomes. | ||
| Summary of key findings of relevant research including: a brief introduction to the topic, summary of the clinical question, basic summary of research results including relevant benefits and harms to treatment | Thorough understanding of main issues that concern application of this evidence. Clear and comprehensive explanations and discussion of implications. | Very good understanding of main issues relating to application of evidence. Very good explanations and discussion of implications. | Presented most of the main issues relating to application of evidence. Good explanations and discussion of implications. | Presented some of the issues relating to application of evidence. Implications discussed with some omissions. | Incorrect or inadequate presentation of the issues relating to topic. | /10 |
| Shared decision making: Client was invited to engage in discussion about the topic. The client’s values, preferences and circumstances were explored. The client’s understanding was checked throughout the session and an opportunity for questions was provided at the end of the session. A decision was made or deferred re: intervention implementation. | Highly collaborative client-focused approach to session, engaging client discussion. | Collaborative client-focused approach to session, engaging client in discussion | Good involvement of the client in discussion. | Could increase involvement in client in discussion. | Limited discussion of implications and client involvement. | /10 |
| Professional behaviours | Outstanding professional presentation and communication style (i.e., appearance, language, rate and tone of voice, eye contact and body language) during discussions. | Demonstrated a very good professional presentation and communication style (i.e., appearance, language, rate and tone of voice, eye contact and body language) during discussions. | Demonstrated a good professional presentation and communication style (i.e., appearance, language, rate and tone of voice, eye contact and body language) during discussions. Minor errors may occur in 1-2 areas. | Mostly adopted an appropriate professional presentation and communication style (i.e., appearance, language, rate and tone of voice, eye contact and body language) during discussions. Some errors were observed. | Inconsistently and inadequately adopted an appropriate professional presentation and communication style (i.e., appearance, language, rate and tone of voice, eye contact and body language) during discussions. Multiple errors and/or inadequate explanations, sign-posting or structure were observed. | /5 |
| Clinical skills | Demonstrated all required clinical skills (4/4), including introducing self, OT role, checking 3 points of ID, hand hygiene. | Demonstrated most of the required clinical skills (3/4) including introducing self, OT role, checking 3 points of ID, hand hygiene. | Demonstrated some of the required clinical skills (2/4) including introducing self, OT role, checking 3 points of ID, hand hygiene. | Demonstrated a minimum of the required clinical skills (1/4) including introducing self, OT role, checking 3 points of ID, hand hygiene. | Demonstrated none of the following clinical skills: introducing self, OT role, checking 3 points of ID, hand hygiene. | /5 |
| Overall grade | /30 |
| WRITTEN COMPONENT |
- Title / Topic
- Research Question: One sentence stating the clinical question.
- Clinical Scenario / Background: Comment on the background/importance of this topic (1-2 paragraphs). Use the literature to support your summary.
- Summary of Key Findings: A summary of the key findings after conducting the CAT. This is usually done in point form, with only the key points made. You should include:
- Number of eligible articles found
- Number of articles actually appraised for the CAT
- 1-2 sentence summary about the findings of each of the appraised articles
- Any relevant issues / aspects of the research question that haven’t been addressed by research
- Clinical Bottom Line/s: This is a brief summary of your overall findings from your appraisal:
- 1-2 sentences stating what the evidence is from the SR in relation to the clinical question (this should effectively provide an ‘answer’ to your clinical question).
- 1-2 sentences stating conclusions/findings from the additional study (prognostic, diagnostic, qualitative).
- Search Methodology & Results: For the treatment/intervention effectiveness search (i.e. the SR), summarise the following:
- Inclusion criteria: what criteria did an article need to meet to be included in the CAT?
- Exclusion criteria: what criteria would exclude an article from being included in the CAT?
- Search terms: identify the following that relate to your clinical question:
- Patient/population
- Intervention
- Comparison
- Outcome(s)
- Search strategy & results: Use table 1 as a template (example below) to outline databases searched the search terms and strategy (including use of MeSH, wildcards and Boolean operators) used, as well as any limits applied in your search. Also include the number of search results you received for each database, and the number of relevant search results after you applied your inclusion/exclusion criteria
- Comment on the efficiency of this search (in a few sentences) and anything you would do differently if you could run it again
Table 1: Search strategy and results
| Results of search for articles about intervention effectiveness (i.e. used to locate SR) | ||||
| Database | Search strategy used | Limits used in search | No. of search results | No. of relevant search results that met criteria |
| [EXAMPLE ONLY] PubMed | E.g. MeSH terms, keywords, synonyms, And / Or used, truncation or wildcards | Date, language etc. | 48 | 8 |
| CINAHL |
- Critical appraisal of each study as follows:
For your 2 included studies, summarise the study and appraise the validity of the study, using the following headings:
- Internal Validity
- What is the internal validity of this study? – comment on the internal validity (risk of bias) of each study. This section is usually approximately 5-8 points / sentences for each study based on internal validity appraisal criteria covered in the subject. You do not have to submit the completed critical appraisal checklist – just your summary of the main findings of this critical appraisal.
- There are specific questions to assess internal validity (risk of bias) which vary for each different study type. Refer back to your lecture notes or to the text book to determine how to assess validity for the different study types.
2. Results
- What are the results of the study? – what were the main findings of the study? Where possible, briefly report the summary statistics. These will differ depending on the study type. These may include mean scores, mean differences, treatment effect, 95% confidence intervals, p-values etc.
- Briefly describe / summarise the key findings: For example, were the results favourable or unfavourable; consider the effect size and precision; were the results considered statistically significant and/or clinically significant etc.
For a qualitative study, summarise the key findings/themes, and support with a quote if available/relevant.
- When summarising the key findings, focus on the outcome/s of relevance
- Note: Outcomes of relevance refers to those outcomes in the article that are of relevance to your clinical question. You do not need to describe all of the results that an article may have measured unless they are relevant to your clinical question or should be considered when applying the evidence to clients.
3. Clinical applicability
- How can this evidence be used to inform clinical practice?
- Consider the study objective – what did the study aim to do?
- Review the Methods section – who did the study apply to?
- Systematic review: selection criteria, search strategy, included interventions, and outcomes
- Prognostic study: describe sampling, participants, and how predictors and outcomes were measured (with what, when, and where)
- Qualitative study: describe sampling, participants, and other key aspects of methods
8. Discussion:
- Summarise the results: Is this intervention effective? Is it clinically significant? What is the current state of the evidence? Does the literature support the use of this intervention? If so, who with and in what settings?
- Outline any key considerations for delivering this treatment – e.g. resources such as time, experience and cost (up to 1 paragraph). You may need to do additional research to answer these points. Include these additional references in your reference list using APA 7th edition referencing style.
- Comment on the need for any other evidence to inform decision-making with respect to this treatment – cost-effectiveness, quality of life issues, etc. (2-3 sentences).
- Where results suggest evidence against the effectiveness of this treatment, discuss the implications. Where evidence is inconclusive, comment on the appropriateness of studies that could provide stronger evidence, or other actions to clarify the situation (1 paragraph).
- Did the key messages from the prognostic study and/or qualitative study provide more information to inform treatment of people with this condition and if so, what were they? (2-3 sentences).
- Outline any limitations of this review – This refers to any limitations in the process of undertaking this review e.g. relating to the available evidence, search strategy etc. (This does not refer to individual limitations that may be mentioned in included studies).
- References used throughout the review to answer the question. Use APA 7th edition referencing style.
- Appendices: Include a copy of the 2 articles
A Systematic Review of Scientific Studies on the Effects of Music in People with or at Risk for Autism Spectrum Disorder
OCTY71-107 Rubric – Critically Appraised Topic Report:
| Criteria | High Distinction | Distinction | Credit | Pass | Poor | Grade |
| Quality of background details | Clinical question and relevance were thoroughly described. Search strategy and search results were thoroughly described with process for selecting citations clearly explained. Literature search clearly tried to locate highest quality of evidence (most appropriate to study design) | Clinical question and relevance were described with good detail. Search strategy and results were described with good detail and clear to follow. Literature search was conducted well using thorough methods and located high level evidence | Clinical question and relevance were described with minor omissions. Presented summary of search strategy and results with minor omissions. Literature search located appropriate evidence | Clinical question and relevance were described with some omissions. Search strategy and results were described with some omissions. Literature search located adequate evidence. | Clinical question and relevance were partially described. Presented limited information on search strategy, search results, and selection process. Literature search did not locate appropriate evidence | / 10 |
| Summary of evidence, including appraisal & interpretation | Accurate clinical ‘bottom line’. Comprehensive summary of internal validity. Key statistical concepts and meaning of results were discussed appropriately and accurately. Key clinical considerations accurately summarised. Comprehensive discussion. | Very good clinical ‘bottom line’. Very good summary of internal validity. Key statistical concepts and meaning of results were discussed appropriately and accurately. Very good and mostly accurate summary of key clinical considerations. Very good discussion. | Mostly accurate clinical ‘bottom line’. Most concepts of internal validity and key statistical concepts and results were discussed appropriately and accurately with minor omissions. Mostly accurate summary of key clinical considerations. Minor omissions in discussion. | Some of the key evidence was accurate and summarised, appraised and discussed with some omissions. Some key statistical concepts were used with some errors/omissions | Incorrect or inadequate use of evidence, with limited summary, appraisal and discussion. Limited statistical concepts and results presented | / 25 |
| Written presentation | All written format and content requirements were met. Layout and presentation supports readability and professionalism. Concise writing style with excellent attention to detail. Correct APA referencing | Most written format and content requirements were met. Very good layout and presentation of report. Clear writing style and very good attention to detail. May have minor errors in APA referencing | Most written format and content requirements were met. Good layout and presentation. Writing style appropriate with minor errors/omissions. Some APA referencing errors | Some omissions in written format and content requirements. Adequate layout and some errors in presentation. Some errors/omissions in writing style and APA referencing | Incorrect or inadequate written format and content. Poor layout and presentation. Multiple errors/omissions in writing and APA referencing | / 5 |
| Overall grade | /40 |
