Assessment of Students

How do you evaluate student competence beyond a written examination? Assessment in EMS education requires looking at the whole picture to ensure future Clinicians are ready for the field.

In this episode of Registry Insider, host Bill Seifarth connects with Dennis Edgerly, Director for the EMS Academy at Arapahoe Community College. Filmed on location at the Accreditcon conference in New Orleans, they dive into the essential components of student and program assessment.

The conversation focuses on balancing cognitive, psychomotor, and affective domains to ensure comprehensive student development. Dennis outlines how the affective domain serves as a daily continuum rather than an isolated unit, highlighting the importance of evaluating professional presentation, preparation, and peer interaction. They also discuss utilizing the clinical judgment model to build deeper examination questions, moving away from superficial recall and aligning classroom education with real-world practice.

Watch the full episode to learn more about optimizing your program assessment tools and leveraging National Registry blueprints for student success.

Transcript

SPEAKER_01 0:13

Welcome to the Ridge Insider for today's episode. I am filming from the Acredicon conference in New Orleans. And joining me today is Dennis Dennis Edgley. He's the director for the EMS Academy at Arapaho Community College in Littleton, Colorado. So, Dennis, thanks for joining me for the first time you've been on one of our episodes. Yeah, it is the first time. Thank you. I appreciate the invitation. So, as I said, uh, we're filming here from uh Credicon, and you did a pre-con on student assessment um a couple days ago. Yeah. So um give the fundamentals as far as what student assessment is, and then we're gonna kind of dive into some different aspects of it. Sure.

SPEAKER_00 0:52

Uh so I think it's uh it's a big thing. It is. Um all paramedic programs um have given by CoA the guidelines that we have to assess and um verify that our students are competent cognitively, uh psychomotor-wise, and affectively. Um the pre-con session, we spent the day talking about different means and methods by which we can do that in a summative format, um, in a formative format. Um, and then we talked a little bit about how to um assess the programs um at the end for outcomes and to make sure that our assessment tools are good.

SPEAKER_01 1:24

Yeah, so it's not just assessing the individual students, it's also useful to assess the program to make sure that one learning has taken place, both at the student and that the program is doing it effectively.

SPEAKER_00 1:36

Absolutely. You could have a great tool that functions really, really well, but if the tool's measuring the wrong thing, um, it's gonna be reflected in outcomes and student performance.

SPEAKER_01 1:46

So um, and and you rattled off obviously cognitive, psychomotor, and affective. Uh, so there are common resources available to help instructors uh better assess from a cognitive perspective. I think there's a lot of resources out there. Some are good, some are not so good. Um, what resources are out there, I guess, for affective domain or psychomotor where educators can maybe learn more or brush up on their ability to assess either their program or their students?

SPEAKER_00 2:13

Sure. Uh yeah, great question. The the affective role, um I we present it as kind of being not an individual component, but more of a thematic attitude or action as the students go through course. So um, and I proposed a couple um uh grading matrices for for affective, um, which focus mostly on presentation preparation and interaction. And so it's uh almost a daily graded event. Um, you know, are they on time? Did they do their homework? Can they interact in um appropriately in classroom discussion? Are they prepared for the skills? Um, how do they interact? Are they polite to themselves, to courteous, to students and faculty members and and things like that? And so I we we built it in as not an isolated, like you might do a unit test. At the end of the unit, you do a test. I don't think affective is that. I think affective is a continuum, and it then plays a role in how they do in skills and do they show up to clinicals on time and do they participate on time?

SPEAKER_01 3:07

All of that is part of the affective piece, which also helps them to ensure they're gonna be interacting with the patients appropriately. Correct.

SPEAKER_00 3:13

Yeah, yeah, yeah. And we there's so many different points in time at which we measure it. I mean, clinical evaluations have affective reflection. Um uh internship has affected reflection associated with it. Um, and then I think in the classroom and lab, um, we talked a lot about how to um evaluate them on a daily basis. Um and just the other thing we talked about is we have to make sure that we we teach it. Um, I wouldn't test an EMT student on how to start an IV or administer medication without first teaching it. Um and so I think the same with affective. We just can't expect students to know how to behave and expect students to know what to do.

SPEAKER_01 3:48

So uh that's affective, which is really helpful. Um are there any tips that you have for psychomotor or cognitive examination or assessments, I should say, from the from the educator perspective.

SPEAKER_00 4:01

Yeah, so the the cognitive piece, uh we we have to make sure that when we're doing and usually we do that through written tests. But we talked about other ways. We can do it formative, at least through um oral exams, uh, one-minute papers. Um, you can get a sense of it, uh, their cognitive knowledge and debriefing through simulations, um, just to kind of see where they are and hear what they have to say. Um the the formal assessments through the written exams, we talked a bunch about making sure that our assessment tools are good. We need to make sure that they're valid and that they're reliable. Um validity meaning obviously that you know we're actually testing what we want to test. Um and reliability, that they're actually um being able to be reproduced. They can test the same student in the same way. So we talked a lot about um that. We talked a little bit about test construction, but the nature of the program wasn't such that we could spend a lot of time on test construction. And we touched a little bit on um evaluating the test. Um but we talked about uh the resources that are out there. Uh, the registry website, I sent them there. There's uh some blueprints on the tests, um, and there's some practice exams on the test, which many of the participants didn't realize that they could actually go and witness what the test was like.

SPEAKER_01 5:06

Yeah, the look and feel as they would experience in a Pearson testing center. Yeah. Um, and like you said, the test blueprint, which is available at our website, nrmt.org. Um, they can go there for certainly uh sample items, some uh example uh with the test blueprints, so they know exactly what percentage of questions are gonna be asked on the various domains. So um where else can people go? I know obviously CoA MSP has some resources as well.

SPEAKER_00 5:31

Yep, uh yeah, CoA has a bunch of resources on the resource page as far as what to do, how to do. Uh there's a bunch of great courses here um for people with test construction and how to write uh the test pieces. Uh we referenced back the the clinical judgment model um a lot. We talked about that um and teaching through that flow of recognizing cues, analyzing cues, um, and but then using that as a model by which they can test students. Um and we talked a lot about the the Blooms piece to increase uh the depth of the question that they're asking um and to avoid just asking some superficial uh questions. Simple recalls.

SPEAKER_01 6:06

Yeah.

SPEAKER_00 6:07

Right, right. Which and we had some good discussion in the class that that's harder than it a lot of people think. Um and we talked about the fact that sometimes they have this great scenario question that they'll write, and there's all these words, and at the end they just ask a simple recall question. Yeah. So they're burying the recall questions in these scenario questions. Yeah, and making it louder than it used to be in the case. Correct. Right, which has all kinds of testing detriment associated with it.

SPEAKER_01 6:28

So uh as it relates to the clinical judgment, um, on our website as well, you can get information more about clinical judgment. Yeah. Again, entering t.org, and all you need to do is search for clinical judgment. And certainly sample items will come up, but equal important, uh bit of uh some research that was published on that, yes, uh, where we were able to kind of validate the model and then provide information about it.

SPEAKER_00 6:50

So which I think is which is great. And and we encourage people not to use it just so much as a uh a tool for for testing, but it's actually helped pre-hospital providers work. Yeah, exactly.

SPEAKER_01 7:00

It's valid. And to that end, if you educate the students to function the field, A, they're gonna pass your exams, or B, they're gonna pass the national registries exam as well. So that's the most important. Any other key takeaways from the session you provided on assessment of the program and or the students?

SPEAKER_00 7:18

Yeah, I we did talk a little bit about um cycle motor and we talked about the difficulties associated with that. Um, like how do you set parameters and how do you set, you know, like with IVs, for example, what's the percentage that you have to have? And um, so there's some good discussion and not a lot of absolute definitions. I mean, people throw out the word competent, but sometimes that's hard to define. Yeah. And then we we did the session ended up with um how we reflect on our on our own program. So we could think, again, we have the best tests ever and the best cycle motor skills, but if they're getting out and they they're not performing well when they get jobs, or if they're not passing the registry exam, uh, that would suggest that maybe what we're doing either educationally or testing-wise, um, is not the best thing. And again, I referenced them back. We referenced back to CoA to the uh the RAM, the resource assessment matrix and student evaluation and graduate evaluations. And we we've brought all those in um and the importance of them as far as um being part of the assessment process. Excellent.

SPEAKER_01 8:20

Well, thank you for sharing the highlights from your uh your pre-con here at Accredicon and thank Richard, obviously, for EMS Education. My pleasure. Thank you. Thank you as well for joining us, and as always, stay safe. Thank you for joining us for this episode. If you wouldn't mind, please click the like and subscribe buttons as well as the notifications so you can get notifications of upcoming episodes. Also, for the latest and greatest happenings of the National Registry, feel free to go to nrmt.org. Thank you very much, and again, stay safe.