The True Value of Metrics for Effective L&D with Dr. Heidi Kirby
Episode Overview
Welcome to another episode of Making Better, the podcast that explores how we can improve ourselves, our teams, and our organizations. In today's episode, we have a special guest, Dr. Heidi Kirby, a renowned learning and development professional with a wide range of experience in various industries. Heidi is the co-founder of Useful Stuff, a website and newsletter dedicated to providing helpful resources for L&D professionals. She is also the host of the Block Podcast, focusing on building learning and organizational culture. Today, we dive into the topic of making training useful by utilizing metrics to ensure we are truly impacting the business. Join us as Heidi shares her insights and strategies for connecting learning and development initiatives to the overall goals of the organization. So sit back, relax, and get ready to make better with Heidi Kirby.
About Dr. Heidi Kirby
Based in Cleveland, Ohio, Heidi is co-founder of Useful Stuff which you can find at getusefulstuff.com and host of the BLOC podcast which stands for Building Learning and Organizational Culture. She got her start as a college English professor but has since worked as an instructional designer for NASA and an L&D leader in the tech industry. She is passionate about helping aspiring and L&D professionals be their best and helping L&D teams become more inclusive and efficient. Today, we are going to talk about making training useful, specifically by using metrics to make sure you are actually impacting the business.
Full Transcript
-
Heidi Kirby [00:00:00]:
We have these frameworks in L D that are very prescribed, right? And they're a lot we've become more learner centered, which I think is great, but we serve more than just our end audience, right. Like, we also have stakeholders and SMEs that are involved, and we also have our overall organization if you're full time or our clients if you're freelance, right. And those people are important too. They're our audience too. And we so quickly forget about the metrics that are tied to what they need.
Matt Gjertsen [00:00:33]:
Hello and welcome to the Making Better Podcast, where we talk about how to make better, whether that is better selves, better teams or better organizations. If you are a business owner, a learning and development professional, a manager, or even an individual contributor in your organization, this show will give you actionable insights to help improve your own performance and the performance of those around you. Our guest today is none other than Dr. Heidi Kirby, based in Cleveland, Ohio. Heidi is co founder of Useful Stuff, which you can find@getusfulstuff.com, and host of the Block Podcast, which stands for Building Learning and Organizational Culture. She got her start as a college English professor and has since worked as an instructional designer for NASA and an L D leader in the tech industry. She is passionate about helping aspiring learning development professionals be their best and helping L D teams become more inclusive and efficient. Today we are going to talk about making training useful, specifically by using metrics to make sure you are actually impacting the business.
Matt Gjertsen [00:01:40]:
Before we get into that discussion, I need to remind everybody new to the show to make sure you hit subscribe so you never miss a future episode. And if you are already subscribed, then I want to ask you to share this show with at least one other person because that is how we grow. I can't tell you how much it means to me. So with that, let's get started. Heidi, how are you doing today?
Heidi Kirby [00:02:03]:
I'm good, how are you?
Matt Gjertsen [00:02:05]:
I'm doing great. So we've known each other for a few years now. I think we first got introduced when I reached out to you to be on your podcast. So I've loved following you for all these years on LinkedIn if you want to. I know I gave a little bit of an introduction, but if you want to tell anybody, listening a little bit more about yourself. Yeah. Who are you? What have you been doing in the space?
Heidi Kirby [00:02:30]:
Yeah. So I'm a former college English professor turned instructional designer. I got my first ID job with NASA and then from there have just kind of bounced around to different industries and organizations. Tons of different like, most of us worked for a tire distributor, healthcare industry, the tech industry, and most recently have been in the customer education space for the last few years, helping with software and product training and things like that. And then, yes, I started a pandemic podcast that you've been on that you mentioned already, the Block. And that's still going to this day. I think we're like 70 some episodes in at this point, which is pretty awesome. And so, yeah, I have that.
Heidi Kirby [00:03:18]:
And then about a year ago, my partner, Matt Smith and I co founded Useful Stuff, and right now it's a newsletter and website for L D professionals, helpful resources for L D folks. But the plan is to keep that going and growing and have some cohort based courses and ebooks and self paced courses and things like that in the future.
Matt Gjertsen [00:03:45]:
I love it. And I remember when Useful Stuff started and I was just like, oh, that is such a perfect name because it really is. Is there a story behind it?
Heidi Kirby [00:03:56]:
We agonized over it. We agonized over it. There was one point where I was like, okay, grab a book closest to you. Grab a book that means something to you and turn to this page and give me a word. We spent so much time just agonizing.
Matt Gjertsen [00:04:14]:
Over what we would call it.
Heidi Kirby [00:04:15]:
And I have these books right here that you can see behind me that are like, really old copies of Anne of Green Gables that are from my grandmother. So I picked a really meaningful book and said, okay, here's my word. And then Matt just picked a silly book and picked a really silly word. And I was like, you're not taking this seriously. And so we just kind of landed like, what do we want this to like, we want this to be useful Stuff. And we're like, what's wrong with that?
Matt Gjertsen [00:04:44]:
Let's do that.
Heidi Kirby [00:04:45]:
There it was.
Matt Gjertsen [00:04:46]:
Yeah, it's perfect. And I think it really highlights a lot of what I feel like your whole persona, your message is on LinkedIn and on the podcast is, how do we take learning development? How do we take talent development and make it more practical, make it more useful to the business? And so with that, I think the core of that really is metrics and knowing how do we connect what we're doing to metrics of the business. When I think back, I know a lot of people, when they think of learning and development, especially, they think back to the last time they were in school and when we can talk about why that's a false connection. But when I was in school, the metrics that mattered were scores. It was grades on test.
Heidi Kirby [00:05:33]:
Why participation?
Matt Gjertsen [00:05:34]:
Doesn't that metric work in the corporate.
Heidi Kirby [00:05:37]:
Frameworks in LND that are very prescribed? Right? And they're a lot fair enough, more learner centered, which I think is great. But we serve more than just our end audience, right? Like, we also have stakeholders and SMEs that are involved, and we also have our overall organization, if you're full time, or our clients if you're freelance. Right? And those people are important, too. They're our audience. Too. And we so quickly forget about the metrics that are tied to what they need and what say, object is nine times out of ten, somehow connected to that learner audience anyway. So it's important to look at the whole picture, right? So sometimes we don't care if our learners liked all of the right things to do, but that may not be where the gap exists. Right.
Heidi Kirby [00:06:32]:
So if we're looking at test scores and how people do on the end of the training exam or things like that, we're not really getting an accurate measure of how they're doing their job. We're getting a measure of how well their short term memory is functioning after they just took an Elearning. Right. And that doesn't even account for bad test questions. We don't even have to go down that rabbit hole. Oh, my goodness.
Matt Gjertsen [00:06:59]:
Yeah, that's bad.
Heidi Kirby [00:07:00]:
In what year was Such and Such founded? That doesn't help anyone, right. So you've got these measures that don't really accurately accomplish what you're trying to accomplish. If the goal of something is we want to increase sales, that's where we need to start the journey and the search for those metrics.
Matt Gjertsen [00:07:23]:
Yeah, I guess on that note, sales is a really clear example, but especially your background in customer education is probably great with this. How do you think about finding those metrics then? If if the core things to learning, if, like, the test is a bad one, if participation is a really bad one, how do you go about finding the metrics that are going to make a difference?
Heidi Kirby [00:07:51]:
Yeah, so I think you bring up a really good point that I want to touch on first and like, participation, engagement, we have these frameworks in LND that are very prescribed, right, and we've become more learner centered, which I think is great, but we serve more than just our end audience, right. Like, we also have stakeholders and Smeves that are involved, and we also have our overall organization, if you're full time, or our clients if you're freelance. Right. And those people are important too. They're our audience, too. And we so quickly forget about the metrics that are tied to what they need and what they're looking for, which is, nine times out of ten, somehow connected to that learner audience anyway. So it's important to look at the whole picture. Right.
Heidi Kirby [00:08:42]:
So sometimes we don't care if our learners liked an experience, right. If it has to do with safety or warehouse safety, everybody wearing their PPE. We don't care if you don't like the job aid or if you think it's pretty or not. We need it to be effective.
Matt Gjertsen [00:08:59]:
Right.
Heidi Kirby [00:09:00]:
And so starting like, what we really need to do and what I think is hard for new people is because we have Addie, right, and evaluation is at the end of the word, we automatically linearly organize metrics to be at the end.
Matt Gjertsen [00:09:18]:
Yes
Heidi Kirby [00:09:18]:
And that is totally bonkers to me because they make the most sense at the beginning in our needs analysis, right. If we think about our learning outcome as the overall goal, right. And typically it's somehow associated with the request that comes in, right. So in the example that I gave, let's say we want to increase sales 15%, right? That's what someone comes to us, they say, hey, we have a company goal of increasing sales 15%. We need your help with this. Whatever objectives we create as we are trying to solve the problem, as we're looking for the gap, that's going to be our metrics, right. And it lends itself to that, right? Like if the goal is to raise sales, we have to find out what's the numbers currently and then how are we going to fill that space. So it could be something that has nothing to do with us, right? So we talked about having a low sales performer and a high sales performer.
Heidi Kirby [00:10:24]:
Maybe the difference is that the high sales performer is selling to corporate accounts and the low sales performer is selling to nonprofits. It's not even this can be different. Ballpark yeah. Or in customer education, we see this a lot. They're giving something away for free during the sales conversation to help make the sale, right? So maybe they're giving away a specific feature. A lot of times they give away training and certification for free if that accounts for that 15%. Well, there's an easy way to solve that problem right there that has nothing to do with us. But maybe it's that people are not upselling or people are not able to explain the value of the product in a way that makes sense.
Heidi Kirby [00:11:07]:
And that's where we can then come in and say, okay, how are people explaining this currently? And then how do we get them to explain it better? And then how do we measure that? How do we measure that people are doing that? Do we record our sales calls? Can we compare the two? I mean, there's so many different ways if we can use those problem solving and critical thinking skills to figure out what metrics make the most sense depending on the situation.
Matt Gjertsen [00:11:35]:
Absolutely. And I think this is really important for any of the listeners this is doubly important for any of the listeners who are not in L and D themselves, but are out in the business. Because I at least have found that a lot of times people out in the business again going back to this idea that they just think of school. So when they think of engaging with a training or a learning development team or doing some kind of intervention, the only thing that is in their head often is knowledge, is changing the knowledge level of the people on their team. And so it's just simply like, what's the point? What are you trying to do? What's the outcome that you're trying to achieve? Why do you think this knowledge is important? I guess. How often have you in your career or have you worked with people, had to navigate that discussion of how do you get the subject matter expert past that knowledge? Thinking into the why does this matter? Thinking what's the outcome?
Heidi Kirby [00:12:32]:
Thinking it's almost every time.
Matt Gjertsen [00:12:34]:
Right.
Heidi Kirby [00:12:35]:
Because we're in this unique space where, like you said, people have enough knowledge of what we do to be dangerous, almost. Right. Like they've had some experience with education or training, so they think they know something about it. Unlike something very technical or like finance. I'm never going to pretend I know anything about accounting for business.
Matt Gjertsen [00:12:59]:
I've done my taxes, but I'm not doing corporate taxes like web development.
Heidi Kirby [00:13:03]:
But people have opinions about training and education because they have an experience. And so every time we walk into a room with a stakeholder or SME, we have that whatever those biases are, whatever those good or bad experiences are, those things that we have to kind of get past. Especially when someone has an idea of elearning and they're like, well, this has to be we need an elearning on such and such. Right.
Matt Gjertsen [00:13:31]:
Yeah.
Heidi Kirby [00:13:32]:
And so we have to move past those biases. We have to build a relationship with those people and ask the right questions to get to what you're saying. Like the why does this information matter? Why is it important? What are we trying to accomplish here?
Matt Gjertsen [00:13:51]:
I love everything you just said because it really, honestly, for the first time, connected in my mind something that when I'm training new managers, one of the things that I say is that when you get a new person on your team, for better or worse, when you first start your relationship, you are the worst manager they have ever had. They're taking that past knowledge of their worst experiences and applying it to you. And the whole process of welcoming somebody on your team and being a good manager is showing them who you are, getting past those preconceived notions, building that relationship. And so the same exact thing is true when you are working with a new stakeholder or SME, you are all of their worst training experiences. They're coming to you with that. And there's such an opportunity there of like, let's just sit down and talk about, hey, what have you experienced before? This is what I've done before. This is how we like to think about it at this company. I don't know of anybody that ever really takes the time to have those kinds of conversations.
Heidi Kirby [00:14:56]:
Yeah, well, because even if they do, sometimes it can seem like interrogation.
Matt Gjertsen [00:15:02]:
Sure.
Heidi Kirby [00:15:02]:
Right. Like doing a proper needs analysis. There's a lot of questions to ask. And I've actually found in my dissertation research where I talked to instructional designers about their experience. The strangest anomaly in my research was two people telling the exact same story of how they tried to ask questions in a needs analysis, but didn't really have a relationship with this mes and didn't really have the rapport or didn't really give enough context as to why they were asking all these questions and the person got so defensive. And these are like two people who've never met each other telling me the same story, that the person that they were working with got so defensive that they asked for them to be taken off the project.
Matt Gjertsen [00:15:45]:
Wow.
Heidi Kirby [00:15:46]:
And in both cases, the person was. And so I thought that that was wild that I found that in my research. It's like to get the data that we need, we have to collaborate with people. We can't just be number crunchers. We have to have conversations and find out where the data is living, which also requires collaborating with other people in other departments and across the organization.
Matt Gjertsen [00:16:13]:
Yeah. It can sound really like an interrogation when you're doing it. And especially because the root of all this, the reason always comes back to there is a performance gap somewhere, for sure. Those questions, you're like, they're always going to turn to, like, you're a bad manager for letting this happen, or you're not doing something which is not what you're trying to do, but if you don't have a relationship, that's how it's going to come off.
Heidi Kirby [00:16:42]:
Of course, even if it's an opportunity. Right. Like, we don't always have to solve problems. It could just be, hey, we don't really offer anything for our new leaders and we're growing, and it's about time that we start doing that. It can still seem like, well, why are you asking me all these questions about leaders?
Matt Gjertsen [00:17:00]:
One other thing that you mentioned that I wanted to pick up on too, is first there's the identifying the problem and then there's measuring the current state, like getting the numbers. And that's kind of a challenge in and of itself very often, isn't it?
Heidi Kirby [00:17:17]:
It is, but a lot of times if you figure out why the person is bringing to you this request, there's usually some hidden metric in there where got the information from, even if it's just anecdotal, right? Well, I had three managers last week tell me that they can't get their guys off the phone when they're supposed to be working on the floor. And so we're having a really bad problem with people using their phones too much and this that. So we just need people to be paying more attention. And so it's a matter of, okay, well, how do you know this and where is this coming from? Sometimes it's, well, we have really bad leadership scores according to our employee Engagement survey. There's a metric right there, right? Okay, well, we've got this employee Engagement score that's telling us this. We've got the current state. This is what we're going to go back and measure when all is said and done right.
Matt Gjertsen [00:18:23]:
And in situations where it is anecdotal. How do you think about, what if there just really is no data? Where it's always been more of a feeling like, can we take on any of that? Do you ever find those situations?
Heidi Kirby [00:18:40]:
Yeah, a lot of times you'll have someone new come in, right, a new leader, and it's like I don't know if you remember the show The Magic School Bus, but there's this student who's like, at my old school, we always did it this way. Right, and you find those people in your workplace, too, where they're like, we need to have a state of the art, two day in person leadership training for every new manager that comes through. Well, why? Well, because at my old place, we did it and it was really successful. Right. And so in those kinds of situations, if you can get your hands on the end audience and get some information from them, even if it's just like casual survey, casual conversation, just collecting stories, I think it's fine to do something based on someone's. Hey, we think this might be a problem, but we still have to talk to kind of the end audience and see is it really a problem? And if not, if it's just an opportunity and they're coming to us and saying, like, hey, we want to do this because we think it could help, or we think it could make an impact, there's still different ways that we can find to measure the current state, especially if we have access to our audience. That can be helpful.
Matt Gjertsen [00:20:08]:
Yeah. I think it's always best when, like you said in your investigation of the reasons for the request, you kind of uncover, oh, there is actual real data out there. That's why sales is so great, because they always have something.
Heidi Kirby [00:20:26]:
Yeah, sales and marketing. Marketing has like, so much nowadays, they have so much data. Like everyone right. Even help desk tickets. Right. That's a great place where we can go to learn what our people are missing and where their skills gaps are and things like that. Performance data. There's so many different areas.
Matt Gjertsen [00:20:47]:
Yeah. But then if there isn't, then I do think kind of like you were saying, there are opportunities to it's not perfect, but do some surveys and use that. Take the baseline. That's honestly one of the reasons why putting, as you said, that evaluation first is so essential. Because if you don't have a baseline, then you have no idea what you're doing.
Heidi Kirby [00:21:10]:
Yeah, exactly. And I know it's sometimes hard to sometimes you'll be told, like, oh, no, you can't go talk to the learners, or oh no, you can't go talk to it. But that's a whole reason why you make friends in your organization. That's the whole reason why you don't want to be so separate from the business and L and D, because the business is us, too. Right. If we exclude ourselves from that, then we don't really have the opportunity to make an impact. And I was always going to my friends and other departments and being like, hey, let me pick your brain about this. You don't have to ask for permission to talk to a work friend about a situation.
Matt Gjertsen [00:21:47]:
Right, that's such a great point because yeah, getting the true, unvarnished truth from people of, hey, we need training. We have this manufacturing problem, this quality problem that's always happening. We need new training on it. And then you go talk to your friend. It's just like, no, we just don't have the right tools. We keep doing workarounds. You're like, oh, okay. And it can be really hard to uncover that sometimes through formal channels.
Heidi Kirby [00:22:15]:
Absolutely.
Matt Gjertsen [00:22:16]:
But I've had a couple of discussions where this has come up, just that the fact that L D by its very nature, is always cross department, cross functional. And so it presents us with this amazing opportunity of being this connective tissue for an organization. But that means you need to get out there and build that connective tissue, because it doesn't just happen. It's the difference between connective tissue and a cancer. If you don't have those relationships, then you're like a cancer coming in and attacking and interrogating and have this even.
Heidi Kirby [00:23:02]:
Just making the wrong thing all the time. Right. Like, you're just not meeting the needs. And eventually the people in your organization and I've seen and heard of this happen a ton. They'll go somewhere else. They'll hire contractors.
Matt Gjertsen [00:23:17]:
Oh, my goodness, yes.
Heidi Kirby [00:23:18]:
Organizations will literally have an L and D team and will hire contractors because they found that their L and D teams are so hard to work with.
Matt Gjertsen [00:23:28]:
Yes. And though I do think that sometimes not always, but one of the pitfalls that can make a team hard to work with is if they are so hard lined about their needs, analysis of having a very specific way to do it. And that's why I think there's a very delicate balance, especially early on or when working with somebody new. I always say the best way to not be an order taker is to start by being good at taking orders. You have to start by showing people when they come to you with a course, you've never worked with this person before. They come to you wanting a course. And you know immediately, that's not the answer here, but maybe you lightly interrogate it and then just say, yes, okay, let's do it. And that first time, you just kind of do it again.
Matt Gjertsen [00:24:18]:
Going back to it's all about the relationship.
Heidi Kirby [00:24:20]:
It is. And it's like parenting. You have to pick your battles. Right? You have to, over time, figure out, how serious is this? On a scale of one to ten, how much pushback should I give? But what I've also found is that if you have a solid process, like if you have a solid needs analysis process in place for your team, or even if you don't, and you come in new and you say, hey, here's what I propose. Our process is if you say to people across your organization, oh, this is just part of our intake process, and you formalize it like that, people are far less likely to ask questions or make you fly past it. Or people honor processes. That's true in the corporate world. And so if you are like, this is our process, this is how we do things, you get a lot further, I found.
Matt Gjertsen [00:25:09]:
I think I would totally agree with that. I've actually been slightly different kind of process, but when I've created templates for people out in the business to use, I was expecting a lot of pushback and there was zero. Like, they love it when you give templates. They love it. Even though you're asking them to remake this work that they've already done, they still love it.
Heidi Kirby [00:25:33]:
Yeah, because you're giving clear expectations. Right. You're setting very clear, like, here's what the information I need to help you. And people like that because it's useful.
Matt Gjertsen [00:25:42]:
Yeah, I couldn't agree more. So anything else we're missing around this whole metric thing? What haven't we talked about? What else is important?
Heidi Kirby [00:25:51]:
I think the only other big thing, and we're building a useful stuff evaluation framework right now because I was doing an ID coaching cohort and everyone was like, let's talk about evaluation and metrics and let's talk about what are the most common frameworks and models out there. And I went back through all of my stuff from my PhD program in instructional design, and I was like, man, not only are a lot of these pretty archaic, and they're not designed for a very modern workplace where we have a million different options of how we can present information and do learning projects, but they're also all for the same exact thing, no matter what you're creating. So if you're creating a one page job aid to teach someone how to properly put back a piece of equipment, you have the same set of criteria as a seven day long in person leadership development training, and it just doesn't make much sense.
Matt Gjertsen [00:26:55]:
No.
Heidi Kirby [00:26:55]:
So different metrics for different projects. Right. And it really should be project dependent.
Matt Gjertsen [00:27:02]:
Yeah, I think that's totally right. And it's great that you're building that out. We'll make sure to link to Useful Stuff in the description so that people can find it anywhere else. People should look for you and connect with you.
Heidi Kirby [00:27:16]:
LinkedIn is probably the best place. If you connect with me on there, I'll link you out to all the other stuff that I do.
Matt Gjertsen [00:27:23]:
Excellent. Yeah, you're very active on there. I always appreciate it. So yeah, I highly recommend whether you're in learning development or also outside of learning development, I think, again, it's all about being practical and helping the people out in the business. Highly recommend everybody connect with you. Well, thank you so much for the discussion today. I know this is always a huge topic, and I know you're always super busy, so thank you for taking the time.
Heidi Kirby [00:27:54]:
Yeah, thanks for having me.
Matt Gjertsen [00:27:56]:
Have a great day
Heidi Kirby [00:27:57]:
You too.
Matt Gjertsen [00:27:58]:
Thank you so much for tuning in today. If you liked the discussion, make sure to hit like and subscribe so you never miss an episode. As a reminder, if your team is struggling keeping up with the training development demands of your organization, we want to help. Better Everyday Studios is a full service instructional design team that can help you with everything from ideation to actual content creation and livery. Please reach out to us using the link in the episode notes below. Have a great day.
Thanks for Listening!
It means so much to me and the guests that you chose to spend your time with us. If you enjoyed listening, make sure you subscribe using your favorite player using the links below.
