Exploring Multipliers in Safety Performance Monitoring

In this podcast episode, a continuation of the SPIs series from last week, the focus shifts to the concept of using multipliers for improved safety metric measurement. 
The host, Jason Starke, emphasizes the significance of utilizing rates as a more accurate way to monitor safety performance, accounting for operational variations. The key idea here is employing multipliers to transform decimal-based rates into whole numbers, simplifying data representation on graphs. The process involves dividing event numbers by operational quantities and then applying a chosen multiplier, leading to clearer and more comprehensible graphs. The episode concludes by highlighting the practical advantages of this approach in enhancing data visualization and decision-making.

Contact Safety Chats

• Tweet us at @BaldwinAviation
• Connect on LinkedIn https://www.linkedin.com/company/baldwin-aviation-safety-&-compliance/
• Follow on Facebook @BaldwinSafety
• Email us at baldwin@portside.co

Learn more about Safety Management System’s at www.BaldwinSMS.com

Thanks for listening and stay safe!

Transcript

0:00

Hello, Safety Samurais! I don't even know if that makes sense. Trying a little bit of alliteration there, but how is everybody doing? Welcome to the Safety Chat Podcast. As podcast where we talk about all things SMS, safety, a little organizational behavior sprinkled in for some extra flavor. the last podcast, this is Part 2 of SPIs. And the last podcast, we talked about alert levels. And, had a little fun with that, got a little derailed because I started talking about other things, but I think we circled back and came back home on alert levels, but in this podcast, we're going to talk about a little different concept again, the self surveying notion here is that we're implementing this into Baldwin, so again, super stoked that we're getting these improvements to safety, performance, monitoring, and measurement. But it's the idea of a multiplier. Now, this is going to be super hard to do without pictures. The idea of a multiplier is actually very common in our industry. And we're going to implement this principle for various reasons in the Baldwin system. And it's something you can absolutely do if you're not a Baldwin client. But, the idea is... that we are talking about rates. So let me start here for a second. Rates, to me, are the premier way to monitor and measure. And the reason being is because unless we are doing the same quantity of operations that we're measuring every single month the exact same and which is it's not feasible because some months we fly more than others or some months we move more aircraft than others. Some months we fuel more than others. So we do vary from month to month. But unless we do the exact same quantity of operations every single month, using measurements not associated to a rate are going to be a little misleading. Let me give you an extreme example. So let's say I flew 10 hours, slow month, the boss only just wanted to go to the golf course and back in Hawaii. And in 10 hours, I had one unstable approach. Okay? So let's say that was the month of July. And so in July, if I'm measuring quantity, I put a little data point in one for unstable approaches, and then I say, okay, great. I'm monitoring unstable approaches. I got one. Fantastic. So now let's say the next month what would that be? August. The boss is starting to ramp up. He's looking at expanding IPOs, all that kind of junk. And you are flying your tails off. And so let's say you fly 100 hours. And let's say you have 8 unstabilized approaches. And so you're like, crap. Now you take a data point and you put it in the 8. You look at the month before and you say, Man, I only have one unstable approach. But the next month, I had 8. And you're like, wow, we've really got a problem here. We've gone, eightfold. We've increased. But is that really the case? And I would argue no, because if you look at it in rate, if you had one unstable approach per ten hours of flying, if you look at a rate, that'd be one divided by ten, which is 1, right? So as a rate, you would have 1 unstable approach per flight hour. Now the next month, if you were to take it as a rate, you had eight unstable approaches and you flew a hundred hours. So your rate is actually 08 unstable approaches. Now, if I remember math correctly, 08 is less than 1, and so therefore, because of the exposure and the increase, I'm not saying you don't have a problem, but I'm not saying it's an increased eightfold because your exposure was greater. So that's why I love rate, because it smooths out that variability and actually gives you The real picture. And so rate to me is, I just, I don't know. I guess I love it. And I love it over measuring quantity because quantity can be misleading because it doesn't take into account the volume of operations or quantity of operations. So now let's get to the. Multiplier, now when I tell you, you had 1 unstable approach per flight hour, the thing you're gonna focus on is what's 0.1 of a unstable approach that makes sense. You can't have one 10th of an unstable approach. You have to remember that's rate it's 0.1 per flight hour. That is your unstable approach rate. And so it, it's odd to look at it that way. So what the industry does, and we see this even with I K O introducing accident metrics because when we look at our industry, and let's go even into the airline industry, we would look at the number of accidents they have and the number of hours they fly globally, which is a quadrillion, and I know that's an exaggeration, but it's a lot of hours. So when you divide that out, if the industry had four fatal accidents. over a million hours of flying. That's a small number, right? It's like 00004? I don't know. I don't have a calculator in front of me, but it's pretty dang small. So what we do is we apply a multiplier to make numbers more palatable and more whole, 04, events per flight hour. If we did the multiplier of 100 hours, we would just simply multiply 0. 04 by 100, which would be 4, and we would say we've had 4 events per 100 flight hours. And that's nice. And you can put a bow on it. You can use any number. You can use a thousand. In this case, if you multiply it by a thousand, it'd be 40 events per thousand flight hours. If you did 10, 000, it'd be 400 events per 10, 000 flight hours. It's up to you. One of the craziest things I heard when I was actually working with another system, and it had multipliers, and it was actually a set multiplier of a thousand. A guy comes back and says, the corporate flight partner, We don't even fly a thousand hours. How can I expect to fly... It's not about what you fly. It's about making it an easier number to deal with. Even if you flew ten hours a month, you could still have a rate multiplier of a million. It'd just be a huge number. But it's up to you, and that's what we're going to implement into the Baldwin system. But I think for corporate operations especially the number of hours we fly in a given month I think multiplying by a hundred is probably a good place to start. Especially if your flight hours are in the hundreds per month. If they go higher, you can multiply by a thousand. But it gives a nicer... View of the data. It's really nice. Now, when we talked about reference levels last time, we also have to consider that when setting a reference level. So let's say, we're measuring our data and we're using rate and we're going by flight hour and then we're multiplying by 100 flight hours. And let's say we find the normal rate, our reference level, of, I don't know, 1 per 100 flight hours. Okay, so that's what our reference level is, and it has to be labeled as that 1 per 100 flight hours. If you just say 1 on your graph, they're gonna be like, 1 what? Not, one event. No, it's one per 100 flight hours, or 01 per flight hour times 100. However you want to label it. I don't want to make this too complicated. But you have to make sure that your labeling is correct so that people understand what you're trying to communicate on the graph. Same with the alert level. Now, let's say your reference level is 1 per 100 flight hours. Obviously, you'd want to scale your alert level the same. Maybe your alert level is 4 per 100 flight hours. And that is what the multiplier is all about. Otherwise, we'd be dealing with decimal points. We'd have 01 is our reference level, 04 is our alert level, and just, ugh, 04 per flight hour, just, to me that just doesn't make sense, it's sloppy, we got decimals in there, and here's the crazy thing, let's say one month you flew a thousand hours, and you had one event, and you have your reference level at 1 per 100 flight hours, and now you have 001 event. You're barely going to be able to see that data point because your graph is going to be scaled to that reference line, right? Of 01, and you're an order of magnitude 10 less, and it's like it becomes a mess. Multipliers are your friend. So here's how you do it. I'm going to go back to the Baldwin system, if you're a Baldwin client, we have something called Operational Logs, and that's where you put in your flight hours, flight days, cycles, whatever you're going to measure your rate by and then the system automatically puts it as the denominator and then you establish the SPI to measure by flight hours, by cycles, whatever, and I can go over that all with you if you give me a holler, and then once you establish that, it's going to ask you what's the multiplier? Okay. So that gives you a nice number and let's say it's a hundred. Okay, so I'm gonna measure the number of unstable approaches by flight hour with a multiplier of a hundred. Now when it asks you for the reference level, in the case of monitoring SPI, we're gonna have to keep that in mind. We're gonna have to be in that frame. So what it's asking me, once we set the multiplier of a hundred, It's going to ask me, okay, what is your reference level? You have to keep that in terms of per 100 flight hours. And then it's going to ask you, what is your alert level? Again, you have to keep that in the framework per 100 flight hours. And so everything then on your graph will be super sexy and look appropriate. And it'll look really good. And I love the work our dev team has done on our graphs, but getting away from the Baldwin tool, again, same principles apply if you're using Excel or some other spreadsheet tool in developing these. It's an important concept It's not critical, in terms of functionality. We can operate in the land of decimals. But with the caveats that I talked about earlier, as far as scaling and just even working with decimals, which are gross, in my opinion. But now we get to work with whole numbers, which is really nice. I really dig whole numbers, as opposed to decimals, I should say. Again, this idea of the multiplier is to make your life easier, to make the day at scale better, and just to make it, I guess I can't say it enough, nice all around. So try it out. Like I said, if whatever is in the denominator is in turn is, reaching into the hundreds, use a multiplier of a hundred. If it's getting higher, use a multiplier of 1, 000. Try it out. It's the same number. You're not going to hurt anything. But just remember, when you set your reference level and you set your alert levels, they have to be in the same terms. They have to be per 100 flight hours or per 1, 000 flight hours. So let's go over how you do this again. In a given month, you're going to take the number of events that you've had, which is your numerator. You're going to divide it by the total number of what you're measuring by, whether it be flight hours, cycles, but let's use flight hours in this case. So you're going to take the total number of events, divide it by the total number of flight hours. That's going to get you the number of events per flight hour. That's going to be the raw number of events per flight hour. And 9 times out of 10, it's going to be a decimal. Probably a small decimal, 02 or 4 or whatever it may be. Then... You're going to multiply it by your multiplier, whether it be 100 hours, 1, 000 hours, 10, 000 hours, whatever you want, 10 hours whatever's appropriate, so we can get rid of that decimal and have a whole number, but what You have to remember, is that changes the nomenclature of what you're measuring. So now, let's say we're multiplying by 100 hours, and for those of you that remember with multiplication of fractions and using units, you have to keep the units, so if we multiply by 100 hours, we have to say per 100 flight hours, or per 1, 000 flight hours. So we have to remember, not only for our data point, for our Y axis, because now our Y axis is going to be per 100, per 1, 000 flight hours. Our reference value, our green arc if you will, or even our target and objectives, is going to be in that same term, per 100 hours, per 1, 000 hours, whatever you choose. And then our alert levels... In terms of monitoring, same thing. It's gonna be in that same term per 100 hours, per 1, 000 hours, whatever it may be. That's the idea of the multiplier. Not as cool as reference levels and alert levels, but still important. Just to make your life easier. Make the graphs easier to read. Just remember to label it appropriately. If you guys want more information on this, give me a shout, drop me a line. When this feature comes out in Baldwin we'll let you all know so that's it. That's all I have. I hope this was informative. I hope it was something that you can use, look up the ICAO statistics, see what they use. I think they use per 10, 000 flight hours, if I remember correctly, but it's used all over the place. And again, it's to make things nicer, to make nice whole numbers. But if you like this podcast, let us know in the comments, let us know in the ratings. If you don't like it, let us know in the comments, let us know in the ratings. Just be nice. But we just want to be of value to you. We want to serve you, and I'm so thankful for what you guys are doing out there, keeping the system safe. I love learning from you, I love hearing from you, and when we go to the shows, I love seeing you. Come stop by the booth, come say hi, let me know what you think. But anything else you want to hear about, let me know, hope to see you in the next podcast, but until then, stay safe.