Week 6 Live Session Recording

This week’s session continued our discussion of feedback from last week, specifically about ways to provide feedback that isn’t just text. We also discussed ways to share hints and additional information within the structure of your branching scenario.

The only thing in the chat transcript for this week was the link to the Life Saver Training, mentioned as an example of effective use of time as feedback.

Read the automated transcript.

11:07:47 Alright! So we are now in live session. Number 6.
11:07:58 And so this is talking about. So the goal for this week is to finish the full draft of your branching scenario.
11:08:10 It is now. This is going to be a draft you’re going to revise it later.
11:08:14 And knowing where some of you are already, in some cases you sort of have a draft already, and so this may be more revision.
11:08:23 But the work that you do this week is going to be focusing on.
11:08:29 If you’ve got sort of structure right you may. You may. A structure.
11:08:33 And you mapped everything out. You’ve written your at least your ideal path, and now it is.
11:08:38 Let’s go back to the some of those alternate paths.
11:08:41 And flesh, the rest of those out. So last week I did talk about feedback a little bit, and Eric asked a really good question, which was, Wait on a lot of this feedback that we’re doing, especially when people have made wrong Choices.
11:08:56 Isn’t that really? For? Not the ideal path? It’s really for the ultimate paths.
11:09:00 And yes, that’s correct. I really have planned to do more of that discussion on feedback today, but we did so many questions about feedback previously that I shifted that a little bit earlier.
11:09:12 But there are a couple of things in terms of feedback that I think as again as you’re looking at these as you’re looking at your alternate paths, and how you do things that you might also think about the visuals for free feedback, now, you may decide that you don’t do
11:09:34 any of this, that ultimately you’re going to mostly be text-based, and maybe you know one background image to set the context.
11:09:44 And you’re not going to do any of this, and that’s fine, but if you want to look at the possibilities and the ways that we do feedback, that is not just text.
11:09:53 Last week we talked about kind of coaching more this week is we will talk a little bit about that.
11:10:02 But also in the ways that we show consequences that are not just dialogue or coaching messages.
11:10:15 So a lot of the times in feedback messages we have something that looks like the coaching message on the left, and and it will see there’s places where a message like the one on the left is probably okay, right?
11:10:36 There are points where you do, in fact, want to tell people really explicitly.
11:10:41 That’s what it is as we talked about last time.
11:10:44 If it’s if they’ve made multiple wrong decisions in a row, and they’re you’re gonna make them restart the scenario.
11:10:52 You should give them something. Probably this direct. That’s instructional feedback.
11:10:58 But if it is in the middle of the scenario, and you’ve got more experts, you also have a couple of other options.
11:11:12 In this case it is. Instead of the coaching, the instructional feedback there is dialogue.
11:11:18 This is a scenario for training doctors. How to do brief counseling for alcohol use.
11:11:26 So this is a doctor. This is a simulated conversation with a patient.
11:11:32 And where you say something, and the patient has this expression.
11:11:38 As well as feedback, and has this, you know this response of the dialogue, and it is a way we talked about in the writing, where, if you can show rather than tell, it can be more engaging feedback like this, where you’re showing rather than telling you can also be really helpful Kathy Moore
11:12:06 is really big on if you do too much direct coaching, and you really like to tell people sorry that’s wrong.
11:12:15 That it feels a little patronizing.
11:12:19 It’s the you aren’t respecting the the intelligence of a adults.
11:12:28 Now looking at Ruth Park’s work and the research on this I, Kathy Borr, is kind of opposed to doing any real direct instruction with coaching, and I don’t think that the research actually supports it quite as far as she goes, so I looked for some something in the middle in some combination.
11:12:48 So this is a screenshot, actually, a really old screenshot from a game called forge of empires.
11:12:58 But if you think about in a video game, right? We do talk about gamification and stealing things for games.
11:13:04 And and we absolutely can steal this. If you think about a game, they’re really good at giving you feedback about in how you’re doing in the game without actually telling you what to do next.
11:13:18 So forge vampires is you have a city, you build up your city, you add, you add, these buildings, you take up our territories on a map, you have battles, there’s all of these things.
11:13:32 You are. There’s a lot of managing resources. You’re balancing multiple factors at the beginning.
11:13:39 There are some tutorials, there are quests essentially where you know.
11:13:43 Oh, you need to do this thing, and you know you need to build these houses.
11:13:47 You need to build off farm that produces goods.
11:13:51 So there are some directions there of. Here’s what to do next.
11:13:56 But ultimately, if you mess up, it’s not going to tell you so in this game I have, for example, a meter that shows me coins and a meter that shows me these are supplies and metals.
11:14:12 And so if I run out of coins and can’t buy stuff, there’s not gonna be a feedback message that says you’re not balancing your resources, and you need to do things that build that have more coins.
11:14:28 I’m gonna figure it out because I don’t have any coins, and I can’t build the stuff that I want to do, or buy the goods, or whatever it is, with the resources.
11:14:38 People are smart enough to figure this out. Very is help documentation available, and people are in guilds. And frankly, a lot of the instructions happens within a guild where you ask other people well, how do I do this?
11:14:52 How do I do this event, or what’s the best troops for for such and such?
11:14:56 But if I run on space for my buildings, that’s a consequence.
11:15:00 It’s not. Gonna tell me I’m just gonna have to like C.
11:15:05 Now branching scenarios that we build for training should probably almost never have as many feedback options as what this game has.
11:15:16 This game’s really complex. There’s ranks and build, you know, the all of these things.
11:15:27 There’s events there. This is my enthusiasm about people like there’s so many variables in this game.
11:15:32 This is, this is much more than I would recommend that you actually do it again.
11:15:39 Or in in a branching scenario.
11:15:41 But I do think that we can look at games and steal some pieces of this in how they show feedback.
11:15:52 Without beating you over the head with it.
11:16:03 So one of the obvious ones is, of course, to do character responses.
11:16:10 This is a pretty common one, and especially if you’re ultimately going to be building this in storyline or Ice Spring, or something where there’re characters in it.
11:16:21 Ice springs, talk faster, builds this in pretty nicely, where they’ve got the characters, and you you can tweak it.
11:16:28 Storyline has built in characters. This is one from what used to be E.
11:16:33 Learning brothers, and is now elb. There’s a range of expressions here right from pretty upset to pretty happy.
11:16:44 These sort of character responses is used a lot in games.
11:16:50 When you have a simulated conversation with somebody. This is this is pretty common that you haven’t.
11:17:01 So some of that’s the conversation responses. Yeah, you’re already drafting that right?
11:17:06 You’ve already got some of that in your dialogue.
11:17:08 Maybe it does make sense to show a visual to go with that.
11:17:11 Though it again reinforces it. You’re trying to give people enough information to make a better decision right?
11:17:20 So if it’s the mad expression that is a signal to people of oh, right?
11:17:27 I didn’t do the right thing, and in some scenarios in particular, reading body language is part of the skill.
11:17:35 So Natalie, as your dotting here. I do think that your scenario, where it’s that?
11:17:42 How do you talk to people and you where you want people to notice the signal of like, oh, they’ve gotten uncomfortable.
11:17:51 You. It probably is worth having some of these visuals for yours.
11:17:55 I was thinking for Eric, too. So because he has, you know he has to.
11:17:58 Hey!
11:18:00 The learner has to look at body language and the react, you know, kind of altered their interactions based on the reaction of the customer to yeah.
11:18:09 Yeah. Yeah, yup, yeah, I think, Eric, you already have that sort of bill.
11:18:19 You have that built into your text descriptions right now of like oh, he’s, you know.
11:18:24 He’s looking. He looks like this. And you can tell that there’s somebody who’s waiting, and this person looks uncomfortable.
11:18:30 So you’re already doing text descriptions of it.
11:18:31 And so.
11:18:33 Now you know, some of this is also frankly, what’s your what’s your budget?
11:18:40 And how much time do you have to build stuff? If budget is smaller and you don’t have, and you have less time to build, I think you can do it in text.
11:18:54 If you have the resources to do it, having some of this body, language can be good, and, in fact, I’ve done interactive video scenarios where, for example, it was the the people working at a reception desk.
11:19:09 For for Wic offices. And yeah, we that was all.
11:19:14 Because that’s similar kind of customer service actually very similar to yours, Natalie, but it’s still sort of the same thing of like you need to watch the people they want people to you know, watch as people are coming in and off to help.
11:19:25 If they’re caring a bunch of stuff. And so all of those things where the video of the setting in that case actually made a difference.
11:19:40 Sometimes there are things that we want to be changing in a scenario that are invisible.
11:19:52 That simulation for doctors, with the simulated conversation for doctors.
11:20:00 What we’re trying to actually change. There is patient motivation to change behavior.
11:20:06 Motivation is not visible. I can make the character expression visible, but motivation’s a little trickier.
11:20:13 So sometimes it’s worth creating some sort of meter of, you know.
11:20:19 Customer happiness or cost, you know. Again, ice, spring!
11:20:25 I know, Natalie, your potentially gonna be using ice cream.
11:20:29 So I will talk about that ice spring has a meter directly built into it, like they have like a smiley face, meter of like. How happy or frustrated is this person, so you can.
11:20:38 It’s sort of built in to do that. There are ways to do this there’s ways to do this in twine.
11:20:48 There’s ways to do this and anything else. This is an actual screenshot from that one that I did for training physicians, it may be, looks like it is you.
11:21:00 There’s ways to do these sorts of things with variables, and you haven’t score.
11:21:04 And you actually do back. That is not what this one is behind the scenes.
11:21:08 I created Graphics and Powerpoint, and I put them as static images and captivate, partly because I that this is an older example.
11:21:20 And this was built in. I think this was built in captivate.
11:21:24 I don’t know. 3, and so it’s.
11:21:28 It’s several versions ago, and I frankly wasn’t as good with variables and advanced actions at the time, and so creating some sort of meter, was not a not really in my skill set, but I did images, and I just kind of manually put it, on things that on the slide it
11:21:46 worked.
11:21:49 So, if you, if a meter like this makes sense for your scenario right now, I would say, just sort of describe it in your scenario, like, if you’ve looked at Eric’s scenario where he says this is gonna be a hint of a hint message, or feedback that’s visible only if
11:22:12 you click on it, do that kind of description. Don’t worry about any technical pieces of it.
11:22:17 That’s we’ll work on that later.
11:22:22 But it’s worth planning and thinking about those things. Now, before we before we start development, you want to plan these things.
11:22:34 Calendars. Sometimes they’re useful. Especially if we’re talking about a project management.
11:22:40 And you’re looking at, you know, managing project, scope and resources.
11:22:43 Have seen a calendar as the you know. It’s the project.
11:22:46 Completion date, and as you make choices, the project completion, date shifts.
11:22:53 Make sense right in terms of a trading resources, and maybe you maybe the answer is that sometimes you should.
11:23:00 Push. The completion. Date out 2 weeks, and in order to meet, because there’s a scope increase but it’s really worth doing.
11:23:08 And so you should just delay delivery by 2.
11:23:12 That might be the outcome in a project, but this is a way to show it.
11:23:21 If you were thinking about, if you had something like a price meter.
11:23:31 So price meters, what sort of skill or objective could be supported by something that should price going up or down, you put it in chat, or unmute, and talk.
11:23:51 I would say, business instinct or decision that leads to more income for the company, or for whoever.
11:24:01 Yeah, like sales decisions. You know, what are you gonna pitch to?
11:24:06 To the customer.
11:24:07 Yeah.
11:24:09 Yeah, sales, things we were talking briefly about.
11:24:16 Clark, Aldrich’s Short Sims. Earlier today.
11:24:19 Before we started the recording one of his single simulations does have, where he’s teaching some economics of things, and he does actually have the like.
11:24:32 Oh, well, if you you’ve got your beverage truck selling lemonade, and if you increase the price that your profits go up, or if you decrease the price and you get more customers, then you know.
11:24:47 So you know, there’s definitely things like that.
11:24:52 Eric. I don’t think it makes sense for your current customer service scenario to do something on price, but you do have a little mention at the end of you know, selling the add on thing, and there might be things for your audience.
11:25:08 We’re showing like average sales over the month would make sense cost of a project could be done for a project management scenario right we talked about the calendar, but if we’re talking about the length of a project, the cost is certainly one as well, I and I think that there was I had seen one
11:25:32 that was something on sales, but it was also trying to use the beater to show not just the short term sales, but over the long term.
11:25:45 What’s sort of the projected income 6 months from now, based on the decisions that you make.
11:25:52 Now you build trust, now, and you get them to buy something small right now.
11:25:55 So maybe you get a smaller right now, but they’re going to come back and buy more, and it’s hard to make visible when you’re talking about kind of fast forwarding in time.
11:26:13 So there are one that you might have, I’m thinking actually of.
11:26:24 So I did. One of those builders or safety training where we did, in fact, have part of what they needed to learn was what all of the different icons that were warnings on their dashboard might mean, and they needed to recognize those things.
11:26:38 So we had like. Oh, you see this, you know, this light goes on on the on the bulldozer dashboard. What does this one mean?
11:26:47 How do you respond? So it’s the difference between knowing what the check engine light is in your car versus low tire pressure.
11:26:56 Right, so it’s sort of that equivalent of that.
11:27:01 And so sometimes I actually am thinking that Emily’s scenario I know Emily’s not here, but Emily’s scenario frankly has some of this work.
11:27:13 The warnings of the clinical signs that the patient has actually fits this more than dialogue, and and I know she mentioned in her post that it’s not so much dialogue.
11:27:26 So Emily, I’m hoping you’re going to listen to this recording later and catch this.
11:27:30 So it’s it’s not dialogue, dialogue isn’t the thing that makes sense.
11:27:33 What makes sense is the signs that the patient has.
11:27:41 Right, yeah.
11:27:37 And maybe even looking at the monitors. The visual of a monitor, and they have to figure that out themselves rather than her telling.
11:27:46 The blood pressure went up to blah blah blah blah blah!
11:27:49 Correct.
11:27:51 You know that that they have to cause that’s how they have to do it in their job.
11:27:54 Correct Yup, and so some of it is you do have to think about.
11:27:54 Nobody is. Gonna tell them, yeah.
11:27:58 Is the skill that you want. The nurses to learn to practice in the scenario is the important thing.
11:28:07 The fact that is, is it the fact that they recognize that if blood pressure is going up that that’s the thing.
11:28:16 Or is the part of the skill that they need to learn is that they’re going to look at the monitors that has 10 different things on it, and pick the important piece of information and I don’t know where the skill level is for Emily’s audience right now, right for those nurses I
11:28:33 would suspect. It’s more the latter. Where it’s actually is.
11:28:37 You need to look at the Monitor, at the whole Monitor and pick up which piece of information is important.
11:28:43 So it might be. That’s one we’re showing a visual of the Monitor.
11:28:49 Probably does replicate the environment in a useful way for practicing decision, making.
11:28:58 I did. I did one that was similar of training instructors on troubleshooting in an Lms.
11:29:08 Where we I had a narrator who was thinking about on the process of like errors of Well, why doesn’t the grade book add up to the total that it’s supposed to?
11:29:20 And so a lot of this was simulated of, okay, you’re looking.
11:29:24 You’re actually looking at the grade book in the Lms.
11:29:26 And looking at what’s there to do? Your troubleshooting rather than focusing on the face of the instructor, which was not the useful piece of information.
11:29:43 And I apologize. Sorry I should warn you. This is a weird picture.
11:29:47 How many of you have done the life saver training from the Uk.
11:29:52 Are you familiar with this one at all? Alright, Nanna’s done it.
11:29:55 Terry’s done it Erica is no okay.
11:29:57 I’ll.
11:29:57 Only 3 day driving license. Years ago.
11:30:00 Yeah, so it’s, this is not something that I created her own, but it has.
11:30:08 It’s a situation where you end up having to give Cpr.
11:30:12 The chest compressions to keep someone a alive. And so one of the things in this as an online training is that you have to click your mouse in the right timing for Cpr.
11:30:28 And you can see that there is a good versus bad meter here for the timing, and they have this like, how do we respond?
11:30:40 And your average sign, your average speed here.
11:30:45 So I will say, I actually do think that this meter would have been more effective if it had been too fast at the top and too slow, and that you were actually aiming for in the middle would have been more useful feedback, because just knowing it’s good or bad doesn’t actually tell you to speed
11:31:06 up or slow down.
11:31:09 This is amazing training, though. Let me see if I can find it real quick and put it in chat I do think that it’s worth checking out.
11:31:24 If you have not gone through this as an interactive video there’s not a whole.
11:31:30 You know the branching it’s been a while since I’ve gone through it.
11:31:34 The branching structure real wide. I do. I think if I it’s been a few years since I’ve looked at, but I think it’s mostly a gauntlet structure.
11:31:46 You get out, you see consequences, and then you get right back on the path.
11:31:49 Because if you’re recording videos when you look at the video production on this, this was an expensive thing to make, they’re not making videos out for multiple paths down.
11:32:00 So time can be effective in this sort of safety training. We’re literally seconds matter.
11:32:12 I think that timing is useful.
11:32:15 But most of the time we should not probably use time as feedback.
11:32:22 So this. Oh, this is the. This shows that it’s at the very top.
11:32:30 This showed how fast I was answering questions, and again, because it is emergency situations.
11:32:37 And you want everybody responding really fast.
11:32:43 It made sense. In this case.
11:32:50 Most of the time. I don’t think the time should be one of the consequences that you use, and I do have people ask about.
11:32:58 Well, how do we set that up? You can set up a timer and twine.
11:33:01 You can set one up and storyline. So what do you think is the potential drawbacks for using time? As a consequence?
11:33:10 It’s not accessible for everybody.
11:33:13 Yup accessibility gets to be is one of the big challenges, for time is that if if it doesn’t, is that people do need sometimes a little bit more time to respond.
11:33:30 And in real life. Often they would have a little bit more time to respond.
11:33:35 And it’s okay. So therefore we probably shouldn’t assess on it.
11:33:40 What else?
11:33:44 Yeah, you have also to notice the exercise you have to fulfill that to read what I want.
11:33:54 If I say, Okay, I, for example, the phone is ringing.
11:34:02 Great Right.
11:34:01 Combine it with time, you say, after 3 rings you’re done, but you have to read also what’s happening.
11:34:08 Probably so you lose time that you can normally can use for the decision-taking.
11:34:14 Yeah, yeah.
11:34:18 So so, you know, some reading time, and people’s reading speeds are hard to predict, and in many jobs you could have a pretty wide range of reading speed and do the job quite successfully.
11:34:34 What about you, Terryus? Are you unmuted? Do you have another idea?
11:34:37 It’s kind of the same thing. I mean it.
11:34:46 Yeah.
11:34:40 It kind of just makes you nervous, I mean, for some people, and you’re not gonna know who is nervous and who’s not so you’re not really measuring all the time what you want.
11:34:51 There isn’t a deloitte example that that the time has no consequence, and I don’t really know what the point of that is, but nothing bad happens, and I didn’t even notice that until somebody told me that there was no consequence for the time running out but I
11:35:08 Yeah.
11:35:09 can’t remember what the reason for that was. But yeah, cause even that lifesaver one kind of made me nervous, and that was a matter of life or death, which is that very very relevant.
11:35:23 Yeah.
11:35:20 But it’s still made me nervous, so I sometimes I was just thinking, oh, I need to hurry up and do this, and not what’s the right answer.
11:35:29 No, but again, that’s how real life would be, and do it again.
11:35:34 Do it faster!
11:35:35 I do think that in the lifesaver training, making people feel anxious is a deliberate design choice that they made, and I think it works.
11:35:46 But I think you raise a really good point of is that the emotional response we want people to have to our scenario, and most of the time the answer is, no most of the time.
11:35:58 That’s not it. Sometimes having time on, there can actually decrease motivation.
11:36:03 There is some research on that that, having timers on things can decrease motivation instead of, you know.
11:36:10 Sometimes it’s the the thought is a stakeholder will come and ask about this it’s like, Oh, I want to add time, because it’s going to motivate people.
11:36:17 It’s going to feel like a game. They’re going to want to go through it faster but it doesn’t give you enough time for critical thinking doesn’t give you time to really read and reflect on it, and especially if you’re learning a new skill.
11:36:29 You don’t need to be fast yet. It can decrease motivation, and it increases. How much people just do random guessing.
11:36:38 At some point. Maybe you do need fluency, and maybe you do need to have something where you’re testing, but not the first practice may.
11:36:50 If you are really do need people to be fluent and to do something fast.
11:36:55 Ideally. What you have is some practice initially, that build skills.
11:36:59 That is not timed, and then eventually, after they’ve had multiple practices, then you start doing things that are sped up.
11:37:09 There’s a program called Dream Box that does math that my daughter’s school uses.
11:37:15 So she has been working on her multiplication and her division fax for a long time, and we’ve spent, you know we are 2 years into learning, multiplication, and division, and initially, none of it is timed.
11:37:28 But they do want to increase fluency, that that eventually you need to do.
11:37:33 You need to know multiplication fast. And so now she is in this point where they’re working on so.
11:37:42 And do being able to do it in your head, and not just write it out.
11:37:47 But that’s only after having done literally hundreds of practice problems before they get to the point where they are actually doing it at speed.
11:37:58 And yes, there are times when you should do it at speed.
11:38:02 Language, learning at some point you have to recognize those words faster.
11:38:06 If you’re going to have a conversation at speed. I am very slow in reading German.
11:38:13 I can do it.
11:38:17 But not fast, and with the help of a dictionary, right? Like, I’m not gonna get there.
11:38:24 I am not fascinating and eventually, if I wanted to have really have conversations, I would need, I need to practice more and get myself to speed.
11:38:32 So. Yes, sometimes time makes sense. But most of the time, if a stakeholder asks you for time as a feedback, you should really think carefully about whether that’s through whether that’s make sense and whether you have given people enough practice to justify using time.
11:38:53 So. Simulations. I think this is another area. Where you can have all sorts of environmental and physical changes.
11:39:02 I am thinking also about, you know. So this could be.
11:39:06 I’ve seen like color change and things I’ve seen sort of physical space things like ladder falls or safety training.
11:39:16 Safety things. Were you some of the one of the safety training things we have has a little electrical short noise in it.
11:39:26 You can show the picture of a fire going out, or a fire spreading, because you didn’t actually take care of it correctly.
11:39:37 Simulations can all you know, all of those can be be useful. I think.
11:39:46 Actually again! Emily’s scenario kind of falls this way, because it’s gonna be the changes with the patient and the changes of the monitor and the environment kind of fall into this again, some of Clark Aldrich’s examples have shown sort of environmental changes.
11:40:05 And then you figure it out from the environmental change rather than being directly told.
11:40:10 That wasn’t the right choice.
11:40:17 So anything else of effective consequences that you’ve seen in in any scenarios anything that I didn’t mention here.
11:40:38 Alright!
11:40:43 So there also have been a we’ve had a couple of discussions and some forum, and some things about how do you do hints or more information?
11:40:55 And there’s a question here of pusher, Poll.
11:41:02 A lot of times our default in e-learning is to push information out to the learners that we decide when it happens that we make it.
11:41:15 We make the decision about when they get each piece of information, and we push it to them.
11:41:21 But in branching scenarios you have a lot of opportunities to have the learner.
11:41:27 All the information themselves.
11:41:31 So compliance training. For example, if you’ve got something with a policy, the traditional way to do training is to tell people the policy and to push that information to them so that then they receive it.
11:41:46 But in a scenario we can put them in a situation that says, You know, you’re a manager, and you’ve got an employee asking for a reasonable asking for an accommodation for a disability.
11:41:58 What do you do? And maybe one of the choices is to look at the policy.
11:42:04 Now you’ve put them in a situation where it makes sense for them to pull that information themselves.
11:42:14 And so policy things maybe it’s the sales information on different products.
11:42:23 Right again, instead of just simply listing the products. Maybe it is you put them in a situation, the customer says, well, I need this, and this.
11:42:32 What do you recommend? Model A or model? B, and if I don’t know that then now I have a really good reason to go look at it, and I have a way to immediately apply that information.
11:42:49 And I think you already I think Natalie and Eric, you both have already kind of started building that into your scenarios.
11:42:59 And I think that’s that can be really helpful to learners.
11:43:03 You can give a lot of information to learners. You can give a lot of information and do a lot of teaching through that.
11:43:08 But it is not quite so much. The here are all the things you need to know.
11:43:15 Have a good day. Right? So you let them control the pace of when they get information, and it also is good.
11:43:23 If you have some of the things where you’ve got variability, in how much your audience knows.
11:43:30 So for Natalie, for your, for your situation.
11:43:36 Where we were talking about how to handle things like definitions of mixing or dead name, or you know, some of those things.
11:43:45 Some of your audience likely already knows that has gone through Dei training and has some background for some of your audiences may be brand new, and none of these things may be familiar, and so giving them some potentially, some ways built into the scenario.
11:44:01 So they can pull it and it may be that you have some choices of like, you know it is the, you know, after trace of the like.
11:44:14 Wait? Why is preferred? Name a problem as a choice, and then go explain why saying preferred name is a problem, or you know, you know, tell me more about that.
11:44:31 You can have those choices be still kind of in that same conversational tone.
11:44:37 And give them an option of like. Tell me more about that, or can you show me an example of this? I said.
11:44:46 Natalie, Natalie sent an email asking about some of this, of How do we? How do we handle this?
11:44:52 And I hit some screenshots from one of my scenarios that I done where there was a lot of stuff where we were.
11:44:59 For example, looking at additional information on how to make things more accessible, some of the audience already knows how to do that, and so I did not want to push it, and everybody sit through that.
11:45:12 But I had several places where I had opportunities for people to say I don’t know how to do that.
11:45:18 Tell me more about that, and can you show me an example, or what other resources are there?
11:45:25 And then I linked to the other resources. And so it’s within the context of the scenario and so it doesn’t feel quite so outside of the scenario, and that I don’t push it to everybody, not everybody will see that information.
11:45:37 But some of them will, I think, hints are also really good way to do that right.
11:45:43 Some people might need a hint. Some people might not. Eric?
11:45:49 From your experience, where do you place those additional info or resources?
11:45:56 Do you introduce them at the beginning of the scenario?
11:45:58 And then, it’s a tab or a resource button that is always on the same place.
11:46:04 Or is it more visible to? Not to people click on it?
11:46:10 Or is it more? A little bit more in in the background?
11:46:16 It depends on what kind of resource it is.
11:46:21 I have had somebody in the last cohort had one where they would be using a form or using some sort of job aid, as they would actually be doing the task on the job.
11:46:36 In that case, clearly made sense to put the job a pretty near the beginning of the scenario and have it available all the way throughout where they can always reference back to so some of those sorts of things are really useful to just kind of have in a general like resources here and then just you know, have it, in
11:46:55 a light box slide, or something.
11:46:58 If it’s hints specific to a particular thing, embedding it within the scenario, I would say by default, is probably, to put it right at the point that people need the information.
11:47:13 There is some research on how people value information based on whether they think they’ll use it now, or whether they think they’ll use it some day, and that is actually not just rewards or not.
11:47:31 Just information rewards in general. It’s the.
11:47:37 If I say I’m gonna give you a hundred dollars today or a hundred one dollars a year from now, which one are you gonna take?
11:47:56 And most people will pick. Well, let me ask what what are you gonna pick? Are you gonna pick $100 today?
11:48:03 Or $101 a year from now.
11:48:06 A hundred a day.
11:48:08 100 today, right? And that is true that most people are going to pick but $101 is more.
11:48:15 But it’s more valuable because you have it right now, and people treat information the same way.
11:48:23 It’s called hyperbolic. Discounting in the research.
11:48:27 If you really want to go, look at this. Julie Dirkson talked about it in couple of her things where she’s done some research translation on this topic.
11:48:36 But one of the things is that motivation for people increases, and the what, how they perceive the value of training increases.
11:48:44 If you give them information and it helps them make a decision right away.
11:48:49 Which is, of course, what we’re doing in branching scenarios.
11:48:53 Yeah, question?
11:48:54 I have a question in terms of pulling them out of the scenario like you mentioned, for example, like I do this with my competencies, where I link them to the actual policy manual to the section that’s very specific.
11:49:12 About that competency so that it’s always up to date right?
11:49:14 Right.
11:49:15 So if I’m thinking about doing this in a scenario and they’re gonna get pulled out of the scenario to like another shared point site right?
11:49:27 Because I wanna make sure they’re always referencing the most up to date information.
11:49:32 Does that impact like, how? What do I have to be careful of in terms of making sure they get back to where they left off?
11:49:40 Or do you have any suggestions? There?
11:49:45 If you can do it embedded in an iframe on the slide, I will say that that can help with that cause.
11:49:57 Then you sort of eliminate that problem. Right?
11:50:00 Yeah.
11:49:58 Or have it in a light box, or something, otherwise you have to give people pretty explicit instructions that say, you’re gonna have this open.
11:50:07 This is gonna open up in another tab. Have this open when you’re done looking at it.
11:50:14 Switch back to this tab where you’re training is, and it will also say that if you’re doing that in a learning management system, you need to test it in that learning management system, because sometimes the switching tabs breaks things or people lose their progress and you will have to test it in that actual
11:50:31 environment. So if you can simulate a part of it, or embedded on the page, or do something like that so that they don’t have to actually go out to the really resource that can be better.
11:50:51 But what do you think, Eric?
11:50:55 The resources online, not only right, that’s the rate their policy, it just needs a little bit.
11:51:05 It’s got an idea. What if the resource is in the physical world like the policy you have to pick it up from an office?
11:51:13 But it’s only there. It’s in the topic version that could be also funny to send people to places just to know where the things are really located in the real world.
11:51:25 Just an idea.
11:51:30 I will say that there are places where it would be useful, and actually there was an old.
11:51:37 There was an old scenario from Michael Allen’s group, from it’s probably 15 year. I don’t. It’s a flash base thing, and I know it’s not available anymore.
11:51:46 But it was where you had questions of. You could go ask Hr.
11:51:52 To do it, and they literally had, like a map of cubicles.
11:51:55 And your little character would go, walk to the cubicle for Hr.
11:51:59 And walk to the to the office for Hr. To go ask the question, because in real life what they would have to do would be to go walk over to Hr.
11:52:08 And ask the question of like, What’s the advice that you would give me here?
11:52:12 And so when you were, you were having to talk to different people on your team.
11:52:17 And literally, it was a cubicle map, and that was, or an office map, and that was the navigation was physically going to those sites.
11:52:26 I will say we kind of talked about that for one of my for some safety training of do we?
11:52:34 Should we have something where it is like looking at the site and figuring out?
11:52:37 Say, where the where the key asks are that have the safety information sheets where the eye washes, where the Ppe.
11:52:47 It’s, you know where the safety equipment is.
11:52:50 Where are the fire? Exttinguishers and we had talked about doing even a like a 3 60 video tour.
11:52:55 And going, and looking at those so you could go find those things Budget felt that it didn’t end up happening.
11:53:05 But there, really, there are some really good examples of that.
11:53:12 So, yeah, so other questions about feedback in all of its various forms.
11:53:19 So this is really so today’s was really like just helping expand the horizon of how does feedback work?
11:53:27 And these sort of hints and more information at how do you how do you incorporate those?
11:53:33 So any questions about that, or any questions about expanding out the rest of your draft for your scenario today.
11:53:45 Okay.
11:53:44 I have one, you stated. The environmental changes, and I’m wondering if I if it’s possible in trying that you have a kind of random generator that just that you can randomly choose.
11:54:01 What is the environment? For example, I, in my scenario, the customer asks for a credit.
11:54:22 Yes.
11:54:11 That does this always. Is there a possibility in trying to say one time he’s asking for a credit, and the next time he isn’t, can you program that?
11:54:25 Or is it easy to to do?
11:54:27 Yes. So you, so, okay, so you’re talking about the point in which.
11:54:34 So you’ve got a spot, and your scenario where you have.
11:54:39 Where you’ve done some of the answer you’ve up sold on the additional call service.
11:54:47 You’ve shut off the international calls and your cause. You’re going all of those other things first before trying to refund the money.
11:54:55 So that you right, that’s the goal, right?
11:54:59 Okay, and that.
11:55:03 70% of the time. Perhaps he would like that. He will then say this.
11:55:10 He, he will say, but what about the money? And 30% of the time he’ll just continue on, anyway?
11:55:18 Yes, you can do random number generation.
11:55:23 So if you wanna do that, describe it? For now, I would say, put it in one passage, for now I think it’s.
11:55:38 Put it in one passage, but kind of like draw a divider line, or something of like, and decide what you want.
11:55:43 Your ratio of this of if it’s, you know, 50 50 odds have in games.
11:55:50 There’s a lot of those sorts of things where you have a variable.
11:55:51 Hmm!
11:55:55 Yeah.
11:55:52 And so if you return, it’s not exactly the same having that sort of thing does increase the how replayable something is by having some randomization in it.
11:56:05 Let me look at yeah, go ahead. Good.
11:56:04 I guess that mixing also more. Excuse me, that makes it also more interesting for experts.
11:56:11 Whoever who are doing the scenario multiple times.
11:56:15 So if you go on the conditions of the fever, or they reaction or medication, sometimes it’s very less a little, and sometimes it’s huge.
11:56:23 Yup!
11:56:25 So you can have different point of view if you run the scenario multiple times right?
11:56:30 Okay. Let me. Drafted up with your 2 different responses in your passage.
11:56:39 And let me work on a sample for you, for twine.
11:56:44 Cause. Yeah, I know it is absolutely possible in Harlow I don’t know the code off the top of my head without testing it.
11:56:51 Hmm, okay.
11:56:53 Thanks.
11:56:54 But I will, I will do that if and I will also say if there’s anything else that anybody’s on the oh, it’d be really cool if I could do this in twine.
11:57:05 Think about those questions and ask me, because one of our lists I think it’s gonna be less than 8.
11:57:14 Probably. Where we’re gonna look at some of these advanced things like random number generators and scores and meters and things.
11:57:22 And I. I have a bunch of little things that I can show with that.
11:57:29 But if there is something else that you want, I’ll make a demo just for that last cohort.
11:57:36 Hmm!
11:57:37 Somebody asked me how to change the color of something based on a tag.
11:57:44 Right? So we changed the background based on tags. That’s used actually a lot where you, for example, have different locations.
11:57:54 So you have the office, you know you have the back office, and you have the front floor where the retail is, or you have the reception desk and the patient room.
11:58:06 And so you tag those as different things, and you can change the background based on that or in her case, she was using it for feedback.
11:58:16 She was doing passages specific for feedback.
11:58:21 And so we we formatted it that way. So that kind of thing let me know.
11:58:28 So I have time to work on it, make it the sample so I can show you how I will absolutely do that.
11:58:31 Excellent!
11:58:33 I don’t have. I don’t have a tutorial on how to do that, but I will make one for you.
11:58:37 Thanks.
11:58:38 Yeah, what are we that we get?
11:58:39 Is? Is there any benefit or not of a delayed hint or information like not putting it on there right away?
11:58:50 But wait a little bit.
11:58:51 So Will Tellheimer has reviewed some, had reviewed some research on feedback, timing, so I’m relying kind of on his stuff, and I think I referencing it in there.
11:59:05 But if it’s not in the course email, me, or put it in the Forum, and I’ll I’ll find the link.
11:59:11 The citations there is definitely.
11:59:12 I’m in the like more stuff the like extra stuff.
11:59:18 Yeah.
11:59:27 That’s true.
11:59:16 Oh, for the for like the more info, if it’s the stuff where it’s the more info, I would say, just giving people the choice of it is probably enough control, like letting learners control it.
11:59:34 Yeah. Yup.
11:59:31 This is certainly an undergraduate. Let adults control the pace of things and let them control the pace, delaying, delaying the instructional feedback.
11:59:43 There is research supporting that piece of it. I don’t have research that says if somebody asks for a hint, give it right away, I will say I’m definitely relying on that’s that’s one that’s relying on my experience.
11:59:56 Rather than a research, but I think that you can.
12:00:05 I would say, that’s enough to just let the learners control the pace.
12:00:11 And not delay.
12:00:17 Right.
12:00:09 Yeah, that makes sense, and it probably they’ll probably notice that if they see it right away versus having it appear while they’re trying to figure something out.
12:00:23 And is that?
12:00:20 Right? Yeah. And and in general for usability, right in general, for usability.
12:00:31 Hmm!
12:00:27 If you click something, people want a react right away. Now, there is sometimes a reason to do.
12:00:36 Let people have one failed attempts, and not show the hint until the second time.
12:00:43 They’re attempting it right. Maybe let them fail once on their own, and then show the hint after they failed 2 or 3 times.
12:00:51 So the first time they tried on their own that I’ve seen that structure quite a bit.
12:00:56 I don’t have research specifically on it, but that feels like that can be an effective strategy.
12:01:07 And make sense that the first attempt is a retrieval practice on your own, and but if you have failed that you need more information to make a better decision.
12:01:16 So, therefore you do need a hand. The logic makes sense to me.
12:01:29 Alright! So next week, Tuesday, we’re looking at a full draft, full language, which I think some of you already sort of have, but you might go back in.
12:01:41 And now, knowing that we’ve got, you know, we’ve talked about ideas for feedback.
12:01:44 Maybe you’re going to go in and think about feedback or descriptions of feedback a little bit.
12:01:48 Other ways. You can always go in and kind of tweak things, and I realize some of you do, in fact, have a complete draft at this point, cause once you get out of roll, get it just goes.
12:02:00 I’m trying to break the writing down so that it’s too much for people.
12:02:05 I know that the time required can be hard when you’re working full time.
12:02:10 So next week again live session next week we’re going to be talking about tools and look and feel because you do.
12:02:19 You are, gonna have a decision to make for next week. Which is, are you gonna stay in twine and keep working in twine?
12:02:27 Or are you going to just use this twine prototype as your starting point as your storyboard effectively, and then rebuild it in storylineer or ice cream, or some other tool?
12:02:36 So we’re gonna talk about the pros and cons of that.
12:02:41 And so then the next thing will be sort of looking at your look and feel. So we’ll look at ways to do the the visuals in twine.
12:02:55 And to do some of the look and feel pieces of things, how to add images so if you wanted to add images or meters, or some of these other things that we were talking about today.
12:03:07 How would you do that in twine?
12:03:10 Yeah.
12:03:10 Okay.
12:03:15 Thank you.
12:03:19 Thank you.
12:03:19 Thank you.
12:03:13 Very good. Thank you all. Appreciate your being here, and and, Eric, if if you didn’t grab that link in the chat for the life saver training, I do recommend you.
12:03:28 Go check that one out. It is a neat, it is a really well built scenario.
12:03:36 Okay.
12:03:33 I made a copy of the link, so got it. Thank you.
12:03:40 Alright, very good. Thank you all very much for being here today and for actively participating and asking lots of good questions.
12:03:49 I am looking forward to seeing the progress in your scenarios in the next week.
12:03:55 So thank you very much. Alright!
12:03:59 Bye!