Analysis Questions for Learning Design

I definitely favor successive approximation over the waterfall method; by prototyping early and often, you not only get a better sense of what will actually be most effective early in the process, when you’re best able to make changes, but sometimes end up refining the goals for the project itself.

That being said, I think analysis is super useful, and I spend a fair amount of time on it early in most projects. The analysis questions I ask vary. However, typically my high level goals are to:

  • Learn what success looks like for this particular project.
  • Dig into the key differences between that ideal situation and the current state of things.
  • Learn about the target audience, their motivations, and their work environment.
  • Explore existing tools and resources, and identify the gaps between what these resources already provide and what is needed for success.
  • Begin to dig into behavior: what should people be doing, why aren’t they doing it, and what are they doing instead?

Below are some of the generic the analysis questions that I have in front of me when interviewing key stakeholders towards the beginning of a project. When I’m actually interviewing, I often jump around, adding and modifying questions as needed.

Target Audience

  • What is their background?
  • Where do they work?
  • What is their work environment like?
  • What are the biggest challenges on the job?
  • What motivates them?
  • What do they worry about?
  • What do they like most about their jobs?
  • What do they like least about their jobs?

Project Goals

  • In the future, a year or two years after we launch this course, how will you know it’s been a success?
  • What do you want people to do differently after taking this course?
  • Why aren’t they already doing this?
  • What are the most common knowledge-related barriers?
  • What are some of the non-knowledge related barriers?
  • What are the changes someone could make to have the most impact?

Skills and Behaviors

  • What would be the best sorts of practices (real life or otherwise) to prepare learners for what you want them to do differently?
  • Are there people already doing this well? If so, how did they teach themselves?
  • What are the best practices?
  • What are the common misconceptions?
  • What are the most common mistakes?
  • What are the most tempting mistakes?
  • What is most challenging?
  • What is the target audience already doing very well?

Resources

  • What resources already exist to help the target audience?
  • How frequently are these resources used?
  • Could these resources be made more useful? How?
  • What information does the target audience really need to memorize, and what information do they just need to know how to find?

The above questions are aimed at project stakeholders. I also try to interview the target audience early on in a project, either through focus groups or one-on-one interviews; those questions are similar, but typically focus on skills, behaviors, and motivations.

What do YOU think is most important to ask at the beginning of a project?

Learning-by-Doing: GuitarBots

Yesterday, I wrote about how most “learning-by-doing” is really is actually “learning-by-doing-something-really-similar-to-what-you-ultimately-want-to-do.” This is the a post in a series where I explore games and online learning that incorporate authentic practice.

GuitarBots is an online game where you learn to play the guitar by playing the guitar. Unlike similar systems, you can use any guitar, including an acoustic one, since GuitarBots uses your computer microphone to detect which note you’re playing.

After watching my husband (Brian) play for months, I decided to sign up for my own account yesterday. Until I signed up for this account, I’d never played a guitar, so please excuse any accidental abuse of guitar terminology in this post.

start_practice

If this were traditional online learning, it would have started with an introductory module on the history of the guitar and perhaps some information about how guitars are constructed. Luckily, this wasn’t traditional online learning.

If this were more performance-focused online learning, I would have interacted with a web-based interface to tune a virtual guitar, by clicking the knobs to turn them and then clicking the strings to play the notes. This probably would have helped somewhat had I then immediately practiced this using an actual guitar. However, I certainly wouldn’t have had the visceral sense of knowing just how much to turn a knob, or how to juggle picking and knob adjusting without dropping the guitar. (I’ve never claimed to be particularly coordinated.)

Instead, in GuitarBots, my first lesson was learning to tune my guitar by tuning my real guitar.

tuning

GuitarBots stepped me through the basic tuning process of turning knobs and picking strings. I adjusted until the onscreen tuner indicated that I’d matched each note on my guitar.

Once I’d finished tuning the guitar, I immediately moved into learning to play a song by playing a song. (A very simple song.)

playing

After each song, I received a report on my timing and accuracy, as well as my history playing this particular song.

feedback

As I played, I unlocked additional songs.

levels

In addition to songs, GuitarBots guides you to practice related techniques.

techniques

I’m told that it’s possible to test out of early levels by demonstrating your competence on a song that very gradually increases in difficulty. With my limited guitar-playing skills, I didn’t bother. However, this seems like a great way to make a system that accommodates learners at a variety of competence levels.

At the moment, I’m not actively looking to learn to play the guitar. However, I’ve watched Brian’s path since buying a guitar two years ago. Since the purchase, Brian tried various books and practice exercises, but GuitarBots was the first time that he practiced consistently.

Some of the motivating and helpful features Brian mentioned:

  • Self-Directed Learning: GuitarBots lets him decide whether to push himself with new songs, or perfect a song he’s already learned.
  • Competition: GuitarBots shows how Brian is doing in comparison to his friends. 
  • Scaffolding: GuitarBots provides guidance for new techniques and feedback on areas where he is having difficulty. Additionally, if he’s having difficulty with a song, he can slow it down and then build it back up again until he is able to play at full speed.

These are all great for encouraging practice – and techniques that are often utilized in games and learning. In the end, though, what makes this game so strong in my eyes is that it creates an experience where learners are engaging in real world practice, versus something similar. Creating these types of authentic experiences – instead of “near authentic” experiences – is HARD, and understandably not yet done much on the computer. Here a key was utilizing the microphone as an input device, instead of depending on the usual mouse clicks, finger taps, key presses, or toy guitars.

The next post also features a game with a novel input device, this time a biofeedback sensor. More tomorrow!

This part of a multi-part post about games that require you to do exactly what you’re learning to do. Previous posts:

 

Games that require you to do exactly what you’re learning to do

Most “learning-by-doing” is actually “learning-by-doing-something-really-similar-to-what-you-ultimately-want-to-do.” This isn’t horrible, but it is different.

For example, I’ve designed and built many online modules that focus on interacting with people. Typically, the user clicks what they want to do/say and sees the results through immediate and/or delayed feedback.

Screenshot from loan officer simulation.

If the choices take into account best practices and tempting mistakes, these types of online interactions can help users practice some of the same sorts of decision-making used in real life. This is almost always more impactful than just passively reading or listening to information about these decisions.

However, the way the user acts and is evaluated certainly isn’t the same as real life. Not once in my interactions with real people in the real world have I been presented with three options for what to do next and asked to click one.

Receiving and judging user’s performance in novel ways is hard, especially with the limited options for input devices on computers and other electronics. On that note, my next few posts will feature games that I think are interesting BECAUSE of the way that they incorporate real (not just close to real) practice into the gameplay itself.

On that note, tomorrow’s post will explore GuitarBots!

It’s about the Experience

I was once again reading Jesse Schell’s The Art of Game Design and rediscovered this quote on page 10:

Ultimately, a game designer does not care about games. Games are merely a means to an end. On their own, games are just artifacts. When people play games, they have an experience. Without this experience, the game is worthless.

You can mentally replace “game” with “course” and I think this still holds true. The course itself, as some sort of product, is not at all the point.

Encouraging Thoughtful Choices

I visited the Monterey Bay Aquarium a few weekends ago. There were many things I enjoyed about the aquarium, including that many of the exhibits look like art installations.

photo (2)

Learning-wise, one of my favorite parts was how they wove the impact of their visitors’ choices into the exhibits.

At one exhibit, I placed an order for a seafood dish at a pretend restaurant. The chef, hostess, and waiter (on video screens) conversed with each other about the impact of my choice on the environment and instilled me with an appropriate amount of guilt based on my decision. (Want to upset the chef? Order shark fin soup.)

These same sort of experiences were woven into other exhibits throughout the aquarium as well. See this seahorse? What sort of shrimp do you think HE wants you to order?

seahorse

Apparently, seahorses often get caught in shrimp nets, making net caught shrimp the worst choice for our seahorse. However, at other exhibits I learned that farmed shrimp, especially imported, can be hard on the environment because they pollute and destroy local habitats. The best choice for the seahorse and the environment according the the seahorse and the aquarium? Box trapped shrimp.

A visitor might remember to avoid net trapped shrimp and shark fin soup, but how does one know the best way to wade through the many other seafood options? The aquarium offers a wallet card and phone app with guidelines on which seafood is least impactful on the environment.

As a learning geek, I really loved this blend of offering choices, showing real-world impact, and giving longterm performance support (in this case, ordering support.) And the fish were pretty cool too.

photo (3)