How ice incorporated kids in a Design Sprint
ice is a new and upcoming cellular provider that’s set forth to challenge the telecom-duopoly in the Norwegian market through higher efficiency and better quality-of-service by utilizing no-legacy technology, digital customer journeys and with an extreme focus on customers, that’s articulated in a goal of “becoming Norway’s most recommended company”
In Norway, Junior High-students will take part in work-experience projects for a full “work-week” during their senior year to gain experience and understanding of what actual work means (I assume). In ice, we quite often get requests for facilitating student’s “work-weeks”, and I, as the “app-guy”, am ever so often forwarded these requests; “if there is something they can test in the app… or anything else, really…”. The problem has, until now, been that I’ve gotten these requests just days ahead of students arriving, and what could have been an interesting and efficient week for both the student and ice, typically ends up with them playing in our social-room, clean the dish-washer, sit awkwardly looking round the room in a meeting they don’t understand, test the app for about 30 minutes and do other rudimentary tasks. This, I feel, is in itself a problem; is this the way we want to portray how it is to work for such a cool company as ice?
As these requests came in, I started thinking about “what if we could plan for proper user-testing focused on students?” or “what if we could do proper interviews to gain deeper insight about what youngsters need from our app?”. If only I could get a notice well ahead of time…
At the same time, I had realized that we’d been missing important age segments in previous Design Sprints and user tests:
In fact, the youngest to test our app (in a user test) prior to this Sprint, was 20 years old. Do teenagers actually like our app? What could be improved? Which problems can we help them solve? Are teenagers’ problems different than adults’? A lot of questions. I therefore had a clear goal of digging into these questions further, I just didn’t know exactly how — yet
Planets aligned
I aired my frustration around these challenges with some of my colleagues, and not many weeks later, a message on our message-board ticked in;
“My 14 y.o. brother-in-law is coming to ice for his ‘work-week’ in a month. Anyone has some useful tasks or ideas for what he can work on?”
Perfect. I jumped on it, checked my calendar, and to my surprise, it was almost empty for that week. Nothing that couldn’t be shifted around. Planets were aligned, and I replied back: “I’ll make a program for him for the whole week”.
I immediately started planning for user-testing, different employees who should interview him, and other tasks i thought could be insightful and interesting. As this planning progressed, the week started to more and more take shape of something… something more than just testing and interviews. It started looking like… a Sprint.
Now checking other relevant peoples’ calendars, I realized that it actually would be possible to do a full-week Design Sprint.
How cool wouldn’t that be? A Design Sprint with a Sprint-challenge focused at kids, with an expert-kid in the room — for the whole week?
I went back to my colleague whose 14 y.o brother-in-law was coming in just 4 weeks, whose also been participating in Design Sprints earlier, and asked him if he thought the kid was up for it. I angled my question several ways and repeated it. In Norway, Junior High students typically have 5-hour-ish school days with lots of different activities, both physical and theoretical. What we were talking about here was to bring a kid into one room for a whole week — one of the more intensive types of weeks we have, and expect him to be productive, give insights, learn things, and gain an impression of how cool ice is as a company. Could this be done in a good way, or would it be a dramatic fail?
Having participated in Jake Knapp’s one-day workshop on Design Sprint facilitation in Oslo last year, combined with having had several different roles in Design Sprints prior, I had acquired enough understanding of how and where to shorten the week. The potential Sprint team members I’d been looking at were all but one experienced in Design Sprints. That helps in shortening. So, I made a plan that didn’t require full days in the Sprint-room everyday. On Monday, before our student had said hi to people we started with a a user-test of him, a task I gave to the app’s PO to complete. Then after lunch, we started the Sprint at noon. Tuesday was also compressed, while Wednesday became a full day (though with some tweaks). Thursday was more intensive for other team members, and Friday we started first interview at 10 a.m.
For me, the major idea behind Design Sprints, is that we have an inescapable deadline: testers will come on Friday, and we have to have a prototype ready. I figured that, with the setting of having 1) a focus on youth and 2) a youth expert in the room, we should further the “inescapableness” by limiting the people to test on on Friday to… teenagers. Though,
most teenagers actually go to school on Fridays (!)
So, I started planning for how to be able to gather a group of five students distributed over a full Friday. And, realizing we had to gather consent form their parents, how to get a guardian’s signature into the flow. Full disclosure; the plan totally blew up (details later), but with part luck, part cleverness and some extra effort, we managed to gather a very good pool of testers in the end. Pewh!
Introducing Emil
I was mistaken; Emil is not 14 years old, he’s actually “almost 15”. Emil at first glance appears like any typical Norwegian teenager; slender, busty hair, curious, engaged, has a love-hate-relationship with school and can’t sit still. Emil likes to play games, he likes hanging out with friends, he is an avid phone user (which teenager isn’t?), and loves unscrewing things like old computers. At the end of his very first task, the interview mentioned above, my colleague and Emil ended up discussing signaling technology and advantages of packed-switching(!). Yet, Emil does not appear to be something that many Norwegians (teenagers and adults alike) are; he’s not shy and not afraid to speak up and share his thoughts and ideas. Now, that’s a very good start.
My plan prior to Monday, was to have Emil as an expert (like the once you typically would interview on Monday afternoon) in the room throughout the week, while ice-employees were the “real” members. Looking back, I can’t believe my prejudice and my need for this limitation. Emil very quickly became much more than an expert. Actually, he was always a Sprint team member. Luckily, it took me but a few minutes to realize my shortcomings, and the week progressed with Emil as a regular team member.
One thing I didn’t reveal to Emil though, was that the Sprint was arranged around him having a “work-week” at ice. This, I felt, could have been received as pressure, and I really didn’t want him to feel any stress over this arrangement.
Focus at the elbow
Last year when participating in Jake’s Design Sprint workshop, I was amazed about his timing (Jake brought no less than three Time-Timers). A one-day workshop where we’re set forth to rush through Monday thru mid-day Wednesday of a Design Sprint — with time for introduction, commentary, questions and “facilitators’ tips and tricks”. Jake spent about 10 minutes of this precious time discussing how to do high-fives.
“I’ve learned that when doing high-fives, the best result is achieved when both high-fivers are looking at (though, not aiming at) each other elbows” — Jake Knapp
“Now, go ahead and try it”. “That didn’t sound like real high-fives, come on, lean in a little bit and focus! Give it some thought”. “There’s a very low bar for high-fives today. We shall do them often, and not just only when I remind you”. “Think about when you should do high-fives. Remind each other”. Ok, I’m paraphrasing from memory here, but I was really impressed. It actually worked. I proudly “stole” Jake’s high-five technique and have incorporated it in many different settings. This Design Sprint was no exception. And again, it worked. Yeah, it’s awkward, but think about it; the whole Design Sprint-week is! Doing something little silly, little rewarding, little awkward like this, really can help the energy in the room. And Emil seemed to enjoy it… maybe after a few rounds. By the end of the week, we were all high-five masters, and we apparently needed mastery for Friday!
Reward, reward, reward
Since we already had locked down the age-range by limiting this in the Sprint challenge, I started one week before the Spint kicked off to recruit youngsters we could use as testers for Friday. As mentioned, most school-aged children apparently go to school on Fridays. This posted a problem; if we could only test after school, we’d probably start 2 pm, earliest. Instead, my plan was to engage with the very nearby High School where we expected that schedules are less compacted, and that there probably would be students that would be available in the morning, and maybe lunch break. In fact, each class’ week-plan is publicly available online, and we could confirm that it would be possible to either get pupils in the morning, or even in the one-hour lunch-break. But High School students only represents the higher end of the range we wanted to test on, so I also engaged with the closest Junior High School, about 1/2 mile down the street. In Norway, Craigslist is not at all used, and any of the similar services are not used by (junior) High School students. Targeted, digital marketing towards children is regulated in Norway, and we wanted to stay clear of any gray zones. So, I needed another approach. For each school, I created a poster looking about like this (translated, localized and cropped for the purpose of this blog-post) and blew it up to A3 (double size of Letter, whatever that is in non-standardized-U.S.-formats) and hanged them on info-boards on both schools:
Ok, so a couple points here. First, we chose the equivalent of US $30 because we felt pretty strongly that parents potentially could get upset over a too high reward (i.e $100) for one-hour work for such young people. $30 seems like a pretty decent reward. Second, by Tuesday afternoon, the posters had been up for three school-days, though not yielded any responses from the High School, and only three responses from the Junior High. Was the reward too low, or was there something else?
We therefore decided to take a break on Wednesday at exactly 11:40 am to walk a few hundred feet due north to intercept the High School students passing out of the school gates aimed at the local shopping center in their lunch break (being neighbors, we’d observed this swarm of students during lunch break often before). We brought A4 (letter)-sized printouts of the same poster and passed them out. The kids didn’t respond to that either, they thought we were selling them something. We changed gear and started crying out “Thirty dollars reward, join us for app-testing”
That helped. A lot. We easily got rid of our 80 copies of the poster this way. It took about 25 minutes. During the Sprint-lunch, I checked the responses, and suddenly we had eight. By mid-day Thursday it had grown to 14. Focus on the reward did help a lot, and $30 was plentiful to get students interested. It was just that they hadn’t gotten the message. Looking back, the poster should probably have looked more like this:
If I was to do this again, I would have done the exercise of standing outside the school gates, but doing it the week before the Sprint, both outside the High School, and the Junior High School, and hand out copies to students and their parents.
Who has a working printer at home these days?
Not many, I guess
Working with people under 18 (and specially when video-transmission is involved) requires consent from their parents. In my plan for recruiting teenagers, I’d prepared an info-email and a consent form for parents or guardians to print, fill out and sign. ice’s legal team felt that an email-only consent from parents would not be sufficient, so we went with the approach: Print out, sign and scan or take picture and return to this email address (being mine). All but one of the parents replied that they didn’t have a printer at home, so I asked if the kids could print out at school and bring the paper home for them to sign. This resolved most of the cases, but for now, let’s just say I did put on some miles on my car Thursday night to acquire signatures.
What?? Her name is … Paul?
As was already evident, my brilliant idea about starting early with recruiting kids for Friday’s test went poof. For many raesons. Another example came from the applicants’ online form to fill in. There were many boys and just a few girls responding. Wanting to have a balance between girls and boys, I tried to secure the few, relevant girls first. Pauline was just outside our main target range in terms of age, but was available in the moring. So, I sent out a text as soon as I had finished laying out the schedule. This was Wednesday evening, and I addressed her properly by her name, Pauline. No response that night. Maybe she’s an early sleeper? Repeated text Thursday morning. Well, she’s at school, right, so she’s probably not engaged with her phone..? Then Thursday afternoon, patience ran out and I called her. I don’t know who was more confused, me or him, but whomever I talked to was definitely not a Pauline. The boy in the other end sounded confused and his manly voice was cracking up in laughter. Further investigation revealed that Paul’s friends had played a prank and submitted the form unbeknownst to him, with most of his data correct, except for the gender and name. Duh, what was I thinking? Kids are kids, and this stuff was bound to happen. ice, being a telco provider, could easily have incorporated a text message verification mechanism or something to that effect, without too much effort. Note taken.
(actual names changed for privacy reasons)
A bit of luck
On Thursday afternoon after the team already had headed home to get a well-deserved rest, I was sitting by the coffee machine with my head in my hands. I didn’t want to let the Sprint-team know the actual status of tomorrow’s pool of user testers, and I was pondering on how to resolve this. In addition to losing the only girl in the pool, another boy had last-minute withdrawn his participation and we were left with only four testers. All boys. I felt this was a fail on my part and, through this screwup, was jeopardizing the whole Sprint. We need five testers the Book says. As I was scrambling through the responses from the posters and hand-outs and checking the available spots to see if I somehow could re-arrange something last minute, a colleague (non Sprint part-taker) from the “App-team” approached me and wandered how the user test recruiting was going. I was baffled at the relevance of the question. She explained that she had a daughter, aged 12, that she assumed would love to join testing. I was very grateful for her approaching me and told her that normally I’d prefer kids that are not children of employees to avoid bias, but since I was left with no girls at this point, I was very happy to take her. Her daughter couldn’t join in the morning hours, so I explained that she’ll have to come in last, which was the only after-school-available spot. She scratched her head and asked, “so, you need more girls? In the morning?” Yes. Please! She said she would check and get back to me. Not an hour later she had not just confirmed her own daughter, she had found a High School student, girl, aged 16, a not-too-close neighbor that was available at 10 am. Thank you! I immediately contacted the girl’s mother, and everything seemed fine, except she didn’t have a printer (!). The damn consent form again. My motivation was at this point extremely high, and (duh, I don’t have a printer at home either) I had to drive to office, print out the form, drive to the girl’s home and get the signature from her mother.
For those who are keeping tabs; yes, we did end up with six testers (one more than recommended). Given all the uncertainty and the effort to gather a representative pool, I’m quite happy with that, and Friday went along just fine — only a few breaks were shortened a bit.
At lunchtime Friday, as we were relieved about how things were going and laughing about all the recruitment screwups, a suggestion for a future improvement came up:
Print the damn consent form on the back of the flyer!
Yes. Noted.
The inescapable deadline
Apart from the Thursday deadline to complete a prototype, I had effectively limited the scope of the Sprint by clearly announcing that the testers on Friday will all be teenagers. In normal settings, this limitation is not a good idea, but given that we had secured Emil, and that the logistics of acquiring youth testers for Friday is much more complex and needs more planning than the Sprint allows for, I decided to do it anyway.
The week progressed pretty much to my (altered) schedule, and except for the chaos surrounding user testing recruitment, I as facilitator did not need to be too strict and remind people about the deadline. We were a team of mostly experienced Design Sprinters, after all.
I was probably most curious about how Emil would receive the instructions, and if he was willing to follow the methodology. Design Sprits are about learning, and learning is Emil’s primary job, so that came easy to him. Norwegian schools uses a lot of different methodology in the education, not just teachers lecturing, and adoption of new methodology is part of his school day.
High-fives all around
Friday 10 am, first tester. Anxiety and excitement in the room as the test was about to start. 10 minutes later, smacks of palms meeting each other were echoing in the room, and relieved laughter was heard. There were a lot high-fives that day. And, as post-it-notes flew up on the white-board, a team-member noted; “I’ve never seen so much green on the board [on a Sprint-Friday] before”. Agreed.
Apart from the fact that the prototype performed extremely well, the open-ended interview-section was time-and-time again super interesting, probably because kids are filter-less, unafraid and typically very open. And it was interesting to see where the different kids put their focus on the open-ended questions. For example, one kid really delved deep into ice’s Business Rules for the various products and how this prototype would affect them(!). Absolutely on the money. We got, what feels like (even though only six youngsters were interviewed), very deep insights into what matters for this age segment, insights ice can utilize in bettering the customer experience way outside of the scope of the prototype, the “long-term goal” and the Sprint questions.
By the time the last tester had finished, we’d given ourselves so many high-fives and had shared so much laughter that our palms were sore and we were mentally exhausted. Yet, seeing the results was, as always, extremely gratifying.
“We did this Sprint because you were coming”
Friday 5 pm. “Was this week different than what you expected it would be?”, I asked Emil. Yes, extremely different. Emil had never heard of, or seen anything like this, and thought it was very cool and a maybe little bit tiring. Check, check and … noted.
I’ll probably never forget Emil’s expression when I told him that the whole week was designed around him. By this time, he must have realized the amount of man-hours being put into this strange way of working. The surprise on his face was evident. Yet, probably as he thought about the results we just had summed up, a slim, almost insecure smile emerged.
So, Emil; Thanks a lot for your time, for being an integral part of the Sprint-team, and for helping ice’s younger customers to get a better user experience. We really couldn’t have done it without you. We’re gonna miss you here, big guy! I’m really looking forward to telling you about the working product.
