Podcast Ep 2: Collaboration for CPSx Study
Podcast Ep. 2: Collaboration with CEL for CPSx Research Study
Guests: Lew Montgomery, Pravin Chopade, and Dave Edwards
In this episode of the ACTNext Navigator Podcast, we discuss intra-ACT collaboration between ACTNext and the ACT Center for Equity in Learning (CEL) to produce a field study about measuring collaborative problem-solving.
Lew Montgomery is the CEL Director of Strategic Partnerships and Strategy. He arranged for 100 students to participate in the Collaborative Problem-Solving game, “Crisis in Space,” in May 2019. Montgomery lined up students from middle and high schools in Iowa City, Coralville, North Liberty and Cedar Rapids for researchers Pravin Chopade and Dave Edwards.
The next Navigator podcast, Episode 3, will examine the game further and go in-depth on some of the research that will come out of the study.
Learn more about the game study here and read the CEL blog about the collaboration here.
Podcast transcript:
Welcome back. This is Episode 2 of the ACTNext Navigator podcast. I’m Adam Burke. Today we’re talking about a video game that measures collaboration.
ACTNext researchers conducted a study with this game in the Iowa City area in May 2019. Two people play a game called “Crisis in Space” or CPSx and they have to communicate to solve some puzzles and save a spaceship. It’s a little like Apollo 13. We’re going in-depth about the game with the research team of Pravin Chopade and Dave Edwards in Navigator Episode 3, so stay tuned for that. Later we’ll do a follow-up when they’ve gone through some of the data and analyzed the result. So stay tuned for both of those episodes. This is an exciting project with real-world applications for measuring collaboration and social and emotional learning skills or SEL.
First, let’s find out about the collaboration within ACT to make this project happen. I spoke with Lew Montgomery. Lew works in the Center for Equity in Learning (CEL) at ACT.
My name is Lew Montgomery. I’m the director of community engagement for the Center for Equity in Learning.
I asked Lew about some of the challenges of arranging the field study in the Iowa City-area.
Montgomery: The ACTNext team was looking to obviously identify the project but they also needed individuals to participate in the project. They reached out to me and said hey Lew, is there an opportunity first for you to help us locate students? I was like okay 8–10 students and I approached the question I said well that shouldn’t be a problem. I know enough local organizations that can get those eight to ten students and they said no we need a hundred.
Houston, we have a problem.
Here’s ACTNext data scientist Dave Edwards.
Edwards: First things first is that Lew is fantastic and he’s really a go-getter and so my first goal was 50 (students) and in our first conversation with him, he said how many do you think you need? I said well maybe like 50 because last summer we did it with the interns and I think we had 33 with 25 interns and then we I guess we had to recruit nine ACT staff and so we I think we had 34 people total in that dataset and so I knew that I wanted to have a similar size but a little bigger and so I was like well let’s do 50 and Saad (Khan) chimes in, no let’s do a hundred.
Montgomery: My eyes lit up like a Christmas tree when they said a hundred folks but we were successful we were able to find the hundred students we reached out to individuals in and around the Iowa City community. We reached out to City High, West high school. We also reached out to Cedar Rapids into Jefferson, Taft School, Northeast Junior high school and really gave us an opportunity to really reach down into getting a very diverse population. So we have students who are academically very high level but we also have students who are around that three-point range as well. It was a little bit of arm-twisting because we approached it towards the end of the school year and as you can imagine at the end of the school year is constant chaos. You start getting into April and May time frame, you got graduation, proms and everything else going on. So really the timing of it was a bit challenging I had to call in a few favors here and there but we were successful you know and our partnership with ACTNext is one that we feel real proud about and for us it was a great opportunity for us to take two departments merge them together with common interest and that’s obviously getting the information and the data that’s needed. So we feel real good about that that’s something that we were positive about and it’s nothing like working with Dave as well as Pravin and the whole staff over there so we’re happy about it.
I spoke with Pravin Chopade by Skype. He’s connected to us today from Michigan where he’s working for the month. He’s usually one cubicle over from me in Iowa City.
Hey, I am Pravin Chopade. I am a research scientist to with artificial intelligence and machine learning team at ACTNext. I have special expertise in machine learning and I am working on this collaborative problem-solving project since the last two years. We use a visual approach to major collaborative problem-solving and social-emotional learning skill using AI and machine learning. We use CPSx, a space exploration video game called “Crisis in Space” that involves two-player jigsaw tasks composed of a series of puzzles in the game. Two players, either an astronaut or an engineer are connected by a video and audio Skype call in separate room participants alternate roles and must communicate and cooperate to deal with what dangers in outer space and complete the mission layers decode encrypted messages avoid asteroid impacts and put explorer satellites in orbit around Mars each game lasts about 40 minutes and breaks are included.
Following the space mission players will be asked to complete a survey about the experience specifically we focus on digital collaborative environments their behaviors are indicative of CPS skills and which can be captured in multiple modalities including video audio and eye tracking recordings and analyzed using machine learning techniques we are using computational framework for evidence extraction and accumulation and performing analysis on data obtained from pilot study specifically this work focuses on implementing various machine learning models for identifying teamwork skills during the play of a collaborative game, Crisis in Space.
Why was it so important to use actual students in junior high in high school and why did you need a hundred of them?
Chopade: Specifically in the summer when we did the first pilot for someone intern they were mostly the master students from different age group but if you see in the ACT we develop different educational and workforce solution for these high school and middle school students. So our objective was to teach this game or develop this game for the population or for the age group which ranged kids for the middle and high school so we just want to know if we have certain models if we make some changes and if you want to implement this for these middle school how will it’ll go. If we have say 20 that will not in a pan to came up with some of the research analysis or is a result. We need a hundred students in order to get more accurate information in terms of the different demographic different diverse group and that will be good population in terms of making some research spending or you want to scale this project to large population so these are all with this objective. We thought of having more students so that these specific models will be more accurate in terms of measurement as well as going for the validity of our research.
How many research papers do you think you can get out of this study?
Chopade: This is a very good question so basically we want to take this research to the educational community and present in good conferences that is our objective second is also want to publish this in a peer-reviewed journal so that it will go to the more scholar network and they will also provide their comments um recommendations as every research you all based on the input from the educational research community so we are considering two conference presentations publications and two journal publications in peer-reviewed high quality journals.
Now at ACT we call connected work it is also known as remote work a growing sector of the workforce is connected and connected work is directly related to this project. How do you collaborate by digital connection?
Edwards: Our team is spread out, like Saad (Khan) is in the East Coast time zone, Alejandra (Andrade) is in Mountain time. Bryan (Maddox) is in the UK you know and so like there’s definitely a lot of remote collaboration work there. Sometimes I think some of the tools could be better you know like if we have some document we need to share or whatever email is always being used but it’s not always great because like that different versions of everything so you know coming up with different methods of doing collaboration is really key I think but I mean there’s always the phone and you know phone is fantastic.
It almost blows my mind to think like what was things like without like before phone like before radio like I can’t even imagine it sometimes and so yeah so from that perspective you know like moving forward into like the future world of collaboration I mean I think it’s really tough it’s um you know from an economics perspective right like I gained a lot by not having to have a person on staff like on site you know like this gig economy kind of thing. So I hire you to do like 20 hours of work or something so I think that that is like really huge I think there’s always going to be issues of quality there. I think there’s always gonna be issues with you know just this first time if I start reading or whatever like are they really gonna deliver this thing they promise so I think that’s always something that’s a challenge and so you know being that being in direct contact with them is huge as far as like collaborating routinely with the same people. You said you had a Skype with Pravin so one thing I found really interesting is that ways that different people do that remote collaboration.
For me I’m very much a person where it’s like you know I know you I see you in the office all the time like I don’t need to turn on the video I don’t need to do video chat like I don’t need to see you but maybe you want to see me you know but maybe I don’t want to be seen because I’m like chomping on a granola bar or something and so there’s definitely a lot of things where sometimes people a little bit more distracted or different people have to collaborate in different ways so like some people like to just need to be emailed every day some people they want to do a call like almost every day or every other day some people really want the video some people don’t and so I think one of the big things there and this is I think something that you can learn from playing collaborative problem-solving game Crisis in Space is that really try to meet your partner where they are. Do what they need done and not necessarily what you think needs done especially when that person is your boss you know like you have to meet their requirements and not just only focus on yourself which I think is something that could be said about a lot going on now.
I think that we’re getting higher and higher abilities to be able to do video conferencing do audio conferencing share screens with each other like share documents back and forth with one another and so I think collaboration is just gonna get bigger and bigger and bigger all of the best ideas these days are cross-disciplinary ideas their places were like a biologist and a data scientist come together like a physicist and a data scientist come together like everybody comes together for the data scientist to make something and the one thing I’ll say about data scientists is and they don’t know anything about anything but they do know like how to fit a model to some data set. It’s just like I don’t know the processes of biology but I do know that if we have all this data we can find these correlations right and so much of it is cross-disciplinary. There’s no way that a person who knows this much about microbiology can also know this much about robotics or whatever so there’s so many of these cross-disciplinary teams where you have to figure out how to speak the other person’s language always is constant pressure.
Clarity of communication making sure that when I see something that’s the same thing as what you mean when you say something and that we develop a common language for order doing and these are all these kinds of things these processes that we are explicitly trying to elicit in our climate problem solving game and so this is where I think having it fuse together some kind of like mindfulness reflecting activities in a classroom is would be just an invaluable experience to understand that like even though I’m playing this game I’m still doing all the same things I would otherwise need to do in a non-game collaboration I can need to set clear expectations I need to make sure that you know what you’re trying to do. I need to make sure that you know what you’re trying to deliver. I need somebody in the group to take some kind of leadership role you know and so there’s just a lot of these things in the larger world of collaboration that I think are directly relevant to our game.
These three people, Lew, Dave and Pravin are a diverse group. I asked them about their backgrounds and working together.
Now tell me about you, Pravin. Where did you grow up? And when you were 10 years old did you want to be a computer scientist?
Chopade: Yeah this is a very interesting question Adam. So I came with a very small village with around 500 population in India. There is a state, Madrassa state and there is a small city I was born.
When I born that time we don’t have electricity for few years so for initial few years I studied with the lamps and later I made in terms of based on my educational aspirations and with different skills. I was good in science and math I made a plan go for higher education called engineering so I always try to do well in my studies with a lot of hard work and passion and I succeeded in terms of I got made assistantship.
I went from small town to another town for another stress schools and then for my engineering district government engineering college and then for masters to another City College of Engineering in Pune and then for achieving higher education I received opportunity to complete pursue my doctoral degree in North Carolina State University in Greensboro so in December 2013. I completed my PhD in computational science and engineering and this journey gives me Federation in terms of how what is the importance of education which helped me to succeed myself as well as to help others because the education means helping or me to work all different projects different assignments and which are part of the bird which are the solutions which we are developing are applicable to the broader community so I came with through ACT.
ACT makes with broader vision to apply my technical scientific skill and develop different solutions which will be applicable to the large community yeah my background was my father was a primary teacher. We were having limited financial resources when we grow up in a small home so that time only means even as I said earlier even at some area there electricity but due to some financial thing initially we did not have any electricity and later when I actually went for my master’s education undergraduate then I started getting the internet so till that time I was really not aware but I also made one thing like a when I grow up and when I go with the this knowledge I will give you back to the community so that I now they will not suffer with the same problem. I want to actually take this education and knowledge to my same community or community around the world those who are in the same those who are actually growing in the same environment so that they should not face the same problem and they should get all the opportunity equal opportunity and they everyone should grow with good education and that will help them to prosper in their own career as well as that will also help them to grow or for in community. That’s my also one of the vision as I grow.
And when was the last time you were home?
Chopade: Yeah, in December of 2018 my father is now 79 years and he was not well, so I made a plan last December. I went four weeks to meet them.
How do you explain what you do to your home and friends?
Chopade: I started in general in terms of how we how the educational environment previously like we used to have the everything on the board like chalk and the blackboard and we have to sit on the ground and things need to explain means now what our teacher used to explain some of the concept like if it is a science if there is a some math everything was on the board but we were not aware what is accurately required to do some of the scientific thing we have to just imagine what teacher was telling.
But now I am telling them we have developed this artificial intelligence and machine learning is helping to develop some of the prototypes so if we have to teach the same thing concept very easily in engineering or any scientific concept we can using the air machine learning we can develop a game or simulations we can at the same time use some of the internet a video tutorial very sophisticated tutorial to show them that this is the concept so I tell them in terms of like these are this how the education is changing we are developing some of these educational measurement to measure some of these skis like specifically math, science, writing.
These are some of the skills but at the same time we have another skill- collaborative problem-solving, creative thinking. So these skills are useful and how this will also help for individual as well as a team to develop. We are developing these tools to play the game and these two rules also help the students to measure their collaborative problem-solving and social-emotional learning skill.
I take the use of the concept from the computer science development in the technology how the world is changing to tell them that as a scientist how I play our role of making the visionary plan in terms of designing some of the concepts having a discussion with our talented team member and making the efforts to implement it and then there are various team members they try to productize it and then that products come from our company.
Lew, I was reading about you. Do you still do a football training camp?
Montgomery: My background is I was a student-athlete at the University of Iowa I played the 1990 Rose Bowl. I was a four-year starter in the as a running back I graduated from the University of Iowa in 1992 with a degree in communication studies. I played it very briefly in the NFL with the Detroit Lions and I have taken my athletic career and the things that I’ve learned athletically and tried to apply them to life skills hard work, dedication, commitment, preparation, utilizing the networks that I’ve had over the course of the 28–27 plus years of being in and around the community and really leveraging that into the work that I do here at ACT.
If you think about it holistically you got 22 players on the football field. Eleven on each side and the importance of being able to collaborate now just taking it from an offensive perspective okay. You know you have a quarterback, the center, offensive line, backs and receivers and the importance of being able to collaborate and communicate in a millisecond to make decisions that will put position you in the best opportunity to advance the football down the field and ultimately across the goal line to fourth score is a concept that that really seems relatively elementary. But in peeling back the onion, it’s very complex. Because you have a defense that also has a responsibility to stop the advancement of that football down the field and to also disrupt your ability to effectively communicate so that you can advance the football.
If you take those two elementary kind of thoughts and you apply them in a workforce environment what you get is collaboration among team members. Say our CEL group who is trying to advance the students opportunity to an equal playing field for folks who sometimes are disadvantaged and along the way you have these bumps in the road and you have this disruption where you’re having to try to advance a scenario or to help a student and you run into these roadblocks. So I think as we think about collaborative in an athletic sense and apply it to a practical sense in a workforce there’s a lot of similarities what I feel at least personally for me is how do you navigate the playing field. How do you apply the lessons learned from being a Big Ten athlete as well as an NFL pro athlete to the workforce and I’ve had to learn some difficult lessons along the way but there’s a lot of similarity.
That millisecond of decision when the quarterback stands up to the line who calls the play in the huddle and that quarterback comes up to the line then he changes that play at the line of scrimmage and you have a hundred thousand people screaming and you simply can’t communicate — how do you still execute that play in a millisecond and a decision-making scenario that still help you advance that football?
I think in a workplace sometimes that happens to where we have to make split decisions in a millisecond or a very timely decisions that quarterback sometimes will have to give a hand signal when he understands that his teammates aren’t communicating sometimes he’ll pat his face mask, sometimes he’ll pad is his thigh, sometimes he’ll give a hand signal. So that’s where it goes back to those nonverbal communication things skills that I talked about earlier is when we think about collaborative and we think about our remote employees and we think about the person that’s sitting next to our table.
One of the things that I value probably very highly in my life is being able to pick up and since a person’s effectiveness and grasping the information you know are they getting it are they’re not giving it there’s a lot of follow-up questions and a lot of things that I try to gauge from a person to make sure that they’re concrete on you know maybe my definition or my explanation or what I’m sharing. Sometimes the roadblocks that are in our way we have to navigate through how we communicate to still advance the football down the field as an organization to reach our strategic objectives and then also being able to filter out all the distractions and the noises that are sometimes around us and still advance the football. So that they’re there the parallels that I see just on a high level among those two areas that’s good one of the things important point.
Chopade: I want to also speak about our AI/ML machine learning ACTNext team starting from AI/ Machine Learning director, Saad Khan. He really played an important role of his leadership, executing and planning the specifics and decision to move forward. Our team member David Edwards, lead data scientist. He is also the instrumental part. He’ll always try suggestions or implementation. Then Alejandro (Andrade) who is helping us for the skill tagging and causal development with Dr. Brian Maddox our consultant from University of East Anglia (UK). He also is the director of MicroAnalytics at United Kingdom.
Both came, Alejandro and Brian Maddox, during the pilot study to help participate in our to school so their participation was important as well as the Scott Pu. He is also the data scientist Iowa City team also help as well as communication team. Adam Burke, he also made his participation in playing the game giving the suggestion as well as making the contact with the media and the press release and that also gives the wide coverage due to his connection and his expertise during the plan and Andrew Cantine from communication team.
Matt Livaudais who helps to design different designs for this CPSx game that also help us. They played also important role in success of this game as well as our project manager Tobi Drake who also helped to coordinate a number of activities from the opposing document to the legal and a lot of other paperwork Deborah Glass.
Most importantly to persons they supported senior director strategy implementation and operation, Ada Woo, also saw that everything we need is there and mostly for doing our visionary leadership.
Dr. Alina von Davier, Senior Vice President ACTNext, is really busy but she is always asking us how things are going is everything is okay? Do you need support? That was the encouragement and her support really helped us not only these but she’s with the leadership team. They are helping us to take this project to the scalable level.
At the same time, most importantly, our company’s leader, Marten Roorda, CEO. He also attended our CPSx presentation when we presented in the ACT research presentation. He also came up with some important suggestion and he is also the source of information for this project so overall starting each member of the ACTNext team and ACT, ACT Equity and this shows really is not the work of only I am doing. This whole success is a story of participation from all the members of the team of the as well as the organization as well as the support from the community outside. So this is a really very good environment which is required for the success of any project.
That’s a good way to end it and thank you for naming all those people, that’s great.
Well, that’s our show. In episode number three, our next podcast, we’ll go in depth with some of the concepts we touched on today specifically the game, Crisis in Space or CPSx, collaborative problem solving. Thank you for listening to ACTNext Navigator!
