Our Next Steps

Post Project Reflections

Review Session Critiques

Throughout our class, our team had a couple of opportunities to share our project at different points to groups of people with stake in the portfolio space. We’ll review how we addressed the feedback that we received from these sessions.

Deliverable 1 Presentation

About halfway through our project, we had completed two user studies related to portfolios. At this point, we presented the insights we developed and the vision statement we had created out of that information.

Feedback 1: You should present more background information to tie back to your analysis with evidence.

In our final report, we’ve included evidence from literature reviews and user studies to back up all the insights that we’ve made.

Feedback 2: Specify the learning outcomes and learning practices your team intends to scaffold.

We decided to focus on developing feedback and reflection learning practices. Our final product scaffolded these two through introduction and exit interviews and an informal digital critique. Once students finished projects, they would be able to look back on their project and clearly see what they learned based on the articulation in their answers to interview questions. Additionally, they can observe the progress from start to finish of the projects based on the moments that they’ve collected.

Feedback 3: How will your solution encourage and support a growth mindset?

Because of our focus on feedback and reflection, students can clearly see how much they’ve improved from start to finish of a project. Based on our research, we know that critiques help students understand key principles in a specified domain and recognize how others look at their work. This helps students grow from novice to expert in each focused domains.

Deliverable 2 Poster Session

Briefly after we started visioning our products, we created a User Journey Map to show the journeys that we envisioned our audience to go through in our project. We presented these journeys with a pitch to stakeholders. Based on our pitch and user journey, our audiences were able to give some feedback that incorporated our ideas.

Feedback 1: Use your persona description to characterize the challenges or needs youth face while documenting their creative projects.

We’ve revised our persona description to show that Megan struggles with understanding key concepts in her current science project. Additionally, she wants feedback from her teachers and peers to help her understand how well she’s doing.

Feedback 2: Who will be creating the interview questions in your system? How customizable are the questions?

After talking to the stakeholders at this poster session and also interviewing teachers during our prototyping sessions, we understood that teachers wanted to be involved. Especially for younger students, they wanted to provide guidance. Because of the feedback we got, we decided to allow templates for domain specific projects which teachers could customize to add introduction and exit interview questions. Additionally, in the critiquing section, teachers would be able to specify points of interest that they wanted students to consider.

Feedback 3: What are the benefits in using your system versus any other system that is already available?

Our system provides a holistic view of students’ projects. Students have one place to go back to reflect on previous projects by viewing their past interviews and critiques. It also provides a safe space for students to shape their understanding of domain project subjects. Our system acts as a mentor to help students develop a documentation habit that they can take and apply to other projects in the future.

Feedback 4: What existing practices can you build upon?

In our prototyping session, teachers brought up the benefits that critiques have on students. To gain the benefits of critique, students need to learn how to provide constructive feedback to personal and peer projects. Our platform is a great place for students to practice articulating and justifying their thoughts.

Maker Ed Open Portfolio Workshop

In the middle of prototyping, we had the opportunity to share our ideas at the Maker Ed Open Portfolio Workshop. Many participants were veterans in teaching portfolio skills to students so were able to provide a lot of constructive feedback.

Feedback 1: What do teachers see from their end? How can they help their students?

Because of this feedback, we decided that we should incorporate a teacher workflow to emphasize what teachers could provide. Because teachers have specific goals that they want students to reach for every project and certain reflections and learnings they wanted to scaffold, we felt that it was appropriate to allow teachers to customize each project students worked on in their classes.

Feedback 2: Post project reflection? Aggregate projects?

Some workshop participants brought up that for their own reflections, they make a lot of interesting findings by looking at a set of projects at once. We believe that this is an interesting concept that can be explored in the future, but currently due to time constraints we decided not to include that feature in our platform.

Feedback 3: Younger students may be able to use this platform as well, they would just need some additional guidance.

It was great to hear that teachers felt that younger middle school students would be able to take advantage of our platform as well. Younger students would be able to build a strong documenting habit earlier on and become more independence as a learner. To structure it better for middle school students, we needed to add additional features like more guiding questions and examples and more motivational goals and reminders. For the time being, to focus on developing the learning practices, we decided to ultimately hold off on focusing on the younger audiences.

Deliverable 3 Presentation

Feedback 1: Your product lacks a clear focus.

We have adjusted our problem and vision statements to create a clearer focus. We are going to narrow down our scope to encouraging a growth mindset within reflection and feedback for students.

Feedback 2: How does your tool achieve a new role for portfolio making and documentation??

The role of portfolios for students now are currently a little more than a collection of projects throughout the years. Our tool positions portfolio making and documentation as a tool for developing important learning processes. This includes developing articulation in meaningful reflection and critiques.

Feedback 3: How can your tool be used to evaluate learning?

Teachers can look at the progress of each student’s projects to make sure they are managing a healthy time management habit. Additionally, they can evaluate how their students are articulating their thoughts in interview questions and peer critiques. This is a skill that is really important, but harder to manage and evaluate without a tool to access these written thoughts.

Feedback 4: Have you considered ranking first, then critiquing to force students to justify their rankings through critique?

We had not thought of this but we think this is a great idea. We had read about ranking in critiquing papers to help students understand where their projects stands in relation to other projects but hadn’t thought about the justification. We’re applying this change to our final design.

What Went Well?

View portfolio making through a unique perspective.

Through our research, we gathered the insight that many evaluators agreed that portfolio making helps students grow in their learning. We decided to look at portfolio making through this perspective of growth. We wanted focus on how students could develop a growth mindset through building a documentation habit which builds student reflection and feedback skills. Looking at portfolio making through this lens helped us create a unique tool for students.

Each study we conducted helped us narrow our focus.

From our expert user session to our final prototyping session, all of our sessions gave us more information to make changes for our final product. We listened to many of the student and teacher user perspectives to create a product that addresses their needs.

Received well by audience.

At our final presentation, audience members liked and wanted us to expand on our ideas. Many of the audience members reaffirmed the findings from our research studies. For example, during one of our prototyping sessions, a teacher brought up the value of critiquing which led us to our informal critique feature in our app. Audience members loved this idea and thought that it was a great addition to the portfolio making experience.

What Needs More Work?

Due to time constraints and resources, there are a couple of areas that we would have liked to explore more but couldn’t. We’re going to talk through the specific parts of these areas that are interesting to us.

Competitive analysis of current portfolio or documentation tools.

This is something that would have been useful early on in our research. We did gather some information about current tools like Schoology and Blackboard through our user interviews, but it would have also been valuable to go more in depth. We could evaluated some lesser used features that could have been related to our research focus points.

Delve deeper into teacher’s role and involvement.

We realized how vital teacher’s roles could be to scaffold the important domain-specific reflections a little too late in the process. It would help further our understandings if we could go back and do a user study on teachers to understand their motivations and goals through current tools and what they hope to be able to achieve in the future.

How we can effectively add incentives.

We did a little bit of user testing around goals but felt that the way we approached it wasn’t the right way. The feedback that we got was that students already make personal goals. We need to take some more time to reframe goal making in a way that motivates and reminds students to use our tool.

Connect back to portfolios

Currently, our tool focuses on the capturing and analysis aspect of portfolio making. Students can certainly use our tool to aid them in creating a cohesive portfolio piece, but it doesn’t directly lead students to make that extra step. With more time, we can figure out this last piece of the process to bring our tool into each part of the portfolio making loop.

Consider expanding to non-multimedia focused project subjects.

Because our prototyping sessions were all done at the Pittsburgh Center of Arts, our prototypes ended up geared toward that group of students. Because of this we were able to take advantage of some of the aspects of their projects that could benefit from multimedia fueled critiques. However there are many other project subjects that can’t really use multimedia for critiques so we would like to explore how we can incorporate those subjects into our tool.