Reconceptualizing Museum Educator Evaluations: Interrogating White Supremacy Characteristics in Teaching Observations

Quána Madison, Senior Educator, and Community Engagement Specialist at Clyfford Museum, and Emily Bullard, Assistant Director of School and Teacher Programs at the Clyfford Still Museum

Introduction

As leaders in museum spaces, we have a social responsibility to identify and address the systemic ways that institutional structures advance and protect white supremacy. At the Clyfford Still Museum, we re-imagined our museum educator evaluations through this lens. We lead the museum educator team in different ways: Emily supervises the team of fourteen paid educators, while Quána, a senior educator and community engagement specialist, models and coaches educators about teaching strategies, healing-centered approaches, and inclusive practices. Both of us support educator professional development and training. We regularly observe museum educators to evaluate their practices and guide their professional development. Our goals in re-imagining our observation process were to make the entire process more collaborative, to make the tool reflective of the complex nature of our work, and to break down characteristics of white supremacy embedded in our previous approach.

Before describing how we reimagined our observation approach, it is important to define our program and how our previous processes worked. In the past, educators received one formal evaluation each year. A supervisor would observe an educator teaching in the galleries and would make qualitative judgments based on their ideas of best teaching practices. Observers completed a straightforward form that asked for observations (implied to be “good” qualities) and questions (meant to be areas of improvement). The educators had a yes/no checklist of duties to complete (examples included: “Did I have a predetermined route through the galleries? Did I keep my group together? Did I confer with students as they worked?”) and a form with eleven qualitative questions about classroom strategies used, questions asked, and how students responded to the art. Educators completed this form immediately after their lesson and then meet with the supervisor who performed the observation. Although these meetings were framed as conversations, the tone often felt like the educator sharing their thoughts and perspective and then the supervisor responding with the “right” way. While the process asked for educators’ appraisals of their teaching, supervisors’ comments and observations shaped the debrief.

Our work to reimagine our process was born out of collaborative meetings with our team of educators in August 2022. The prospect of evaluation always made educators nervous, and our current team was outspoken about their perceptions of and past experiences with educator evaluation. Many educators who had previously taught in classrooms shared negative experiences with observation. Some reported feeling like the process conflated their worth with their work performance and focused only on ways to improve, including little to no effort to show appreciation for their successes. We responded to their stories with empathy and curiosity by critically reflecting on our museum educator evaluations.

While this conversation developed, we also did culture-building activities with our educators to kick off the school year. During these team-building conversations, which occurred throughout the fall of 2022, educators wondered if they could brainstorm norms of collaboration that would apply to their work with each other and students. We invited educators to share their thoughts about what made for successful and unsuccessful working environments. Here, conversations merged with our observation discussion as educators began to wonder how we would measure their use of our norms and evaluate how they could successfully collaborate with each other and with students.

As we began to extract elements of their feedback to include in our new observations, we also turned our attention to how our previous observation system upheld white supremacy culture. White supremacy centers on the idea that white people, their customs, culture, and beliefs are superior to people of other racial backgrounds (National Education Association). As described by Tema Okun, institutions value white supremacy characteristics and “to some extent require them and constantly reproduce them” (para. 1, 2022). White supremacy culture affects us all, though not in the same ways, and white supremacy “operates in collaboration with other oppressions; they reinforce and reproduce each other” (para. 8, 2022). We contemplated ways to undermine the white supremacy characteristics of perfectionism/only one right way of thinking, sense of urgency, and power-hoarding. We decided to reconceptualize our observation criteria, restructure the process, and introduce opportunities for sharing power with our museum educators to dismantle these characteristics. We piloted our new educator observation tool in January 2023.

In sections to follow in this essay, we will discuss:

• the complex history of educator evaluation in museums
• how our processes were informed by classroom practices
• how we reimagined our educator observations to dismantle characteristics of white supremacy culture
• what our new observation tool is, and how it works
• our vision moving forward
• how others can apply these strategies to their work

By guiding you through the history of educator evaluation and our reimagining of what an educator observation tool can be, we hope to answer the question posed in this issue: How can art museum educators promote practices that undermine myopic ways of seeing and develop inclusive strategies to thrive within white supremacist institutional structures? We hope to address this issue by utilizing novel evaluation methods that experiment with dismantling white supremacy characteristics within the context of their own spaces.

inStill Gallery Experience at the Clyfford Still Museum facilitated by Kaitlyn Tucek and Missy Brown.The Clyfford Still Museum is home to the life’s work of abstract expressionist artist Clyfford Still. Rather than the portrayal of a lone genius, the Museum seeks to broaden the definition of a single-artist museum by uncovering Still’s place in communities and how his work thrived with the support of others. Still’s vision allows for visitors to have their own ideas about the artwork, and the Museum seeks to facilitate personal discovery, reflection, and interpretation through collaboration with our communities. Photo Credit: Fireside Production.

The Complex History of Educator Evaluations

Museum educator evaluation research is limited compared to classroom evaluation literature. Why evaluate in the first place? Often we must account to an authority: evaluation for a grant, evaluation as part of a strategic plan, or evaluation to guide annual performance reviews that determine wage increases. If these authorities didn’t exist, would we still evaluate? Excellence in Practice: Museum Education Principles and Standards presents accountability as one of the key functions of museum educators (American Alliance of Museums Standing Professional Committee on Education, 2005). We are accountable to our communities. We measure our effectiveness using evaluation. During multiple conversations with educators, we recognized that accountability to each other also drives evaluation. Our educators described a culture that practices vulnerability and responds to our mental, emotional, physical, and spiritual well-being. Successful evaluation can be an opportunity to open up, make mistakes, and be vulnerable with each other, fostering growth for our entire team.

Using different titles for educators complicates finding research about educator evaluations. Several peer-reviewed research journals in the fields of education, art education, STEM education, and museum studies analyze person-led experiences in museum spaces (Spect & Loreit, 2021). Our review of available literature found that most sources evaluated tour performance. Few sources describe the process of evaluating educators themselves. Other literature focused on volunteer docent evaluation. Some museum professionals have described docent evaluation as taboo due to concerns about volunteer docents quitting or feeling hurt when evaluated (Powell Flanders, 1979). A few educators in art and history museums proposed rubrics and core competencies as starting points for evaluation (Darr, 2002–03; Schep et al. et al., 2017). In some museum spaces, peer observation constitutes one tool for educator evaluations (Littleton, 1994; Harding et al., 2000). We began our process by making lists of educator skills and competencies and comparing them to our educators’ lists of positive working environments and collaboration tools. As we reflected critically, we wondered how these characteristics could serve as more than a rubric or checklist. We also considered including our educators’ perspectives in the observation process.

Classroom Observations informed our Processes.

In the past, we modeled our museum educator observations on classroom teacher observations. Our inStill Gallery Experiences program incorporates many classroom teaching practices, and we often describe the experience as “turning the museum into a classroom.” While we unearthed issues with the structure of our observation process, we also learned how classroom teacher observations can be problematic.

Structured classroom observations in the United States began in the 1950s (Brophy & Good, 1986). There needs to be a consensus about the definition of quality teaching practice, how teachers demonstrate quality teaching, or how observers recognize the features of quality teaching (Polikoff, 2015). Some folks suggest responsive practices that support diverse cultural backgrounds as connected to teacher quality (Ladson-Billings, 1995; Paris & Alim, 2014). Other folks define it by the accuracy of instructing content or by the ability to foster a positive climate in the classroom and promote respectful relationships with students (Hill, Ball & Schilling, 2008; Pianta & Hamre, 2009). Teacher observation tools and processes vary significantly based on the organizational culture of an individual school, school district evaluation practices, state-level education policies, and federal education reforms. Many school districts have created individual evaluations to measure and assess instructional quality (Cohen & Goldhaber, 2016). Some examples of teacher observation tools include over 116 criteria to evaluate during a single lesson (Schmoker, 2012).

Today, classroom observations are the most common form of teacher evaluation. They are used to make decisions about compensation, tenure, and dismissal (Campbell & Ronfeldt, 2018). Administrators use teacher evaluation instruments in school spaces to conduct instructional observations of teachers. Many different frameworks are used for teacher observations (Schmoker, 2012). Teacher observation instruments typically use criterion-based measures. Classroom observations that assess perceivable teaching practices can provide actionable feedback for educators who desire to grow and improve (Cohen & Goldhaber, 2016). In our past observations, we used similar criterion-based measures without making explicit connections to our institutional and departmental values.

In many classroom observations, a single authority figure critiques and evaluates the teacher without collecting their perspectives. This hierarchy also existed in our observation process. We chose to disrupt this traditional hierarchy by interrogating how our observation process perpetuated white supremacy culture and reflecting on potential antidotes.

Emily Townsend, CSM educator, facilitates an artwork conversation in the Clyfford Still Museum galleries. Photo Credit: Fireside Production

Identifying and Dismantling White Supremacy Characteristics in our Observation Process

We looked critically at our observation process to understand how it reinforces white supremacist cultural values. White supremacy is internalized and operates as the standard to which all other non-white groups are compared. Characteristics of white supremacy culture may show up as attitudes, behaviors, and beliefs expressed structurally, systemically, institutionally, and interpersonally. Organizations may consciously or unconsciously use these characteristics as norms, thus limiting the space for other cultural norms and standards (Centre for Community Organizations, 2023). Education systems and museums have a long history of perpetuating white supremacy. In our former observation process, we identified the following white supremacist characteristics: perfectionism, only one right way of thinking, sense of urgency, and power-hoarding. Next, we explored antidotes to these characteristics.

Perfectionist institutions use guilt, fear, and shame as motivators for performance. There is little appreciation expressed for the work folks are doing. There needs to be more effort put into reflecting on lessons learned that could improve practice (Centre for Community Organizations, 2023). We revised the goals of our observations to include: celebrating the strengths of educators, practicing strategies, being a safe space to recognize and learn from mistakes, learning ways educators want to grow, and how they want us to support them with their growth. We contemplated our unconscious decision to model museum educator observations on classroom teacher observations, acknowledging that it is one way but not “the right way” or “only way.”

We slowed down the whole process to address the sense of urgency that was baked into our previous evaluations. We began conducting observations earlier in the year to allow us time to meet with each educator meaningfully. We created a pre-brief component, allowing educators time to reflect and confer with us about their well-being, lesson plans, and professional development goals. We added much more time to the reflection process, allowing educators up to a week to reflect and complete the debrief form. In the debriefing meeting, we focused on a few strategies we observed rather than expecting educators to address every indicator, emphasizing quality over quantity. We repeatedly asked educators to provide feedback about our evolving observation process and tool.

To address power hoarding, we brainstormed ways to share power with educators throughout the process, being transparent about valuing and utilizing their input. When our educators suggested changes to our norms of collaboration, we welcomed their ideas instead of feeling threatened by them. During our observations, we now ask educators to identify their areas of strength and seek their input about what strategies to look for. We follow the educators’ lead about where they want to grow, allowing them to be the experts on their own needs. Furthermore, we seek ongoing input on our tool, process and have made some edits based on educator feedback. As we continue to use the new tool and hear about educators’ experiences, we adjust as necessary.

Our New Observation Tool & How It Works:

Image 1: Museum Educator Observation Tool by Emily Bullard & Quána Madison/Clyfford Still Museum. This shows the various strategies educators might use to facilitate belonging and collaboration through teaching moves.

Museum Educator Observation Tool by Emily Bullard & Quána Madison/Clyfford Still Museum. This shows the educator pre-brief tool, where educators identify how their particular practice intersects with the dimensions of facilitation.

After our initial collaborative conversations with educators, we developed lists of collaboration skills, teaching strategies, and qualities of positive work environments. These lists felt like the beginnings of a rubric, but we intentionally avoided that formatting. We organized our lists of skills into three categories: belonging, collaboration, and teaching moves. We organized the tool into concentric circles based on how the skills work within each other. For educators’ teaching moves to be effective, they need to have established a sense of collaboration. Educators must establish a sense of belonging for the visit to be collaborative. Implementing teaching moves without setting the larger atmospheres of belonging and collaboration may result in learning but could feel cold, didactic, and boring.

We also knew we did not want to repeat the top-down system of an observer using only their discretion to appraise educators’ teaching practices. Throughout multiple meetings, we sought educator feedback as we redesigned the observation tool and process. Concepts like “vulnerability,” “transparency,” “power sharing,” and “responding to human needs” came directly from our collaborative conversations. In another meeting, educators requested a glossary and explanation of what terms could look like in practice, recognizing that each educator has their approach. We defined each term and provided multiple examples of how the skill might look rather than suggesting a single right way. We also asked educators what the terms mean to them and what they look like in practice. In another session, educators asked for explicit connections between our indicators and departmental values, which we included in another revision.

Our previous system also needed more authentic opportunities for educators to provide input into their observations. There was no pre-observation conversation, and the post-lesson discussion took place immediately after the lesson, giving educators little time to reflect meaningfully. We sought to eliminate these issues by adding a pre-observation conversation within a week of the observation and then allowing multiple days for educators to reflect on their practices. Our pre-visit conversation form asks educators how they are doing as people, invites them to describe how they use belonging, collaboration, and teaching skills in their teaching, and allows them to identify what they do successfully and what they would like the observer to appraise. Our post-observation survey asks educators what they are proud of, what they would do differently if they had another opportunity, what feedback they received from their collaborators (other educators and students), and other questions that invite thoughtful and authentic reflection on their process.

Luisa Zamora, CSM Educator, facilitates an artwork conversation in the Clyfford Still Museum galleries. Photo Credit: Fireside Production.

Our Vision Moving Forward

Our shared learning and next steps are still developing. We continue to reflect, redevelop, and reposition our thoughts each time we engage in observation and throughout this research and writing process.

Here are some insights from our new museum educator observation process:

• We have come to identify more ways that white supremacy characteristics exist in the ways we have approached our work, our organizational space, and institutional practices. Slowing down to recognize these features has led to fruitful conversations about specific tactics to dismantle them.
• Reconceptualizing our observation process alongside museum educators has been a meaningful way to practice responsive, human-centered leadership.
• Educators report a more positive atmosphere for teacher observations. One educator shared, “I appreciate how thoughtful the new observation process is. It is not arbitrary. I like how you allow us to tell leadership how we want to be supported in reaching our goals as educators. I like how the new process encourages self-reflection and supportive dialogue about teaching strategies.”
• We plan to adapt and apply our new tool when observing educators facilitating family programs and adult programs.

We are still pondering some questions:

• Why borrow from classroom strategies and practices?
• Is this an attempt to professionalize the practice of museum education? Does it work?
• Is part of the wonder of a museum that it is explicitly not a classroom?
• How do we make observation and evaluation an ongoing and sustainable practice?
• How can we continuously evolve our observation tool and process?
• Can we incorporate peer observation or student feedback to include more voices?
• Can we redefine our observation metrics alongside, rather than to, the educators?
• Can we consider contextual factors that educators cannot control but can respond to?

How Others Can Apply Our New Observation Process in their own Space

If you want to explore reimagining your museum educator evaluation process in your own space, consider exploring the following ideas in your context:

• What systems and processes underlie your evaluation approach?
• How might white supremacy characteristics be inherent in those systems?
• What antidotes can you employ to dismantle white supremacy characteristics?
• What are the power dynamics of your evaluation approach?
• Whose voices are valued? What makes them valuable?
• Whose voices are missing? What insights could they provide?
• How could you create a more collaborative evaluation process at every stage?

We hope museum professionals reading this article are inspired to reflect on how white supremacy characteristics operate within their organizations. We encourage leaders and advocates to consider their museum education evaluation processes and experiment with dismantling white supremacy characteristics within the context of their own spaces. We invite you to connect with us if you want thought-partners to dialogue with in the future.

Quána Madison (she/her/hers) is a professional artist, a community healing arts facilitator and an educator with over 20 years of experience in formal and informal education spaces. She is a senior educator and community engagement specialist at the Clyfford Still Museum. Quána deeply values diversity, equity, inclusion, accessibility and healing-centered engagement. She is a mental health and affordable housing advocate. She has professional experience with educator evaluation. A graduate of New York University and Colorado College, Quána holds a MA in Education and a BA in Philosophy.

Social Media Handles: Instagram: @quana.madison Linkedin: www.linkedin.com/in/quanamadison [Google link to image — 72 DPI — Quána Madison Headshot for Viewfinder.jpg]

Emily Bullard (she/her/hers) After receiving a BGS in Art History, earning an MA in Curriculum & Instruction, and spending a year as a classroom teacher resident, Emily Bullard became a museum educator. As Assistant Director of School & Teacher programs at the Clyfford Still Museum, she supervises a team of paid part-time educators, seeking opportunities to invite each of them to be their authentic selves in the galleries. Emily is committed to DEAI, reflecting on and confronting her privilege and creating liberatory spaces for educators and students.

References

American Alliance of Museums Standing Professional Committee on Education. (2005). Excellence in practice: Museum Education Principles and Standards. pp. 7–8. https://www.aam-us.org/wpcontent/uploads/2022/03/Excellence-in-Practice.pdf

Briggs Kemeza, A. (2019). Embracing Individualism and Encouraging Personal Style in Gallery Teaching. Journal of Museum Education, 44(2), 147–154. https://doi.org/10.1080/10598650.2019.1594566

Campbell, S.L. & Ronfeldt, M. (2018). Evaluation of Teachers: Measuring More than We Bargained for? American Educational Research Journal, Vol. 55, №6, pp. 1233–1267

Centre for Community Organizations. (2023). White Supremacy Culture in Organizations [White paper]. https://coco-net.org/wp-content/uploads/2019/11/Coco-WhiteSupCulture-ENG4.pdf

Cohen, J. & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, August/September, Vol. 45, №6, pp.378–387

Darr, K. (2002). Rubrics: An Evaluation Alternative. The Docent Educator, 12(2), 14–16.

Davis, B. (2016). Connoisseurship and Critique. E-Flux Journal, 72. https://www.e-flux.com/journal/72/60496/connoisseurship-and-critique/

Eisner, E. W. (1976). Educational Connoisseurship and Criticism: Their Form and Functions in Educational Evaluation. The Journal of Aesthetic Education, 10(3/4), 135–150.

Eisner, E. W. (2004). The roots of connoisseurship and criticism: A personal journey. In M. C.

Alkin(Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (pp. 196–202). ThousandOaks, CA: Sage.

Fertig, B. (1979). …And Measuring the Immeasurable. Roundtable Reports, 4(3), 1–16. https://doi.org/10.1080/07394365.1979.12089424

Garmston B. & Wellman B. (2019) Thinking Collaborative, Seven norms of collaborative work [White paper].https://www.thinkingcollaborative.com/_files/ugd/6a5cc9_d080a1bbb7c84e43934a019d684214d2.pdf

Harding, R. L., Kennedy, M., & Raphling, L. (2000). Tour Evaluations-Taking a Collaborative Approach to Following Up. The Docent Educator, 10(2), 14–16.

Hill, H.C., Ball, D.L. & Schilling, S.G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal of Research in Mathematics Education, Vol. 39, №4, pp. 372–400

Jenkins, J. (2020, January 29) Responsive Leadership: Needed Now More Than Ever. Leadership Now. https://www.leadershipnow.com/leadingblog/2020/01/responsive_leadership_needed_n.html

Ladson-Billings, G. (1995). But that’s just good teaching! The case for culturally relevant pedagogy.

Theory Into Practice, 34(3) y 159–165.

Littleton, J. (1994). Oh No! Not Another Test! Peer Observation to Improve Performance. The Docent Educator, 3(3), 18–19.

Merriam-Webster. (n.d.). Connoisseur. In Merriam-Webster.com dictionary. Retrieved May 31, 2023, from https://www.merriam-webster.com/dictionary/connoisseur

National Education Association. (2023). White supremacy culture resources. [White paper]. https://www.nea.org/resource-library/white-supremacy-culture-resources

National Museum of African American History and Culture. (2023). Whiteness [White paper]. https://nmaahc.si.edu/learn/talking-about-race/topics/whiteness

Okun, T. (n.d.). Characteristics. WHITE SUPREMACY CULTURE. https://www.whitesupremacyculture.info/characteristics.html

Paris, D., & Alim, H. S. (2014). What are we seeking to sustain through culturally sustaining pedagogy? A loving critique forward. Harvard Educational Review, 84(1), 85–100.

Polikoff, M.S.(2015). The stability of observational and student survey measures of teaching effectiveness. American Journal of Education, 121 (2), 183–212.

Powell Flanders, M. (1979). EVALUATING DOCENTS: Mentioning the Unmentionable. Roundtable Reports, 4(3), 1–3. https://doi.org/10.1080/07394365.1979.12089423

Schep, M., van Boxtel, C., & Noordegraaf, J. (2017). Competent museum guides: Defining competencies for use in art and history museums. Museum Management and Curatorship, 33(1), 2–24. https://doi.org/10.1080/09647775.2017.1387590

Schmoker, M. (2012). The madness of teacher evaluation frameworks. The Phi Delta Kappan, Vol. 93, №8, pp.70–71.

Specht, I., & Loreit, F. (2021). Empirical Knowledge About Person-Led Guided Tours in Museums: A Scoping Review. Journal of Interpretation Research, 26(2), 96–130. https://doi.org/10.1177/10925872211065653

Toch, T., & Rothman, R. (2008). Rush to judgment: Teacher evaluation in public education. Washington, DC: Education Sector.