A Fool’s Errand
When I was first asked given a list of dates I could choose for this lecture, I noticed that April Fool’s Day was on the list. How could I pass that up? So, let me lay out a few of the foolish things I hope to do during this lecture. I plan to use science as an analogy to discuss other topics. This is something that I almost always find infuriating. Scientists and engineers like precise, quantifiable things, and analogies usually tend toward the sloppy and vague. Also, I plan to use these analogies for discussing teaching and outreach, implicitly asserting that these are of comparable importance to engineering research. If at that point I am not pelted with stones, I will attempt to provoke the stone throwing by verging on the overtly political. Hopefully by then I will have earned the title Professor, Fool or both.
Strong as Glass
Glass is a common material we are all familiar with, but silicate glasses that are widely used for building materials and commercial products are only one example. What defines glasses is that can be cooled from the liquid state into the solid state without forming crystals. Instead the network of bonds that holds the glass together still looks as it did in the liquid. The bonding between atoms remains disordered, connected to each other seemingly at random. It is often asserted that a glass is just a very slow liquid, but this description downplays the miraculousness of glasses. For any practical purpose glasses at room temperature do not flow. They are no less solid than crystals, and they manage to do this without ordering their atoms.
Glasses occur in almost every kind of material including ceramics, plastics and semiconductors. Metals are rather difficult to make into a glass, but about 50 years ago it was discovered that this was possible by cooling the metal very, very quickly. About 20 years ago materials researchers figured out how to make metals into glasses without cooling them quickly by instead carefully tuning their elemental compositions, designing new alloys. These metallic glass alloys are super strong, and, since they don’t crystallize as they solidify, they are easier to form into desired shapes without the need for costly machining.
Recently Apple Computer licensed the metallic glass technology in the hope of making durable cell phone and computer cases. Some of this technology has also been used by OMEGA for their Seamaster Planet Ocean Liquidmetal Limited Edition watch. The video below shows a nice example of the formability of metallic glasses. Please don’t be put off by the fact that they had me use my sexy robot voice to narrate the video.
Unfortunately metallic glasses have an Achilles heel; when they exceed their high strength, they fail catastrophically. This happens by a process known as shear banding. When the forces on the material become too high one portion of the material slices off from the rest. The plane of separation forms spontaneously, seemingly at a random location, but always along a direction that promotes slip between the resulting pieces so as to relieve the stress. This phenomenon provides a big challenge for materials scientists. I study this problem using computer simulations in which the motion of all the atoms in the metallic glass is simulated under an applied a stress. These simulations provide a window into how the shear band develops. The simulations show that the glass structure becomes increasingly disordered as it slips. This weakens the material, and the feedback between disordering and slip causes the glass to become unstable producing a shear band.
To make this blog post more readable I decided not to delve into the scientific details of how we understand this scientific problem from the vantage point of thermodynamics here. Rather I have packaged that in an associated blog post that I am calling (in homage to Pedro Almodovar and Charles Dickens), “Glass on the verge of a mechanical breakdown: A tale of two temperatures.” Please follow the link to that blog if you are interested.
Disorder as a virtue
In addition to thinking about glasses I have also been working on two rather different projects that are equally close to my heart. The first is the incorporation of computation into the engineering curriculum and the second is engagement in science, technology, engineering and mathematics outreach to Baltimore City Schools through a unique partnership that involves the Whiting School of Engineering and the School of Education here at Hopkins. As I think about these domains of my work, it seems to me that all involve issues of disorder and breakdown. When I think about disorder, however, I don’t think about it in a pejorative sense. Disorder is natural and sometimes beneficial. Disorder is complex and can, in certain circumstances cause a system to be robust. After all, disorder gives the metallic glass its phenomenal strength. It also allows it to move seamlessly between liquid and solid without going through the disruptive process of crystallization that would lead to less ideal properties.
In many ways student learning is about forming bonds between elements of knowledge, some pre-existing, some recently acquired. These connections must be formed so as to provide a robust network of understanding useful for grappling with real world problems. When this network is put together haphazardly or assembled rapidly without proper supports students can experience distress. This distress arises because students realize that the constructs they are using to draw inferences are fragile. This network of mental connections is much like the network of bonds in the glass, which only become strong if properly constituted and gradually formed.
Schools provide another example of a robust but random system. The networks of social relations, between and among students, teachers, parents, administrators, community organizations and others are what make a community function well or poorly. Achieving excellence in meeting students’ varied educational and developmental needs requires rich and varied connections between all parts of the community. Growing this network takes slow nurturing. Attempts to reform education by regimentally imposing an ordered top-down structure are not only disruptive; they often result in systems that are less responsive and more subject to breakdown due to unintended consequences not foreseen by the reformers. Again, as in the glass, slowly nurtured disorder carries the day.
How do Engineers Learn to Compute?
Over the past three years I have started engaging in education research. Our department is actively building a research focus in computational materials science, which involves using computer simulation to study existing materials and design new ones. But little of this had percolated down to the undergraduates. Students were required to take an introductory programming course, but were permitted to take this course any time and many would decide to put this off to the end of their studies. As a result it was difficult to assume any baseline of computing knowledge in our core coursework.
Our department has now implemented a required first-year programming course called Computation and Programming in Materials Science and Engineering (CPMSE) in which students learn to write simulations of materials problems. The aim was to give students solid baseline programming and computational thinking skills while also demonstrating how computing functions in a Materials Science and Engineering context. The class was structured in an active learning mode where students spend class time working on problems alone and collectively and discussing their results. As a follow-up to this foundational experience we implemented twelve computational modules that students would encounter over the following two years in their six core courses. These modules aimed to leverage students’ computing expertise to increase understanding of core disciplinary concepts. They could involve a small amount of programming such as a visualization of a quantum wave packet in MATLAB. They could involve using software one might encounter in an industrial setting such as a finite element package called COMSOL that students use to simulate structural failure.
To measure the effect of this curricular change we wanted to assess student adoption of computation as a technology. We used a construct developed in the context of other innovations called the “Technology Acceptance Model” that assumes that people are more likely to adopt a technology if they feel they have mastery of it, agree that it is useful and then develop an intention to integrate it into their life and work. We asked students to self-report about their feelings of ability, utility and intention. It was gratifying to see that these measures increased markedly when students took the foundational programming course in our department. However, the most interesting result became apparent only when we continued the assessment into the following term as students started to engage in their core coursework. Students who had taken this contextualized introduction to programming showed significantly higher technology acceptance measures than students who had taken multiple other programming courses, even those who had taken 3 or more computing courses. This has led us to believe that contextualizing the teaching of computing within disciplinary learning is critical for training the next generation of engineering students. 
The issue of disorder came up when we tried to get further into the students’ heads. As the students took the introductory computation class or worked on computational modules in their core courses we assessed their learning of the associated disciplinary concepts. We did this by surveys and short multiple choice tests, but we also pulled a few students out and had them do similar exercises after the fact and talk through their thinking process in a kind of guided interview. The multiple-choice tests showed that most, but not all, of the modules were accompanied by statistically significant changes in student conceptual understanding. The goal of the guided interviews was to understand what distinguished success from failure in students’ use of computation to understand important engineering concepts.
These interviews revealed that what we are asking students to do is quite complex. These students start with a set of prior knowledge some of which is relevant and correct and some of which is irrelevant or false. In the process of mental model building students attempt to map their prior knowledge to the concept in order to incorporate the concept into their understanding. During the learning process a student tests their understanding of an engineering concept by using it to relate an engineering sytem to a mathematical model in order to solve an engineering problem. Once an answer is formulated in the mathematical representation, they must again deploy their conceptual understanding to translate this answer back into the context of the engineering system so as to establish the meaning of the result. This is also essential to evaluate the correctness of the solution and the appropriateness of the mathematical model.
In asking students to work with computation we are adding more levels to this hierarchy of mapping. We are asking students to use additional conceptual understanding to map the mathematical model onto an algorithmic model and then into either a program or a representation within a software environment. In the former case, which is particularly emphasized in the CPMSE course, this mapping is very explicit. In the latter case the software environment obscures many of the details of the algorithmic model and program.
So, how do students react to the additional demands of the computational tasks? In many instances the addition of the computation appears to increase or reinforce the students’ understanding of the disciplinary content. In the introductory computing course 58% of students noted that the “class workshops on the projects were very interactive and engaging” and 47% noted that “The projects are very interesting and challenging at the same time.” One student commented that, “Projects were very interesting and helpful to learn the methods and tools we learned in class.” In the context of the think aloud exercises students also expressed this sentiment in more detail, stating:
Actually just like sitting down in front of MATLAB just testing new things definitely helps, you just have to spend some time playing with it. This exercise was extremely similar to the computational modules I would say. I did the Monte Carlo essentially twice in structures and in thermodynamics so it’s pretty embedded in my mind by this point. I think like the programs definitely help some with comprehending the material because it’s kind of hard at least for me it’s hard to imagine. It seems kind of like a black box. You throw in a bunch of energies somewhere. Lower the temperature. It might orient itself differently and then you get the new orientation right in the program and actually seeing in our case the red and blue squares moving around. It’s a good visual aid to see what’s happening.
However, in many instances students report what is described in the educational literature as “cognitive overload”. That is they report that having to juggle this hierarchy of models taxes their ability to concentrate. In the introductory programming class 61% of students noted characterized the content as “very difficult and highly mathematically intensive.” One student noted, “Some of the projects seemed a bit too background heavy. Since this was still technically a programming course, it felt unfair to have to devote days to unraveling the complex math behind the theory of the project. While it was fair to expect students to develop a program to solve problems they had not previously encountered, certain projects were simply too involved.”
This issue of cognitive overload also came up during the analysis of student think-aloud exercises related to the computational modules performed in class. When asked at the end of the exercise to reflect on the most difficult concepts examples of students’ responses included:
Why exactly E to the negative delta? Or why kBT? But I remember thinking that out once, by reading the book and Wikipedia, but I just forgot now. That would be something very nice to know.
I guess just Monte Carlo, and more generally, like, actually understand the math behind it and stuff, instead of just grasp[ing] at it.
I understand the random number between one and zero. But why— what is this X one, where does it come from?… It’s not necessary to answer the question, but it’s just the one part I don’t really understand.
It seems clear to us by analyzing many such student responses that asking students to make these additional connections required by combining disciplinary learning and computation is highly worthwhile, but at the same time requires providing additional supports. In particular students need support connecting the dots between the concept and their prior learning as well as between the engineering system, the mathematical representation and the computational representation. They also need active feedback so as not to make improper mappings between concepts and representations and amongst various representations. Exercises that employed software environments that shrouded the algorithmic details did not run this risk of cognitive overload to the same degree. However, a side effect was that students had difficulty extending their limited computational experience with the software environment beyond the examples provided to address closely related problems.
In summary, students are most productive in their learning when they are connecting the dots properly between elements of their knowledge. By gradually building the pattern they build a cohesive conceptual model and make sense out of previously disjoint fact and observations. Figuring out how we can help students build appropriate cognitive connections between disciplinary concepts, mathematical models and computational constructs is critical, as is helping them discard improper connections. Providing well-crafted learning contexts and well-timed assistance is important for avoiding cognitive overload that derails the learning process and prevents students from benefitting from a rich learning environment.
Finally I want to launch into a discussion of the work I have been doing regarding outreach within Baltimore City Schools. In 2012 we were awarded a $7.4 million National Science Foundation grant that is the first ever Community Enterprise for STEM (Science, Technology, Engineering and Mathematics) Learning in the country. The project, named STEM Achievement in Baltimore Elementary Schools (SABES), focuses on building STEM as a value within three Baltimore City communities by engaging with grade 3-5 students and teachers, their school administrators and parents/caregivers in collaboration with City Schools, non-profit afterschool providers, community development corporations and local area STEM businesses. An integral part of this is deploying JHU students, postdocs and faculty as mentors within the afterschool environment.
In the SABES project we are trying to build connections within each neighborhood and between these neighborhoods and Hopkins in order to foster STEM education despite a system that is highly unstable. Of our three SABES schools in our year one cohort, two had new principals last year and one had their afterschool partner shut itself down and had to find a new partner. The challenges of working in such an unstable environment are many, and the national policy prescriptions used to address our educational challenges over the last twelve years have been, in my opinion, totally counter to the needs of our Baltimore communities. We need to foster a web of connections, not organized according to a one-size-fits-all master plan, but grown in the context of our communities as they struggle to meet the educational needs of their students.
Why are you teaching science?
My original motivation for embarking on the project that became SABES was the privilege of supporting my husband who, in addition to being a snappy dresser and a Facebook addict, is a fantastic 3rd grade teacher in the Baltimore City Schools. It was clear from the beginning that his job as a teacher was going to be important, challenging and heart wrenching. The surprise was that the system in which he and other teachers work is constrained in many ways that severely limit teacher effectiveness.
I want to make clear that I am not singling out Baltimore City Schools. Discussions with teachers from around the country have led me to believe that these constraints, which arise due to the increasing desire to achieve educational reform through narrowly tailored mandates, are felt in communities of every income level and demographic. The effect of these constraints, however, is much more acutely felt in communities that deal with the most vulnerable students: those contending with poverty, specialized educational needs and linguistic barriers.
In my husband’s particular case, he was placed in an elementary school with a high fraction of children living in poverty. Due to underperformance on high-stakes tests in English language arts and mathematics, his first school had been designated for “school improvement.” This meant that further sanctions, including the possibility that the school would be shut down and all staff required to apply for re-hiring, hinged on dramatically improved performance on these tests. So what determines adequate performance? Success is defined as a certain number of students exceeding an arbitrary cut score that determines if students are deemed “proficient.” At the same time parents are free to move their children to other schools resulting in a ratcheting down of the school’s budget. This eventually precipitated wrenching staffing issues that led to the displacement of all new teachers assigned to this school. In the midst of all the many absurd aspects of the situation, one particular shock to me was that although his job ostensibly included the teaching of science, when he was observed teaching science he was reprimanded…
Why are you teaching science? That is not a tested subject in this grade.
What is No Child Left Behind? (NCLB) On its surface some aspects of the federal education law passed in 2002 and still in effect today (despite local waivers to some provisions) seem quite logical. On the elementary and middle school level the law mandates state testing of children in mathematics and reading on an annual basis starting in 3rd grade. It mandates testing of science in grades 5 and 8. Other traditional subjects are not tested. It connects high stakes consequences to schools, teacher and administrators in schools in which a designated fraction of students do not reach proficiency, an arbitrarily determined benchmark. That fraction was mandated to reach 100% by 2014. Starting in 2012 most states received Federal waivers from these unrealistic targets in return for agreeing to other constraining accountability measures.
Did NCLB change the trajectory of educational trends in the US? The data says otherwise. The National Assessment of Educational Progress that has been given since 1969 shows a gradual increase in Math and Reading scores since the inception of the assessment. Although gradual increases continued, NCLB was not accompanied by any significant change in the rate of increase of these scores; if anything the rate of increase dropped. Neither has NCLB succeeded in increasing the rate at which we are closing the racial achievement gap, as was an explicit intention of the law.
there is always a well-known solution to every human problem — neat, plausible, and wrong. —H.L. Mencken
NCLB was just such a solution. However, it is clear that the effect of NCLB was not neutral. It is a standard systems engineering principle that narrowly targeted optimization has a high probability of degrading other equally important but unmeasured aspects of the system. This is not a new concept in the area of social science and policy, where “Campbell’s Law” states,
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. 
Cambpell is also quoted as saying,
…achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.
Multiple reports prepared since 2008 have found that NCLB has resulted in a decrease in the number of hours spent teaching science and social studies in the elementary grades and a concurrent decrease in science achievement on the science NAEP assessment.,,
In my opinion this should be a cause for concern for anyone who cares about the future of the scientific and engineering enterprise in the United States. SABES is a local attempt to refocus attention on the things we know lead to increases in student STEM achievement: building teacher capacity and knowledge so that they can teach students through hands-on science and engineering activities. At the same time we aim to see how far we can reach beyond the classroom to address issues that hold children back who struggle with poverty and linguistic barriers. To do this we are building a connection between what goes on in the classroom and the world of the child. Most Baltimore city children grow up not knowing any engineers or scientists. To find a way to communicate to students and to their parents and caregivers the value of STEM we are creating spaces in which we can engage students in STEM projects that are meaningful to them. For that reason our partner schools are required to host afterschool programs in which students can engage in STEM activities in a more flexible context than is possible during the school day. Members of our Hopkins community visit these afterschool programs not once, but on a regular basis so students can gain mentorship and build relationships of trust.
Recently we have started to pilot the most exciting part of SABES, transitioning these afterschool students toward working on projects of their own devising. We asked 4th and 5th grade students in the Barclay Elementary/Middle school to walk through their neighborhoods and determine an issue they want to address using STEM. The 4th grade was unhappy with the level of trash on the streets and decided that they are interested in building giant trash collecting vacuum robots. As a result they have met with city street sweeping program experts and have started taking apart vacuum cleaners to learn how they work so they can build their own prototypes. The 5th grade decided that they wanted to use STEM to address homelessness. They have met with anti-homelessness advocates and have been thinking about how one would go about building an emergency shelter that would keep a person warm and dry. Hopkins volunteers have been invaluable in making these activities possible and expanding these students’ horizons.
Rather than go on ad nauseum I would like to end by showing you a short video I put together in which some of our Hopkins volunteers and the students in SABES speak for themselves.
I thank you for your attention to these longwinded and foolish comments. I want to finish by thanking my husband, my parents, my students and postdocs through the years, my science and education research collaborators and the SABES Team, my Johns Hopkins and University of Michigan colleagues, particularly Joanna Mirecki-Millunchick for being a strong support when I was just a baby professor, and the National Science Foundation for being so generous in supporting me throughout my career.
Michael L. Falk is a professor in the Materials Science and Engineering Department (joint appointed in Mechanical Engineering and in Physics and Astronomy) at Johns Hopkins Whiting School of Engineering in Baltimore, Maryland.
 Thomas S. Dee, Brian Jacob, “The impact of no Child Left Behind on student achievement,” Journal of Policy Analysis and Management, Vol. 30, No. 3, pp. 418–446, Summer 2011. http://onlinelibrary.wiley.com/doi/10.1002/pam.20586/full
 Jaekyung Lee, Todd Reeves, “Revisiting the Impact of NCLB High-Stakes School Accountability, Capacity, and Resources: State NAEP 1990–2009 Reading and Math Achievement Gaps and Trends,” Educational Evaluation and Policy Analysis, Vol. 34, No. 2, pp 209-231, June 2012. http://epa.sagepub.com/content/34/2/209.short
 Sean F. Reardon, Erica H. Greenberg, Demetra Kalogrides, Kenneth A. Shores, Rachel A. Valentino, “Left Behind? The Effect of No Child Left Behind on Academic Achievement Gaps,” Working Paper, Center for Educational Policy Analysis, Stanford University, 2013. https://cepa.stanford.edu/content/left-behind-effect-no-child-left-behind-academic-achievement-gaps
 “Instructional Time in Elementary Schools: A Closer Look at Changes for Specific Subjects,” Center on Education Policy, February 2008. http://www.arteducators.org/research/InstructionalTimeFeb2008.pdf
 Rolf K. Blank, “What Is the Impact of Decline in Science Instructional Time in Elementary School? Time for Elementary Instruction Has Declined, and Less Time for Science Is Correlated with Lower Scores on NAEP,” Report prepared for the Noyce Foundation, 2012. http://www.csss-science.org/downloads/NAEPElemScienceData.pdf
 Thomas S. Dee, Brian Jacob, Nathaniel L. Schwartz, “The Effects of NCLB on School Resources and Practices,” Educational Evaluation and Policy Analysis, Vol. 35, No. 2 pp. 252-279, June 2013. http://epa.sagepub.com/content/35/2/252