By Daniel G. Wilson and Anonymous
In the past few days the focus of campus discussion has been on issues of race and the protest, “Walkout and Speakout,” organized by the Black Justice League. The BJL assembled at Nassau Hall Wednesday and issued three demands:
- A purge of Woodrow Wilson’s name from campus buildings and the public policy department.
- Mandatory “cultural competency” training for staff and faculty and a new distribution requirement in the history of marginalized peoples.
- The establishment of a cultural space on campus dedicated specifically to Black students.
In the interest of a more comprehensive view of campus sentiment, I released a TypeForm via the residential college listservs. The survey asked participants to state their opinions on several elements of the Walkout. The raw results of that data and the original questions can be found here.
The survey was open from 10:00am on Thursday, November 19 to 1:00am on Saturday, November 21, receiving a grand total of 948 responses in 39 hours. This represents about 18% of Princeton’s 5,275 undergrads—more than enough to be considered a statistically significant population.
The charts and visualization in the link above are, however, not controlled for demographics, voluntary response bias, and undercoverage. There are several opportunities here to adjust our visualizations and draw conclusions even more representative of the general Princeton population; I worked with my close friend Anonymous to produce what you’ll see below.
Anonymous: As a data scientist, and someone who has a piqued interest in the silent majority on the internet, I took the chance to analyze this data as soon as the survey came up. For data collection, as Danny said, we took into account Race, Activity, Year, and Familiarity of the Protest. However, as we can see in the plots below, most people seemed to be well-versed in the events that transpired over the past couple days. Therefore, we have no particular significant insights based on who “knows” more about the events.
The aim of this data collection and analysis is not to make any overarching conclusions or push any biased opinions: rather, we only wish to make statements of fact based on the data.
I designed the original survey with several control questions regarding familiarity with events, participant race, class year, and participation in the Walkout, which will allow us to control for over/under represented participant groups. A few areas of interest in which survey data differs from the overall campus population follow.
Race
The first area of interest was the division of the racial demographic category. The following graph shows the by-race composition of respondents vs campus undergraduates.
(Enrollment data sourced from Princeton Profile 2015–16, which can be found at http://www.princeton.edu/pub/profile/PU-profile-2015-16.pdf).
This indicates that Asians, Blacks, and Pacific Islanders are overly represented in survey results, and Whites are under represented. This may be evidence of minority issues being of greater interest to campus minorities.
The following represents the by-race mean of responses to each survey question.
A score of 5 is defined as neutral in the survey, as indicated by the black line. This indicates that the demands and protest are most favorably looked upon by Blacks, and interestingly, least favorably looked upon by Asians and Pacific Islanders.
Notice that the responses to the first BJL demand (removing Woodrow Wilson’s name) appear predominantly negative. In fulfillment of the BJL’s revised demands, Mr. Eisgruber has asserted that he is “confident that the board will put together a process on which to collect views and information on this question.” To the BJL, Mr. Eisgruber, and the Board of Trustees: we have already done so.
Also of interest is the racial composition of Walkout participants. The following represents responses by race which affirmatively indicated participation.
Interestingly, almost half of respondents who indicated they participated in the Walkout were White.
Class Year
Another area of interest is the class year participation rate. We conjecture that the Princeton population is fairly evenly divided among class years, and our survey received comparable participation.
The following represents the by-class mean of responses to each survey question.
It appears that opinions did not dramatically differ between class years, though underclassmen appear to have a slightly more negative view.
The following represents responses by class year which affirmatively indicated participation.
This indicates relatively uniform participation in the Walkout across class years with the lowest participation rate in the freshmen class.
Walkout Participation
Perhaps the most significant area of interest is the rate of participation in the Walkout. 17.6% of respondents affirmatively indicated participation in the Walkout. The Daily Princetonian reported that “nearly 200 students convened outside Nassau Hall,” and the Facebook event page indicates 252 attendees.
(Daily Princetonian report can be found at http://dailyprincetonian.com/news/2015/11/students-walkout-and-speakout-occupy-nassau-hall-until-demands-of-black-justice-league-are-met/)
If we then overestimate (to control for inaccurate reports) the number of Walkout attendees to be 300—above both the Daily Princetonian report and the Facebook attendee count—then an estimated 5.69% of Princeton’s 5,275 undergrads attended. Notice that this means attendees of the Walkout are over represented in survey results by a factor of 3.
First, let’s look at the breakdown of question responses by whether or not respondents actively participated in the Walkout.
The vast majority of positive survey responses were submitted by students that actively participated in the Walkout. Remember, however, that active participants are overrepresented in the data. This indicates that the mean opinions reported by raw survey results (which can again be found here) are skewed more positively than the opinion of campus as a whole; that is, if the data were perfectly representative of the undergraduate population, the predominantly positive votes cast by Walkout participants would represent a lesser part of the whole than in the above results.
This suggests that an adjusted dataset can be produced, controlled for the percentage of students that participated in the Walkout out of all respondents, scaled to match the percentage of students that participated in the Walkout out of all undergrads. The expected value of overall campus sentiment was produced by separating the data into those who attended and those who did not, and scaling each with respect to their actual campus proportions.
The following data is a projected representation of true campus sentiment adjusted for over representation of attendees. The orange bars represent the adjusted aggregate responses of all students, attendees and otherwise. The blue bars represent the original distribution.
Fraudulent Submissions
One common criticism is that the data in the survey is skewed because individual users could submit multiple responses. This is theoretically possible; however, TypeForm records the ip address of each submission as an alphanumeric string. The total number of duplicate strings, indicating multiple submissions from the same device, is 6.
Note that this number could include multiple different people submitting via the same device, i.e. a shared or friend’s computer. Also note that while users could, in theory, change their ip address or device to submit multiple fraudulent responses, I believe it would take prohibitively long such that fraudulent submissions from a single user would be few in number. The survey already took an average of 5:38 to complete via computer and 3:58 via smartphone. We can therefore conclude that the data is not invalidated by the ability to submit multiple times.
Anonymous: I would like to make some disclaimers regarding the data analysis and data collection:
With regards to why there are no “multi-cultural” categorical variables in our data analysis, we have to realize that the data itself will become very messy fast. I went ahead and double counted for multicultural persons by adding to the categories they claimed they were. For example, if one survey was “Asian White,” I would place that person in both the Asian and the White Category. While not ideal, we believe that Princeton represents their statistics with regards to racial demographics in the same manner: therefore, I believe that our overcounting is consistent with Princeton’s methodology.
In our adjusted means plot, we have done some math that may be inconsistent with traditional statistical methods. We went ahead and tried to take the adjusted means of how campus representation would look like based on our data from the surveys. Even though this extrapolation is a little quirky, we found that the results were rather intuitive and followed what should have happened to the data.
Concluding Notes
We have shown that there is a significant amount of information to be gained from this data, and that we are well-able to control for many of the objections raised to its methods of collection. The intent of this survey, to bring campus as a whole into the conversation in a more formal, visible manner seems therefore to have been successful.
We’d like to thank those who offered their help and advice during this arduous process: SuMin Park ’18, Prof. Robert Vanderbei, Natalia Chen ’18, and Kármen Riv ’18.
For those interested in further analysis or fact-checking, the raw data can be found in its entirety here.
