Transitioning from Small Scale Teams to an Ed Tech Powerhouse
Written By: Paul Martinez, Ph.D.
Working at Western Governors University (WGU) has provided me a completely new understanding of what it means to be a part of an innovative, technologically driven higher education institution. In this post, I would like to share my personal experience of transitioning from working at research centers at several higher education institutions to joining the Institutional Research Department at WGU. I will specifically focus on the fundamental structural differences in team dynamics and data tools across the institutions I worked for.
For those who are not aware, WGU is a private non-profit online university that was officially founded in 1997 in the United States by the governors of 19 states. Unlike other universities, WGU provides a competency-based learning model, as opposed to the traditional cohort-based class model, which allows its students to demonstrate mastery of a competency, and as a result focuses on outcomes and real-world performance.
It was at WGU that I first worked alongside data engineers, data analysts, data scientists, and a project manager. I could have never imagined how crucial each individual role would be for efficiency in the process of mining, cleaning, and analyzing data. To contextualize the novelty of this team structure, at my previous research positions, the majority of my colleagues, who were graduate students like myself at the time, did not have much experience as analysts and were expected to wear multiple hats, switching between data engineers, data analysts, and data scientists. I believe we managed well and did the best possible work we could with the training we had. Little did I know that my conceptual understanding of “efficiency” would be forever changed once I was introduced to my current team, received better training and gained access to more tools.
What exactly are the benefits of working with a team composed of data engineers, data analysts, and data scientists? From my perspective, I would argue that having well-defined roles allows each member of a team to contribute to a particular task and hone in on their expertise when tackling such tasks. For example, data analysts are typically required to identify trends that help leaders make strategic decisions. To do so, they use tools such as SQL to query relational databases, programming languages like R and SAS, and visualization tools like Power BI and Tableau. Data scientists, on the other hand, will typically design data modeling processes and create algorithms, automation systems, and predictive models using programming tools like Python, Java and/or machine learning. While data scientists and data analysts may have overlapping tasks at times, collaboration between them is instrumental to completing projects more efficiently and systematically. For instance, let’s imagine that Report A is required on a monthly basis. If the data analyst and data scientist don’t collaborate, the data analyst would update their SQL query code to reflect the updated numbers and refresh their Tableau dashboard every month to produce Report A. If the two collaborate, however, the data analyst can share the SQL query code with the data scientist once, and the data scientist can then fully automate the report to update itself every month. While this example is an oversimplification, my experience at WGU has reflected the collaborative scenario to a certain extent. Having access to an entire team of data experts has allowed me to fully automate projects that are “magically” updated every month without having to worry about a thing.
Next, I would like to discuss the data tools and some of the key differences that make WGU distinct compared to my previous experiences. Every researcher position I held prior to coming to WGU did not have an organized data warehouse with well-kept historical records and a process in place to clean, manage, and maintain data. The tools we did have at our disposal, which were mostly based on our training, were Excel, SPSS, and/or Stata. Being exposed to my current programming tools and data warehouse made me realize how much more efficient one can be when working with a more advanced infrastructure. Recently, WGU decided to transition over to using Databricks, which is a “Lakehouse Platform [that] combines the best elements of data lakes and data warehouses …This unified platform simplifies…data architecture by eliminating the data silos that traditionally separate analytics, data science and machine learning. It’s built on open source and open standards to maximize flexibility.” (for more information visit https://databricks.com/product/data-lakehouse). One huge advantage of using Databricks is the multilingual empowerment it provides. For instance, Databricks allows for multilingual coding in notebooks, mixing languages based on the task at hand (e.g., Scala + SQL+ Python). (Follow the link for more information https://databricks.com/session_na21/10-things-learned-releasing-databricks-enterprise-wide). Additionally, Databricks allows you to create an entire report from the data importing/merging stage to the analysis stage, and even automate the notebook to run every month if needed. Using Tableau, you can then connect to the Databricks server, import the report, create a dashboard for optimal visualization and have it automatically update each month. The toolkit I am now equipped with allows me to be more efficient than I could have ever imagined, and what’s even more impressive is that I have yet to tap into all of Databricks’ features given my short time using the platform.
Overall, I believe WGU is using cutting-edge technology, which allows for higher rates of productivity, more collaboration among data experts, and a systematic process of mining, cleaning, managing, and analyzing data. I would argue that having access to a diverse team of data experts should be a key feature in any research center or university. Having reliable data is necessary when making any business or strategic decision. Ultimately, WGU’s forward-thinking stance on technology allows for optimal student record keeping and outcome reporting. Finally, I would like to clarify that all statements made here are opinions based solely on my experience and therefore do not represent the perspectives of my colleagues, my department, or the institution.
About the Author
Paul Martinez, Ph.D. is a research analyst at Western Governors University. Martinez’s role focuses on state/federal reporting as well as ETL, report and targets development, and executive report production. He holds an Ph.D. and a M.A in Sociology from UCLA, and a B.A. in Sociology from Sonoma State University. Martinez’s previous work focused on using quantitative and social network analysis methods to examine race/ethnicity, gender, and class inequities in k–12 and higher education settings.