Section 12: Areas for Further Research
Our inquiry into these four communities of transformation (CoTs) reveals several areas in need of further research, particularly as we seek to understand the phenomenon of STEM education community formation and the task of scaling up reform. This section outlines several areas that would benefit further inquiry.
The Continuum from Community of Practice to Social Networks
We entered this study anticipating that some of the entities that were our subjects might reflect a network approach, and others might be more appropriately understood as communities of practice (CoPs). Through our research, however, we found that all four of the entities were best understood as communities—in particular, communities of transformation (CoTs). While the entities that we studied did not turn out to be networks, we believe that there may be reform efforts appropriate to some settings that could emerge from a more passive, networked approach. Such an approach would be defined by more informal interactions, loose structure, and a focus on fostering relationships based on common disciplines or research interests, rather than on nurturing communities like those in our study. We believe that future research should be aimed at identifying other variations of reform communities, particularly ones that act like social networks, and the challenges and opportunities of those groups. In the end, our study was able to focus on the benefits, design, and lifecycle of CoTs, and another study could do the same for networks.
Communities of Transformation
Through this study, we identified a new variant of communities of practice, which we called communities of transformation. Since we did not go into the study with the purpose of identifying and documenting the structure of a CoT, we feel that it is important that further research be conducted to better understand it as a new variant of a CoP.
As outlined in section 4, we identified three primary features of CoTs: 1. Formation and documentation of a philosophy, which is an innovative approach and radical departure from existing practice; 2. Living integration of the philosophy throughout events and activities, modeled by the leadership of the organization; and 3. A community that supports the new practice once an individual returns to a status quo community. We identified some unique aspects present in how these entities are formed (importance of formulating a philosophy), how they sustain themselves (ways that the philosophy is embedded into the design), and their outcomes (a network of change agents that helps create and sustain change in status quo spaces).
One of the important outcomes of these CoTs is that they are able to create deep changes related to STEM reform; for example, they challenge the underlying assumptions about being a scientist and approaches to conducting science. Such deep philosophical issues are often not embedded in the evidence-based teaching practices for which these communities advocate, but they emerge organically from the philosophical orientation of the work of the communities. Advocates for gender parity and improved success of minorities in STEM have identified these types of deep changes as central to the project of rethinking science and making it inclusive of broader groups of people. Much work is still needed to fully understand how these communities of transformation can contribute to the philosophical changes needed to make science more equitable. We imagine that future research can look at challenges related to the philosophical side of these communities; for example, can it lead to philosophical divides, dogmatism, or lack of flexibility?
We also found that CoTs are not always intentional about preparing people for returning to the particular status quo cultures present on home campuses. For example, some faculty participants return to campuses that are supportive of innovative pedagogies, but the faculty are not necessarily prepared to maximize their impact in these settings. Therefore, it is important to look at the ways that these communities provide potentially different types of support for individuals that are returning to such different environments.
In addition, our study only examined CoTs that were non-organizationally located. However, this is not a necessary feature of such communities. Future research should seek to identify CoTs that may exist within organizations in order to examine their abilities to engage community members in deeper forms of engagement. Do such organization-based communities exist? Can they exist?
Social Networks within Communities of Transformation
In section 2, we reviewed the literature on social networks because the communities in our study clearly foster relationships among their members. However, our study did not focus on understanding the networks of relationships in these communities; rather, we focused more on participant engagement within the community and the ways in which community design can influence outcomes. Future research should examine the extent to which the informal networks of personal relationships formed in these communities contribute to spreading the reform strategies. Members of the POGIL Project leadership have begun this work, examining how their network of workshop facilitators has grown and their approach has spread to different parts of the United States. Social network analysis is a useful analytical tool to identify the ways in which community connections are made and grow, as well as to study the spread of adoption of these STEM reform strategies over time.
Non-organizationally Located Communities of Practice
Because the entities in our study are either not organizationally located or began outside of organizations, they face some specific challenges related to sustainability; we were able to identify many of these challenges in section 10. However, we did not go into the study considering the non-organizational character of these communities as a central influence on their design or outcomes. After seeing how the non-organizational character of the CoTs shaped sustainability, we wonder about its impact on these other areas. For example, how can the right level of mentoring be maintained within a loosely organized group? Perhaps there needs to be a balance between in-person and virtual means of connection in order to achieve outcomes in these non-organizationally located communities of practice. We did not specifically study these types of issues. Future studies should continue to explore the impact of not being organizationally located on the way these entities operate and on their ability to meet their STEM reform objectives.
Our survey was focused on individual, departmental, and institutional impacts from participation in the STEM reform communities. We initially had questions related to broader impacts, beyond the institution, but our advisory board suggested that we remove these questions to avoid cognitive overload on the individuals taking the survey and to go deeper into the impacts that were at the localized level. In addition, they advised us that individual members of the community, the target of the survey, would be less likely than leaders within the communities to identify broader impacts. We appreciated the advisory board’s advice, and we believe this was justified and important feedback. However, we also found that there are many broader impacts that these communities of transformation were able to contribute, identified in our interviews but not reflected in the surveys. We believe that continued research to identify these broader impacts will help us to understand the contribution of these communities in the long term. Currently, we have qualitative data about broader impacts, but further study is necessary to ascertain whether these findings are generalizable.
Other Approaches to Examining Outcomes
It is challenging to examine outcomes in broad-ranging CoTs such as those we studied. Because of the scale of these communities, we do not have the ability to observe changes in practice or to follow departmental changes—approaches that would be feasible in the case of a local CoP. Logistically, self-report data was one of our only viable options. One approach to measuring outcomes would be to examine institutional data from institutions at which faculty who belong to these communities work. Such research could examine if there have been noticeable improvements in objective measures, such as retention and persistence in STEM disciplines or changes to the number of STEM majors. Outcomes assessment could also be enhanced by identifying comparable institutions at which these strategies and practices are not employed, thus creating a benchmark for institutions active in these CoTs. Another approach for understanding departmental and institutional outcomes would be to survey deans or department chairs at a random selection of campuses involved. Even then, however, it would be a challenge to obtain a large sample from those campuses; surveys among academic leaders typically have low response rates. Thus, we suggest that future research focus on a subset of institutions with faculty involved in these communities. Research can then try to document changes in individual faculty, departments, and the institution broadly to try to evaluate the impact of these communities.
Comparison of Different Models of STEM Reform Efforts
Approaches to STEM reform differ in terms of their targets: some focus on disciplinary societies; others focus on institutional change (such as the Association of American Universities projects); some emphasize networked approaches (such as the Bayview Alliance networked-community approach); and others focus on individual faculty (such as the CoTs in this study). There have been few studies that examine and compare these various approaches to STEM reform, and we lack data on whether some work better for certain purposes or reasons. For example, we know from our research that the CoTs we studied are particularly important for faculty who are isolated on their campuses and do not have other reformers to work with. However, individuals who reported having a lot of support on their campuses told us they were less likely to be deeply involved with these CoTs and to have ongoing involvement. There might also be ways that various approaches and strategies for STEM reform can be aligned to support one another. Research that compares these different approaches could seek out these types of alignments and synergies.
The Tension between Stability/Sustainability and Appeal of Informal Community
We identified a model for long-term sustainability for these groups. We did wonder if, as they became more
organizational and began to be more formal, the communities might lose the very features that faculty found most
engaging. For example, participants identified the informal peer-to-peer work, in which ideas could be debated, as one of the most attractive aspects of these communities. Yet, as the CoTs moved in the direction of increased stability, we could see that they lost some these types of features. Future work should examine these groups in their later, more formalized state to identify the impact on faculty engagement and outcomes associated with moving into a more sustainable, long-term model.
The Challenges of Expansion
Our research identified key avenues for expansion, leverage points to gauge which directions to expand into, and challenges of expansion (see section 9). While we unearthed some solutions to those challenges, we did not find practicable strategies to be used in every area. Future research could explore additional ways communities can overcome the challenges we identified.
The Effect of the Changing Faculty on STEM Reform
All of these communities expressed concern about the changes in the academic workforce, and particularly about the ways that these changes may impact their ability to connect with faculty. As faculty are increasingly off the tenure track, overburdened, and under-supported, how can communities initiate and sustain reform? It is important that future research explore the impact of the changing faculty on efforts to improve STEM education in general, and in particular among these CoTs.