Section 6: Design for Engagement & Outcomes
One of our central research questions was: How do members and leaders of communities of practice (CoPs) perceive CoP design (membership, structure, communication, activities, and organization to support new knowledge development and action) shapes the ability to achieve goals (around undergraduate STEM pedagogical change and diffusion)?
We utilized both qualitative and quantitative data to answer this question as we examined how these communities are designed to facilitate engagement and contribute to outcomes. Our findings related to the ways that communities of transformation (CoTs) are designed to engage faculty and accomplish their outcomes, and we found this to be distinctive from the literature on communities of practice and social networks. Therefore, the findings below represent distinct and new ways to think about designing STEM reform communities to be successful. These findings are also supplemented by section 7, on the formation and lifecycle of CoTs, which reviews how these communities developed the engaging philosophies and communities described here. These two sections together are meant to help readers understand how CoTs can be structure to best meet their goals.
Communities of Transformation as Unique and Important Professional Development Opportunities
Before describing how these communities are designed for engagement and the achievement of outcomes, it is useful to compare the opportunities presented by CoTs to other professional development opportunities available to faculty, such as conferences or workshops organized by disciplinary societies or on-campus professional development resources. This comparison can show some of the potential advantages of CoTs.
In disciplinary societies, research is often the main focus of professional development. Typically, these societies bring together large numbers of faculty and have few venues for more intimate interactions that lead to deep learning. In contrast, the reform communities of transformation in this study focus on educational outcomes, particularly as they arise from developing faculty as teachers and contributing to individuals’ efficacy for leadership and change. These communities engage in faculty development more intentionally, drawing on research on teaching and learning that is ignored by many traditional forms of professional development. Also, as a result of the interpersonal component of these CoTs, these communities provide ongoing support for faculty members over the course of their careers—we noted that many of the communities we studied would provide letters of support for faculty in their tenure and promotion processes.
Among on-campus professional development opportunities, Centers for Teaching and Learning often offer training around the latest development or issue in pedagogy (e.g., problem-based learning), but this training may not be aligned with faculty members’ interests. Most such teaching development is practical in orientation. In contrast, the CoTs in this study provided teaching-oriented professional development that was decidedly philosophical in orientation; participants gained a deep connection to this new approach to teaching, something impossible through a simple skill-development session. Because the CoTs were founded on a specific philosophy toward STEM reform and teaching, they provide faculty with sustained, focused development opportunities, rather than ones where the focus constantly changes.
These communities also differ in the level of engagement and trust developed through involvement in them. As we mentioned earlier, the benefits cited in conjunction with our interview and observation data suggest that these communities are incredibly fulfilling, both personally and professionally. Individuals indicate this sense of fulfillment in their descriptions of the trusting relationships that are formed through their involvement. A defining feature of these communities is the ongoing opportunity to develop relationships in intimate settings, through intentional engagement with both leadership and community members. This shines through in the trust that participants describe as operative within these communities. In CoTs, engagement is modulated across the various sizes of gatherings, offering faculty opportunities to participate in both large group and more intimate settings.
In Table 6.1, we outline these key differences we identified among professional development opportunities through campus-based efforts, disciplinary/professional societies, and the communities of transformation we studied. These differences, which were not only communicated to us by community members but also observed by us through participant observations at community meetings and events, suggest some of the ways in which these communities benefit individual faculty participants and their home departments and institutions.
Table 6.1 Comparison of Typical Professional Development through Campus-Based Opportunities, Disciplinary/Professional Societies, and Communities of Transformation
|Campus-Based Opportunity||Disciplinary/Professional Society||Community of Transformation|
|Nature of Involvement||Topic of focus tends to rotate (although some campuses have professional learning communities on a topic); involvement is occasional and not on-going; weak relationships formed||Annual involvement through meetings; typically weak relationships formed through participation||Involvement through a mix of annual events that bring people together for an extended period of time, with combination of large and intimate spaces within those meetings and a mix of ongoing communications and involvement throughout the year; potential for building stronger relationships|
|Focus||Focused on teaching more generally||Focused almost exclusively on research within a discipline||Focused on teaching and skill-building, utilizing a cohesive philosophical approach across experiences; community forms around this philosophy|
|Size||Typically an intimate size, which can be intimidating to people who want an introduction or are not sure about the involvement||Overall meetings typically large and impersonal; those focused on teaching have few places to gather||Mix of large venues that allow networking with national leaders, as well as small spaces to interact; allows a space for newcomers as well as more advanced participants to engage|
|Engagement||Dyadic; sometimes a more passive learning style||Typically a passive approach to discussing teaching, such as a workshop or preconference event||Highly interactive; collaborative and active learning approaches|
|Trust among Participants||Trust is difficult to develop through one-off workshops and trainings||Trust is typically lacking, with few opportunities to development meaningful connections||Trust frequently developed through intimate settings, targeted interactions, and connections that are able to grow over time|
Designing for Engagement
These four CoTs offered a wide variety of ways to engage individuals, including workshops, newsletters, social media, presentations at disciplinary conferences, websites, resources, and publications. Two design facets were most influential in contributing to participants’ sense of engagement with the community: 1) the philosophy, which was most often epitomized through a signature event; and 2) relationships, which were formed through peer-to-peer learning, brainstorming with others engaged in similar STEM reform efforts, and opportunities for mentoring.
We captured our findings about how to design for engagement and outcomes in the model in Figure 6.1. The central and most important aspect for these communities was the philosophy, with all the other elements emerging out of this core. Resources, key events, and others are all important as they embody this philosophy. Thus, CoTs for STEM reform embody their philosophy of pedagogy in their activities, resources, communications, and all other activities. The next circle is personal interactions that progress from peer-to peer learning to more focused brainstorming to formal mentoring. We begin by describing the themes that emerged from the first phase of qualitative data collection, followed by general findings from the survey portion of the study.
Philosophy as Design
When we asked faculty what they thought was the best or the most meaningful aspect of the CoT in which they participated, they described the philosophy of the community, which they thought was best symbolized or actualized through the signature events of each community. However, they also noted that philosophy was embedded in most of the community activities and communication. Our data about the formation of these communities demonstrate how much leaders invested in carefully constructing a compelling philosophy, typically articulated through a seminal document, posted on their websites, and reiterated throughout their materials. Each community then spent significant time embedding the philosophy in the activities and work of the CoT. This speaks to the intentionality invested by leaders within these CoTs to craft a philosophy that would be pervasive to participant experiences. The leaders of these CoTs also talked about ways they personally believed in and worked to spread the philosophy. We present the ways in which each community enacts its philosophy below as the best evidence of this theme.
The POGIL Project
For the POGIL Project, as participants described workshops and the annual national meeting, they noted how the meetings utilize the principles of the specific the POGIL Project pedagogy (active learning through guided inquiry and process features, like teamwork), which means that meetings include very active sessions, are learner centered, and use process principles, such as assessment and group work. In addition to using the specific group-work process, each session at the POGIL Project’s meetings ends with the specific the POGIL Project approach to assessment, dubbed SII (Strengths, Improvement, Insights). As a result, the philosophy that undergirds the STEM reform promoted by the POGIL Project also is reflected in the way that members of the community interact with one another at events and through communications. Additionally, the physical spaces in which all of the meetings and events take place also reflect this philosophy. The POGIL Project organizes its events so that sessions are always in rooms with movable tables and chairs, boards and other working spaces, and technologies that enable more active learning.
SENCER’s philosophy, documented in the SENCER Ideals (see Appendix 4A), focuses on capacious questions, context over content, interdisciplinarity, and the connection of science to civic issues and relevant problems. Making meaning, pursuing intellectual curiosity, and fostering a democratic sense of openness are additional strong values held as a part of the philosophy of SENCER. We witnessed each of these characteristics active in the SENCER annual conference. Each of the speakers alluded to questions of meaning and important problems that need to be resolved through science, and leaders implored people often to reflect about the higher purposes of education. One participant described how the philosophy resonated and permeated activities, events, and communication: “They bring a fundamental epistemology that’s very different than a lot of groups of people that I know to all their work. They really have this much more phenomenological approach to teaching, which is kind of an awareness of the methods they’re using in teaching and just a willingness to think about values, and mission, and care, and love, and the things that actually make us more and more human. They can talk about things like truth, beauty, and goodness without it being seen as something polar to the very materialistic approaches of Western science.” Reflection on meaning and questioning was built into each of the sessions. The annual event took place at a picturesque university on the west coast, where there are a variety of gardens, atria, and benches—all areas for reflection. The events reflected the interdisciplinary philosophy by not only inviting faculty across disciplines, but also by using sessions consistently to discuss the need to work across disciplinary boundaries. Also, educators from non-traditional groups often considered outsiders in academe, whether informal educators (e.g., museum staff) or part-time instructors, are actively included in SENCER discussions, and participants from these groups noted “feeling empowered.”
For PKAL, the What Works documents capture their philosophy of active learning, experiential learning, and acknowledgment of diverse learners’ needs. The Summer Leadership Institute (SLI) epitomizes the focus of their philosophy on the importance of developing leaders to enhance STEM reform. Leadership using these principles is the mantra of PKAL. The principles related to What Works are seamlessly connected to the activities of the SLI. Over the years PKAL has conducted a series of signature events, assemblies, and workshops for their various sub-groups, with each of these events intentionally encapsulating the key principles of “what works” in STEM reform. Further, participants spoke about how the underlying philosophy of PKAL includes value of mentoring, relationships, and developing people and their careers. As a result, events always include groups of mentors to provide feedback, time to build relationships, directed time to meet with mentors, and a focused discussion of careers and ways to develop individuals. While most of the CoTs have some elements of relationship building and mentoring that we will describe in the next section, this was a particularly significant aspect of the philosophy and practices of PKAL.
BioQUEST’s signature summer workshop embodies its philosophy related to teaching as the process of problem-posing, problem-solving, and peer persuasion, known as the Three P’s. The workshop is an opportunity for faculty to be problem-solvers, creating new materials to help teach biology. Participants also engage in the role of problem-posing to understand the types of issues addressed through their problem-solving activities. In this setting, they engage in groups to do their work, which involves peer persuasion. Thus, the underlying Three P’s are utilized to shape the workshop structure and process. Participants described this approach as being on the cutting edge, future-oriented (i.e., oriented toward what will be needed in the future training of scientists), challenging, and creative. BioQUEST does not develop prepackaged materials for this work, but expects participants themselves to create new teaching materials. The persuasion aspect of this philosophy can also be seen in the almost conflictual dialogue that takes place as people debate ideas to move towards more solid pedagogical materials. For example, workshop participants were asked to present their emerging ideas to other faculty attending a co-located conference occurring at the same time. This forced them to persuade and respond to criticism from relative outsiders who were not as aware of the pedagogical approaches or philosophies. Like the POGIL Project, the work spaces for BioQUEST meetings are set up to be highly interactive, with a computer for each individual, movable tables and chairs, and small rooms for breakout sessions or group work. BioQUEST participants often leave the workshop without a finished product, but they have learned a process that will involve future problem-posing, problem-solving, and persuasion, which fits into this philosophy of STEM reform.
Personal Interactions: Peer-to-Peer Learning, Brainstorming with Others, and Mentoring
In addition to the pervasive philosophy itself, members mentioned interactions with others in the community as the most engaging component of participation in these CoTs. Personal connections/interactions were reported in the data over 200 times and were by far the longest report from our hyper-research codes. The main categories of interaction that were mentioned in interviews or observed were: 1. peer-to-peer learning; 2. opportunities to follow up and brainstorm about practice with peers and collaborators; and, 3. mentoring. All of these interactions progress toward more directed or involved interactions and provide energy for faculty to continue to reform in the face of departments and institutions that may not be supportive. One participant summed up the sentiment expressed by many as follows: “One of the best aspects was simply meeting people that were really impressive people that had lots to share. The sort of willingness to share. I would say my personal relationships or personal experiences with individuals in the network has been the best thing for me.”
Participants noted their own engagement with the overall model of learning from peers, and they highlighted the importance of this model for these communities. The peer-to-peer model was evidenced within all four CoTs through all of the events we attended and observed. Many people shared that the other opportunities for professional development, available through centers for teaching and learning or attendance at disciplinary conferences, often lack connection with peers teaching similar courses within similar disciplines. They felt that the information that they garnered through the CoT was more directly applicable because it came from peers attempting to do the exact same work in similar contexts. The CoTs also emphasize the importance of peer-to-peer learning in person and in relatively small settings, while large disciplinary conferences and workshops were mentioned as ineffective settings for developing real relationships. The processes of developing trust and of participants getting to know one another are critical if advocacy and mentorship are to play a role in STEM reform. Thus, people emphasized the importance of in-person opportunities to build relationships that eventually turned into more detailed brainstorming and mentoring experiences (described below).
The signature event of each CoT provides ample opportunity for meeting people. The settings for these events are often isolated, helping to ensure that participants spend time together. Lengthy introductions are included at the beginning of these events, and faculty typically work in pairs and teams to maximize interaction. There are typically planned social hours, meals, and field trips to ensure time to bond.8 Most events include late evening social time with wine, games, or both. Many also institute specific programs or policies to ensure that people develop relationships, such as a buddy system between a newcomer and an alumnus/a. At the end of each signature event, people are invited to continue contact, and they are told about other opportunities, such as future events, authoring resources, joining the listserv, and newsletters. The notion of using the community as a resource is activated in the signature events of the CoTs.
Brainstorming with Others
Faculty also spoke about connecting with people to brainstorm and obtain advice. One participant described leaving the first event attended through a CoT and understanding this important ongoing and developing relationship: “You were engaging with someone and making a connection [at the event], and chances are, if you actually did go back and implement that plan, you could contact those people for help.” Another participant described the role of communities over time: “A sounding board, advisors, mentors. Always a place to go with questions. Okay, it was a safe place to go when sometimes it wasn’t safe to be in my department with certain opinions.” Many participants came to their first event thinking this would be a one-time interaction, but left with a strong notion that they had a group of people that they could now contact for support in their teaching. These more informal connections often developed into more formal mentoring relationships over time.
Over time, individual phone calls and e-mails among participants turned into more formal mentoring relationships. This formal mentoring was integrated into various practices: for PKAL, into its leadership institute and regional networks, and for SENCER, into its fellows program and annual dinner. This opportunity for more formal mentoring relationships was mentioned as one of the most valuable aspects intentionally fostered through the communities. One participant talked about the roles of mentors: “Really having recruited and picked good people as mentors, and having them be available as resources or sounding boards was helpful, faculty always mention that. The mentoring was vital to my success.” Designing for engagement means attracting the right types of individuals, as well as creating a sense of accountability to give back to the community. Some of the communities even name these individuals; for example, in PKAL they were called the village elders. These 30–50 mentors understood that they were part of this sub community that had a mentoring responsibility, and they actively enacted this role for extended periods of time, some up to 30 years. One of the village elders described how gratifying it is to be a mentor: “I’m now mentoring probably eight or nine young people, and I keep in contact with them when they’ve got things going on, they brainstorm with me. And I find that reciprocal mentoring as just wonderful.” Later in the conversation she mentioned how the community itself helped mentors to see that they can also learn and be enriched through these interactions. Communities can help to foster these mentoring relationships in ways that make them mutually rewarding. These mentoring relationships turn into very tangible advice about departmental politics, grants, tenure and promotion, publications, and the like. Mentors often wrote letters for grant projects and tenure and promotion files.
Trends in Quantitative Analyses Related to Design
The survey findings reinforce the importance of philosophy and relationships in contributing to participants’ effectiveness in STEM reform work (See Table 6.2). These results are especially telling considering that the survey was administered to a much more representative sample of community members than were represented in the leadership interviews, suggesting the importance of philosophy and culture in these communities. We anticipated that the faculty participating in the survey would be less involved with the CoTs than those we interviewed, and that these elements of philosophy and personal interactions may be less important to them. However, when participants were asked to rate the importance of various design characteristics in contributing to their work, participants on average ranked items pertaining to philosophy and relationships as more important than other design principles, such as communication vehicles, mechanisms for feedback, sub-groups, grant activities, and events. Innovative and new ideas and community philosophy are the first and second most important design characteristics for participants. The innovative and new ideas disseminated by the CoTs in this study stem directly out of their philosophies, like guided inquiry, active learning, or case-based approaches. Participants indicated that the characteristics were also influential in contributing to institutional reform efforts.
Table 6.2: Importance of Design Principles for Individual and Institutional Effectiveness in STEM Educational Reforma
|Mean (SD)||Mean (SD)|
|Innovative and New Ideas||2.91 (0.78)||2.58 (0.86)|
|Community Philosophy||2.87 (0.86)||2.51 (0.91)|
|Opportunity to Connect with Other Faculty||2.79 (0.89)||2.43 (0.93)|
|Opportunity to Network with STEM Leaders||2.69 (0.94)||2.49 (0.95)|
|Community Culture||2.66 (0.89)||2.36 (0.88)|
|Safe Space||2.64 (0.96)||2.39 (0.95)|
|Inclusive Practices||2.54 (0.94)||2.27 (0.93)|
|Opportunity to be Mentored||2.49 (1.02)||2.31 (0.94)|
|Community-Specific Resource||2.45 (0.94)||2.21 (0.91)|
|Community Leaders||2.45 (1.00)||2.21 (0.96)|
|Different Opportunities for Involvement||2.44 (0.92)||2.25 (0.90)|
|Seminal Documents||2.42 (0.94)||2.32 (0.93)|
|Opportunities for Early-Career||2.38 (1.05)||2.40 (0.96)|
|Opportunities for Mid-Career||2.37 (0.97)||2.32 (0.94)|
|Heterogeneity of People Involved||2.37 (0.97)||2.26 (0.97)|
|Presence at Disciplinary Meetings||2.27 (0.86)||2.06 (0.82)|
|Opportunities for Late-Career||2.26 (0.99)||2.19 (0.95)|
|Communication Strategies||2.26 (0.86)||2.12 (0.86)|
|Sub-Groups/Grant-Related Initiative||2.25 (1.02)||2.30 (0.99)|
|Local/regional Events||2.24 (0.94)||2.11 (0.89)|
|Annual Events||2.13 (0.95)||1.98 (0.84)|
|Mechanisms for Feedback||2.06 (0.88)||1.99 (0.87)|
|Community Staff||2.00 (0.99)||1.87 (0.92)|
|NOTE: aScale: 4-point scale: 1 = Not at all important; 2 = Somewhat important; 3 = Very important; 4 = Essential|
The other top-rated design characteristics by-and-large reference the importance of personal interactions in helping participants’ effectiveness in STEM reform. These characteristics include opportunities to connect with other faculty and STEM leaders, having a safe space, inclusive practices, and the opportunities to be mentored. It is clear that having the opportunity to connect with others in an environment that is safe and inclusive, like the environments we observed at the community events, carries through to this representative sample. The consistency across the interviews, observations, and survey data clearly indicate that philosophy and personal relationships and interactions are central components for aspiring communities to attend to when pursuing STEM reform.
In addition to the these analyses, we also utilized ordinary least squares regression models to examine the extent to which design characteristics of these communities, along with different types of engagement in these communities, were associated with our five outcome factors (learning & improving practice, skill development for leadership & change, networking, departmental change, and institutional change). Tables 6.3 and 6.4 display the coefficients for these variables in the models.9
Learning and Improving Practice
We begin first with the model for learning and improving practice. Several engagement variables are significantly, positively associated with participants perceiving that they learned and improved practice, after controlling for the other variables. These are (in order of effect size): indicating greater continuity of involvement, attending more national events, indicating a longer tenure of involvement with the community, and authoring materials for the community. Thus, individuals who have been involved in the communities longer and more continuously can be expected to exhibit greater benefits in terms of learning and improving practice, while also attending more events and authoring more materials. Participants placing greater importance on several design variables is also positively associated with the participants perceiving benefits related to learning and improving practice. Specifically, participants who have been exposed to and value community philosophy, community leaders, community culture, innovative and new ideas, and community resources report greater benefits related to learning.
Skills for Leadership and Change
Beginning with engagement variables, we find that attending national events, indicating more continuity of involvement, and having more peers from one’s home institution involved in the community are significantly, positively associated with this dependent variable. As for design characteristics, participants noting the importance of community leaders and community philosophy result in the two largest positive effects for this perceived benefit, followed by valuing innovative and new ideas.
Again, several engagement variables are significantly associated with networking after controlling for other variables in the model. Specifically, having more continuous involvement, attending national and regional/local events, and authoring materials for the community are positively associated with networking, while having more years of involvement in the community is negatively associated with networking.10 Turning to design variables, greater emphasis
placed on community leaders and community culture is positively associated with networking, as well as valuing
opportunities to connect both with other faculty and with STEM leaders.
Thus, the nature of faculty engagement in reform efforts matters, but effectiveness of engagement is also tied to important design characteristics. Our models suggest that several engagement and design variables are important across our outcomes. We measured a variety of possible engagement activities, including attendance at a variety of events, engagement in different community activities, membership in different groups within the community, and the nature and extent of involvement generally in the community, yet we found that continuous involvement and attendance at annual events were the most important engagement variables in our study. In addition to engagement, we found that culture and community leaders also play a role in contributing to these outcomes.
Reinforcing the Importance of Philosophy and Interactions
It makes sense that annual events, continuous involvement, and community culture, philosophy, and leaders were significant in our models, given our understanding of how these communities of transformation strategically try to impact STEM reform (Table 6.3). Our interviews with key leaders and our observations of these annual, signature events revealed the importance of these events in reinforcing the underlying culture and philosophy of these communities. We saw that community leaders and event organizers act intentionally to infuse the community philosophy into the content and organization of sessions, and we observed how the culture of each community is present at these events in the ways that members interact with one another and with the key leaders. In other words, community is genuinely fostered in these environments, and it is influenced by the philosophy communicated by leaders in both event content and organization. We also observed individuals who had attended many events engage in deeper engagement and relationship building at these events, behaviors that can be attributed to their continuous involvement and ability to remain connected to the community.
Deeper Engagement Matters for STEM Reform
The findings related to continuity of involvement, engaging multiple members from a campus in the community, and length of involvement suggest the importance of deeper engagement in contributing to the outcomes in our study (See Table 6.3). These variables speak to an active engagement, which signifies faculty engaging more deeply, for extended periods of time, and with their peers. These communities can foster such active engagement in order to influence faculty members and allow them to see benefits from their involvement in these communities. As we described earlier in this section, the ways in which these communities differ from traditional professional development opportunities can contribute to this deeper engagement.
Table 6.3 Engagement and Design Coefficients from OLS Regression Models for Three Individual Outcomes Related to STEM Education Reform
|Skills for Leadership & Change||Networking|
|Engagement and Design Variables|
|Characterized Extent of Involvement||.11***||.03||.08**||.03||.17***||.03|
|Attend: National Event||.09**||.03||.08**||.03||.08**||.03|
|Design: Community Leaders||.10***||03||.19***||03||.15***||.03|
|Design: Innovative & New Ideas||.09***||.03||.06*||.01||.01||.03|
|Design: Community Philosophy||.15***||.03||.12**||.04||.06||.03|
|Design: Community Culture||.09**||.03||.07||.04||.13***||.04|
|Activity: Author Material||.05*||.02||.02||.03||.05*||.02|
|Years Involved with Community||.08**||.03||.04||.03||-.09**||.03|
|Design: Community Resources||.08**||.02||.03||.03||-.01||.02|
|Number of Peers Involved with Community||.00||.02||.05*||.02||.04||.02|
|Attend: Regional/Local Event||.01||.02||.03||.02||.11***||.02|
|Design: Connection with Other Faculty||.01||.03||.00||.04||.11**||.03|
|Design: Connection with STEM Leaders||.04||.03||.03||.03||.09**||.03|
|Activity: Present at Community Event||.00||.03||.01||.03||.03||.03|
|Activity: Present Material at Prof. Meeting||.03||.02||.02||.03||.03||.03|
|Activity: Publish about Community Work||-.02||.02||.00||.02||.01||.02|
|Group: Leadership/Board Member||-.07||.08||.07||.09||.04||.08|
|Design: Diff erent Involvement Opportunities||.04||.03||.02||.03||.00||.03|
|Design: Safe, Supportive Space||.04||.03||.02||.03||.01||.03|
|Design: Inclusive Practices||.00||.03||.02||.04||-.01||.03|
|Design: Opportunity for Mentoring||.04||.03||.04||.03||.03||.03|
|NOTE: * p < .05; ** p < .01; *** p < .001; Variables were standardized prior to entering them in these models, which results in the coefficients representing effect sizes.|
Community Engagement and Design Matters more than Institutional, Professional, and Personal Characteristics
While we were able to observe significant effects in their relationships to our outcomes, the same was not true by-and-large for the other factors that influence faculty behavior in our models. While we noticed an occasional significant coefficient scattered throughout our models, in general a participant’s institutional type, discipline, and rank/appointment status were not significantly associated with the outcomes we measured, when accounting for our other engagement, design, and motivation variables.11 We think this points to the potential for CoTs resembling those in our study to contribute to overcoming some of the typical barriers to reform mentioned in the literature, including institutional reward structures or policies that value research productivity over improving teaching, disciplinary cultures that place more emphasis on research, and lack of institutional leadership toward a culture that values teaching (Austin, 2011; Henderson et al., 2011). While it is true that institutions have complex policies and structures that are difficult to change and address—such as those related to tenure, promotion, and rank—and disciplines exert control over how faculty may engage in STEM reform, we observed no meaningful differences across these indicators in our models. For the most part, regardless of where faculty work, their position, or their discipline, they reported greater benefits related to teaching, leadership, and networking through engaging with the important aspects of these communities, and they expressed motivation to attain these benefits. While we can only speculate on the future impact of these communities, their efforts could very well contribute to the bottom-up change that scholars in the field identify as possible through faculty engagement (e.g., Sunal et al., 2001). We turn now to the organizational outcomes.
Organizational Outcomes Related to STEM Reform
After controlling for institutional, professional, and personal characteristics, we identified several participant involvement and CoT design variables that are significantly associated with departmental change (See Table 6.4). Three aspects of involvement and activities are positively and significantly associated with participants reporting departmental change due to their involvement in the CoTs: length of involvement, peer involvement, and presenting materials related to community involvement at professional/disciplinary meetings and conferences. In addition to these involvement variables, the salience of three community design variables is also positively and significantly associated with departmental changes—community culture, community leaders, and innovative and new ideas of the community. We observed that community involvement and design shared many of the same relationships with institutional change as it did with departmental change. Length of involvement, greater peer involvement, and presenting materials related to community involvement at professional/disciplinary meetings and conferences are all positively associated with institutional change, after controlling for institutional and faculty characteristics. However, not all involvement experiences have positive effects; for example, involvement in a project or grant-based group is negatively associated with participants reporting institutional change. Of our design variables, participants perceiving community culture and community leaders as important for their work is positively associated with institutional change.
Deeper Engagement Matters for STEM Reform
Prolonged involvement and presenting about the community to outside audiences were both associated positively with our outcomes. These two variables suggest that a deeper engagement in these CoTs can have benefits for individuals hoping to influence broader institutional goals for STEM reform, in addition to the individual benefits highlighted above (See Table 6.4). In order to present about a community’s pedagogical or other reform strategy to outside audiences, a faculty member must engage long enough and deeply enough to feel comfortable with the material and be able to communicate the strategy’s nuances to audiences less familiar with the work. These types of presentation also communicate an individual’s expertise, which can translate to increased legitimacy and leadership on the individual’s home campus. This in turn can help these individuals to foster more change through their efforts. Additionally, the fact that prolonged involvement is positively associated with the reported institutional change outcomes reinforces that STEM reform is complex and takes time (Austin, 2011; Henderson et al., 2011). Designers and participants of these and other future CoTs cannot labor under the false impression that brief engagement in these communities will lead to larger changes; our findings indicate that those who are involved for longer periods of time report greater change. This suggests that such efforts must persist long enough to engage faculty over longer periods of time to see these changes take hold.
Table 6.4 Engagement and Design Coefficients from OLS Regression Models for Departmental and Institutional Change Related to STEM Reform
|Departmental Change||Institutional Change|
|Engagement and Design Variables|
|Years Involved with Community||.15***||.04||.13***||.04|
|Number of Peers Involved with Community||.12***||.03||.19***||.03|
|Activity: Present Material at Prof. Meeting||.11**||.03||.10**||.03|
|Design: Community Leaders||.09*||.04||.12**||.04|
|Design: Community Culture||.13**||.05||.13**||.05|
|Design: Innovative & New Ideas||.09*||.04||.04||.04|
|Characterized Extent of Involvement||.05||.03||-.05||.04|
|Attend: National Event||.06||.03||.02||.03|
|Attend: Regional/Local Event||.05||.03||.06||.03|
|Activity: Present at Community Event||-.01||.04||.07||.04|
|Activity: Author Material||.04||.03||.01||.03|
|Activity: Publish about Community Work||-.04||.03||.01||.03|
|Group: Leadership/Board Member||-.01||.11||-.15||.10|
|Design: Different Involvement Opportunities||-.01||.04||.03||.04|
|Design: Community Resources||.03||.03||.02||.03|
|Design: Safe, Supportive Space||.03||.04||.06||.04|
|Design: Inclusive Practices||.01||.04||.01||.04|
|Design: Connection with Other Faculty||-.04||.05||-.03||.05|
|Design: Opportunity for Mentoring||-.01||.04||.04||.04|
|Design: Connection with STEM Leaders||.07||.04||-.02||.05|
|Design: Community Philosophy||.04||.05||.07||.05|
|NOTE: * p < .05; ** p < .01; *** p < .001; Variables were standardized prior to entering them in these models, which results in the coeffi cients representing eff ect sizes.|
Collective Effort for STEM Reform
Our other key engagement finding points to the idea that change is a collective effort, as faculty who reported having more peers involved in the same community with them reported greater change on both the departmental and
institutional levels (See Table 6.4). The communities in our study can serve as venues for multiple members of an institution to engage together in activities related to STEM reform, and the efficacy of such arrangements was reinforced by qualitative data. We observed this as a successful strategy in particular when CoTs encouraged members to engage in their activities as campus teams. Team members met in the spaces created by the CoTs, and they participated in conversations about bringing strategy back to their home campuses and about how they might work together when they returned. The qualitative research revealed, however, that such collective action does not only occur due to intentional focus by CoTs on gathering together institutional teams. Interviews with active community members revealed that there was a positive effect associated with simply having peers at their home institutions who had experienced the same workshops or developed through the same community. These commonalities acted to give participants a sense of shared language and trust on their campuses, even when they had not participated concurrently together with the other individuals. As a result, we conclude that organizational learning within an institution is enhanced by having multiple members of that institution participate in the same community. Such contributions to institutional reform are often cited as benefits of involvement in communities of practice (Allee, 2000), and the CoT model seems to offer this insight into how it can be augmented.
Community Design for Departmental and Institutional Change
We also identified several design characteristics of communities of transformation that are positively associated with departmental and institutional change (See Table 6.4). Specifically, the presence of key leaders and community culture are positively related to both institutional and departmental change. Based on our observations and interviews, we are not surprised to see the data bear out this association. First, leaders in the community play important roles of influence for community members. The four communities in our study were all founded by visionary individuals who continuously recruit a cadre of leaders to be active in community operations and generally visible at community events. These community leaders exemplify effective leadership styles, which serve as models for community members for how to lead change efforts when they return to their campuses. This allows for diffusion of these practices from non-organizationally situated communities, such as the CoTs studied, outward to multiple institutions. These leaders also set the tone for community gatherings and events in which faculty learned from one another in peer-to-peer settings, fostering cultures of active engagement, trust, and support among community members.
Cultures of Personal Support
These communities also exhibit cultures that value and place personal support at a premium; the presence of such cultures are associated with change outcomes on participating faculty members’ campuses. The culture that is fostered in CoTs through community events, resources, and communication materials is a supportive one in which faculty are provided valuable feedback and mentorship to help them seek changes on their home campuses. Faculty further experience this culture of personal support through the sharing of strategies and best practices, as they learn in a peer-to-peer environment (Kezar & Gehrke, 2015a). These findings related to culture should be understood in conjunction with the importance of key leaders, as those individuals not only set the direction and tone of the community, but serve as mentors and provide personal support for community members. As a result, leaders can provide guidance and even consult on departmental and institutional changes being sought by community members, thereby creating a sense of personal support that is both localized in mentorship relationships and also pervasive across many experiences and activities available through the CoT. The combined value reported to arise from these key leaders and community culture suggests that these factors should be key considerations for CoTs to influence broader change in STEM reform. Specifically, these types of communities will gain by being intentional about the ways that their leaders represent and model cultures of support across a range of engagements and activities.
In general, we identified several key strategies for STEM reformers to use in order to engage faculty to contribute to broader organizational change outcomes at their home institutions. Reformers interested in starting or designing CoTs to contribute to STEM reform should a) provide adequate support to keep faculty involved for longer periods of time; b) seek involvement from multiple individuals within single institutions in their communities; c) engage community members in activities and development to help them gain mastery over the material in order to communicate their work beyond the community; and d) identify key leaders who can both support faculty and foster a culture of engagement. By using these strategies, we found that these efforts can effectively engage faculty in the kinds of bottom-up change efforts advocated for in the literature and shown to correlate with broader institutional and departmental outcomes for STEM reform. In section 7, we shift attention to the broader shape of these communities over time; in particular, we describe their lifecycles in order to highlight how effective communities form and continue to evolve.
9. The beta coefficients indicate the strength of the expected relationship for each variable to the dependent variables. Each coefficient represents the portion of a standard deviation increase in the dependent variable for a standard deviation increase in each variable. For example, a participant who scores one standard deviation above the mean for extent of involvement is expected to score .11 standard deviations higher on the dependent variable “Learning & Improving Practice.” These tables show only the coefficients pertaining to the design and engagement characteristics. We controlled for a variety of other factors in the models, including institutional characteristics (e.g., control, size, Carnegie classification), professional characteristics (e.g., position classification, discipline, motivation for involvement), and personal demographics (e.g., gender, race/ethnicity).
10. This seems at first paradoxical, as one might assume that the longer exposure people have to a community the more connections they can make and the greater their networks can grow. We think this finding suggests that the benefit of community involvement in growing one’s professional or personal network likely comes through in early years of involvement, as individuals who are looking for support in STEM reform are likely reaching out to others and growing their support network for engaging in such work. Longer involvement is positively associated with other outcomes, suggesting that as individuals are involved longer in the community, their goals for support likely shift from needing others to gathering knowledge and skills for personal growth and institutional change.
11. In this report, we only highlight relevant findings related to the communities. For more detailed analyses including institutional type, discipline, and rank/appointment status, please see Gehrke & Kezar, 2015.