TSAG Pilot Study

Published on November 13, 2024

On November 1, 2024, the CTSA Team Science Affinity Group (TSAG) will conclude a pilot implementation study (under the direction of Dr. Betsy Rolland) of a suite of Team Science training and intervention modules. The goals were to assess the feasibility and usability of disseminating a central menu of five evidence-informed interventions to CTSA hubs across the nation. We are currently analyzing our data and excited to announce the forthcoming release of a number of publications related to our efforts. As soon as our results are available for public consumption, we will post them on this study website. Questions may be directed to [email protected].

Study Background

In early April 2023, thirteen (13) CTSA hubs applied to implement one or more of the following five modules:

Implementing CTSA Hubs: Duke University, Harvard University, Tufts University, New York University, Oregon Health & Science University, Pennsylvania State University, University of Cincinnati, University of Chicago, University of Kansas, University of Michigan, University of North Carolina-Chapel Hill, and University of Wisconsin-Madison.

TSAG Pilot Study Committee Members: Kristine Glauber (Duke), Beth LaPensee (Univ of Michigan), Victoria Lutgen Gronau (UTSW), Wayne McCormack (Univ of Florida), Heather Risser (Northwestern Univ.), Betsy Rolland (University of Michigan), Eli Samuels (Univ. of Michigan), Susan Simkins (Penn State), Whitney Sweeney (Univ of Wisconsin), Kevin Wooten (UTMB), Brenda Zierler (Univ. of Washington).

Study Overview

As evidence-informed interventions that draw on the Science of Team Science literature, these modules have been shown to improve the practice of team science in a least one local pilot environment. Our study focuses on how feasible and usable it is to disseminate them more broadly across the nation. Each module leader delivered a virtual training to all participating CTSA implementers. They provided a set of implementation materials (e.g., facilitator’s guide), and made themselves available following the training to provide modest help in adapting the intervention to a CTSA’s local context. They also shared a standardized evaluation module for CTSA implementers with a list of common metrics. At the conclusion of our pilot study, we are excited to share that multiple CTSA hubs have implemented our Team Science interventions, with participating hubs distributed across an array of different sizes and types of hubs. By using a set of common metrics to assess feasibility/usability, this study will provide us with a deeper understanding of the needs of translational teams and CTSA hubs around Team Science, as well as with insight on specific sub-populations (e.g., early-career researchers or research staff). Ultimately, our pilot study aims to offer CTSA hub leaders with an off-the-shelf toolkit of evidence-based interventions that facilitate the conduct of high-impact, interdisciplinary science.

FAQs

Who was eligible to participate?

Any CTSA hub.

How did implementation work?

CTSA implementers took part in a training led by module leaders, after which they will be responsible for setting up local systems for recruitment, implementation, and data collection. Following implementation, CTSA hubs submitted their evaluation data to their module leaders.

What kind of support will implementers receive?

While we did not have grant funds to help hubs implement our study, module leaders generously made themselves available to provide implementers with modest technical support. Some modules also developed communities of practice where CTSA implementers brought questions and shared resources

How does IRB work with this project?

The UW-Madison IRB Office certified our study as program evaluation and not human subjects research. Participating CTSA Hubs will sign a data use agreement to share evaluation data with UW-Madison.

I am interested, but where can I learn more?

We held an informational session for all interested CTSA hubs on February 22, which you can watch here.

More detailed information about each module can be found below.

Participating Modules

Collaboration Planning: University of Wisconsin-Madison

Module Description

Collaboration Planning is a 90-minute intervention for translational teams to help them think through their approach to collaboration, proactively addressing the areas that most frequently cause conflict in teams, including authorship, communications, and project management. Originally designed by team-science experts at the NIH and NSF drawing upon decades of research on teams and collaboration, a collaboration plan may be useful in the following ways:

 

  • To help your team create strong team processes from the beginning, which has been shown to result in improved scientific outcomes
  • As a template for writing a multi-PI plan
  • To address team-science-specific review criteria in future grant applications
  • Skill-building for research staff and early-stage investigators in team leadership and collaboration

UW–ICTR’s Collaboration Planning intervention has been developed and tested with more than 40 teams at UW–Madison. Over the course of this testing, we have honed the content to meet the needs of teams at various stages of development.

Module FAQs

What is the expected time commitment for the module training?

Collaboration Planning requires a three-hour training commitment for CTSA implementers.

What is the expected time commitment overall?

Collaboration Planning sessions run roughly 90 minutes per team, with some additional logistic/prep work.

What kind of experience is needed for CTSA implementers?

Facilitation experience and project management support.

What is the target audience? 

All teams.

This short (7-minute) video explains collaboration planning and its potential benefits to you and your team.

CTS Team Training: University of Florida

Module Description

The CTS Teams intervention is described in some detail in this publication, which is required reading for considering this training intervention:  McCormack WT, Levites Strekalova YA. 2021. CTS Teams: A New Model for Translational Team Training and Team Science Intervention. Journal of Clinical & Translational Science 5(1), E183  https://doi.org/10.1017/cts.2021.854

For the purpose of this TSAG training/intervention pilot study, it is hoped that partner programs will include both components of the CTS Team training model:  didactic (a one-semester Team Science course) and experiential (TL1/T32 support of cross-disciplinary teams [pairs] of trainees).  The CTS Team training model was developed and implemented at the University of Florida Clinical & Translational Institute by Wayne T. McCormack, PhD.  Since 2016, TL1 trainees have applied for training grant support with a research partner (PhD and or MD-PhD students) who is earning their PhD degree in a different discipline, graduate program, and college.  Students apply typically in their second year of PhD studies, before they defend their PhD dissertation proposals.  They propose “team specific aims” that involve collaborative research, require cross-disciplinary collaboration to accomplish, and become embedded in their respective PhD dissertation research projects.   Teams are typically supported for two years, beginning in the Fall semester.  We use a variation of this team training model in a UF T32-funded cancer biology training program with both predoctoral and postdoctoral trainees.

Didactic training in team science & collaboration skills is provided by a one-credit semester-long “Team Science” course (one 50-minute class session per week, 15 weeks).  The course syllabus may be viewed here:   https://mccormacklab.pathology.ufl.edu/teaching/gms-6945-team-science/.  Ideally partner programs would implement the entire CTS Team training model, however, we are open to the possibility of some programs only adapting the “Team Science” course (didactic training) to their program, to compare learning outcomes with the full program.  Resources available to support implementation of the course include PowerPoint files, recorded lectures, lecture transcripts, and (under development) facilitator guides for each session.  It is important to note that this course involves multiple homework assignments that trainees should work on in their teams and/or with consultation with other lab members and/or their mentors.  Homework assignments allow trainees to  immediately apply team science principles to their own research.

The experiential element of this program is the two-year TL1/T32-funded research experience.  In practice this experience actually extends beyond the two-year funding period until students complete their PhD degree.  Team specific aims are embedded into dissertation research projects, and are sometimes not completed within two years.  Outcomes from the first several cohorts include improved self-efficacy for clinical research skills, a significant increase in the frequency of participation in cross-disciplinary collaborative activities by both trainees and mentors, and intention by co-mentors (most of whom had not previously collaborated with each other) to continue collaborating beyond the period of training grant support.  CTS Teams provide a unique environment in which to study the dynamics of cross-disciplinary team mentoring.  Mentor/mentee training materials and team mentoring assessment tools are under development, and will be piloted as part of this project.

Module FAQs

What is the expected time commitment for the module training?

Two four-hour (half-day) workshops, one focusing on the didactic training (Team Science course) and the second focusing on the experiential training component (CTS Teams).

What is the expected time commitment overall?

Six months to two years.

What is the target audience? 

Predoctoral & postdoctoral TL1/T32 trainees.

This short (5-minute) video explains CTS Team Training and its potential benefits to you and your team.

Research Jams: University of Michigan

Module Description

Research Jams are facilitated sessions designed to assist groups in surfacing cutting-edge research ideas, building collaborations, creating shared research agendas, and developing strategic action plans. We have developed multiple Research Jams frameworks that have different purposes and intended outcomes. We are best positioned to disseminate our Ideation Jam – a framework ideal for groups who want to mobilize toward addressing a complex or ‘wicked’ research problem. A half day session, often with participants who are meeting for the first time, uses design thinking methods to move through phases of blue-sky thinking, pattern building, prioritization, mapping interest, and action planning. Ideation Jams foster a psychologically safe environment for co-creation and utilize a range of activities that promote individual ideation and collaborative exploration. We distill all Ideation Jam outputs into a report or virtual whiteboard to share with the group. We have hosted Ideation Jams with participants, including faculty, staff, and community partners, that bring a range of expertise and experience. While we will always gravitate toward an in-person experience, we have delivered many successful Ideation Jams in a virtual format, which are a great option when geographic distance presents a barrier to collaboration.

More information about Research Jams can be found here:

INTERVENTION DELIVERY: For CTSAs that have research groups that may benefit from an Ideation Jam, we can offer step-by-step support in its design and delivery. We will work with the point of contact for intervention delivery to 1) build a process timeline, 2) meet with the ‘faculty champion(s)’ of the effort to understand short- and long-term goals, 3) refine the Ideation Jam framework based on needs and desired outcomes of the faculty champion, 4) walk through all steps of the final Ideation Jam framework, 5) co-facilitate the first session, and 6) help with post-session activities (distribution of survey, creation of report, debrief with the faculty champion). We also have a large library of resources we can share, including process documents, timelines, recommendations for visuals, email communications, report templates, supply lists, participant surveys, whiteboard templates for virtual sessions, etc. Following delivery of the first Ideation Jam, our team can be available for guidance and advice with additional sessions moving forward; with enough interest, we would also be open to building a community of practice for ongoing learning and discussion.

Module FAQs

What is the expected time commitment for the module training?

3-4 hours.

What is the expected time commitment overall?

12-15 hours.

What kind of experience is needed for CTSA implementers?

Facilitation experience, project management support.

What is the target audience? 

New teams.

TeamMAPPS: University of Texas Medical Branch

Module Description

What Is It?

  • TeamMAPPS (Team Methods to Advance Processes and Performance in Science) is a behavioral skills training program specifically developed to increase the collaboration and effectiveness of scientific teams and scientific team members.

Who Developed TeamMAPPS?

  • Funded by the University of Texas Medical Branch at Galveston, well-published team science scholars (Dr. Eduardo Salas, Dr. Maritza Salazar, Dr. Theresa Lant, and Dr. Kevin Wooten) developed TeamMAPPS based on available evidence surrounding team effectiveness. The program was developed specifically for scientists and students to develop behavioral skills by learning mastery steps through Team Science cases and exercises, self-assessments, reflective exercises, and back-home application planning.

What Does It Cover?

  • TeamMAPPS currently has five specific modules, including an Introductory Module (Welcome, Overview to Team Science, and Pre-Assessment Scales and Exercises), three Core Modules (below), and a Synthesis Module (Summary of Course, Reflective Exercises, and Implementation Planning). Core Modules include:
  • Awareness and Exchange (Sharing Unique Information/Promotive Voice, Inquiring/Probing, Reframing and Integrating)
  • Psychological Safety (Perspective Seeking, Acknowledgement and Inclusion, Addressing Issues and Resolving Conflict)
  • Adaptation and Correction (Monitoring and Debriefing, Reflecting and Analyzing, Creating Change/Development Plans)

How Delivered/Facilitated?

  • TeamMAPPS can be delivered in three modalities across learner groups and teams differing in maturation:
  • Completely Online/Individual: web-based asynchronous delivery (across all learner groups)
  • Face to Face Classroom/Seminar: web-assisted using all modules or selected modules (e.g., Team Science workshop across all learner groups, graduate class, trainee or scholar seminar)
  • Whole Team Intervention: web-assisted using all modules or select modules for nascent or experienced scientific teams (e.g., study/proposal planning, team startup and development, change intervention)

Future Modules

TeamMAPPS will be providing future developments including:

  • Smartphone-Based Personal App
  • Integrating Community Members Module
  • Integrating Commercialization Members Module

For more information, contact Dr. Kevin Wooten at [email protected]

Module FAQs

What is the expected time commitment for the module training?

Ten hours to review Train the Trainer video and Trainer manual. Optional consultation with UTMB Trainers.

What is the expected time commitment overall?

Variable. Depends upon delivery (3 modalities). Average across modalities and across all modules is 8-10 hours.

What kind of experience is needed for CTSA implementers?

Facilitation skills, coaching skills, basic knowledge of team development and processes.

What is the target audience? 

Trainees, graduate students, postdocs, early, mid, and late career, nascent whole teams, mature whole teams.

Toolbox Dialogue Initiative: Michigan State University

Module Description

Developed by the Toolbox Dialogue Initiative (TDI), the Toolbox dialogue method is an evidence-informed facilitation approach that surfaces implicit assumptions and differing perspectives in complex, cross-disciplinary research projects for joint consideration and coordination. TDI has facilitated this process with more than 475 research teams and groups around the world, including work with the National Science Foundation, NASA, and multiple universities. For additional information about TDI see our website https://tdi.msu.edu 

What is a Toolbox Dialogue?

A Toolbox dialogue is a capacity building workshop that consists of an opening preamble to prepare the participants for the experience, followed by a 60 to 90-minute structured dialogue using a survey-like instrument called the “Toolbox” (Eigenbrode et al. 2007) and ending with a co-creation activity designed to leverage what is learned in the dialogue. The Toolbox instrument uses a series of statements or prompts to structure the dialogue by providing topics for discussion and inviting reaction using a 5-point response scale (disagree to agree), along with don’t know and not applicable choices, as would be found in a survey. The prompts are designed to stimulate dialogue (Rinkus et al. 2021), a joint activity involving interaction among participants that requires active engagement through listening, reflecting, exploring, elaborating, analyzing, questioning, negotiating, and feedback-seeking.

Participation in a Toolbox dialogue facilitates reflection and perspective taking behavior in teams enabling collaborators to enhance their mutual understanding (Hubbs et al. 2020), specifically about the often-implicit beliefs, values, and priorities that structure their contributions to research. In doing this, a Toolbox dialogue supports the creation of a culture of communication and collaboration within the team. The dialogue is then leveraged through a co-creation activity to produce a deliverable (e.g., project concept map, project glossary, collaboration plan, success metrics) that the teams can use throughout their collaboration.

Toolbox Dialogue Method Training for CTSA Hub Leaders

This 3-part training is designed to enable CTSA Hub leaders in facilitating Toolbox-style dialogue with research teams. Parts I and II will provide a theoretical, experiential, and practical introduction to the Toolbox Dialogue Method, with a focus on designing and facilitating Toolbox-style dialogue. Part III will involve follow-up coaching and be designed with input from participants to further develop their capacity to design and implement Toolbox-style dialogues, address challenges faced or anticipated, and provide instruction on how to use Toolbox dialogues to facilitate other interventions (e.g. collaboration planning).

All sessions will include time for participant introductions and networking, discussion and Q&A, and breaks. Participants will also receive reference materials from the training including a handout summarizing all content, presentation slides, sample Toolbox instruments, and other resources. Training elements will occur virtually using the Michigan State University Zoom Pro License. Participants are expected to have a stable internet connection that will allow for use of video, particularly during the dialogue portions of the workshop. This workshop will be highly interactive and require a stable internet connection for audio and video use.

Workshop Costs

Virtual Toolbox Dialogue Method Training Workshop Parts I, II & III (9 hours): $1,500 per participant

Module FAQs

What is the expected time commitment for the module training?

The Toolbox Dialogue Method Training requires a commitment of 6-9 hours.

What is the expected time commitment overall?

A full Toolbox Workshop for a team of researchers is 2-3 hours. However, a Toolbox Dialogue can range from 20-90 minutes, depending on the goal of the dialogue, and can be used to support many team activities including but not limited to: visioning, ideation, collaboration planning, periodic reflection, and even meetings!

What kind of experience is needed for CTSA implementers?

Facilitation experienced preferred, no specific expertise required.

What is the target audience? 

Teams (newly formed/forming, recently awarded or established).

This short (4-minute) video explains the Toolbox Dialogue Method