EdTech Evidence Exchange Platform: What does it take to collect technology experiences from 147 school districts? Hint, A LOT!

During the 2021-2022 school year, the EdTech Evidence Exchange research team documented over 1,500 educators’ firsthand technology experiences across 147 school districts. Their experiences power the EdTech Evidence Exchange Platform (fondly referred to as the Exchange Platform), an online data collection and display system that captures educators’ technology experiences.

Accommodating the Impact of COVID-19

COVID-19 continued to turn the education system on its head during the 2021-2022 school year, reaching far beyond classroom-based teaching and learning. It was an overwhelming year for anyone working in a school system. Staffing shortages pulled district and state leaders away from their day-to-day work to act as substitutes in classrooms; directors of transportation were driving bus routes; and students and educators alike experienced anxiety and confusion as they entered school buildings for the first time in months. 

When we introduced the edtech evidence available on the Exchange Platform to leaders, they quickly recognized the value for supporting their technology decision-making, particularly given the influx of technology to support emergency remote teaching. However, the hard-hitting implications of COVID-19 closures and staffing shortages demanded leaders’ attention. To break through the COVID-19 noise while remaining sensitive to educators’ needs, we got creative with how to help educators share their experiences. Over time, we modified our approaches to meet educators’ immediate decision-making needs and continually used lessons learned to refine our methods. 

Recently, we stepped back to analyze our learning and distilled 6 reflections to inform future data collection. 

Reflection 1: Build Networks and Authentic Connections 

Educator attention is at a premium. Given their incredible number of responsibilities and many requests for their time, educators rely on trusted colleagues and organizations to present the most valuable information and opportunities to support their work. With that in mind, we built relationships with educators’ trusted leaders and their membership organizations. We prioritized clear communication and transparency about the value of our edtech evidence to their work, and we expressed empathy for educators’ experiences. Then those leaders’ connected us to their educators to communicate the value of the evidence.

For the Exchange, this included:

• Forming a state-level advisory board to build trust and understanding of a variety of state networks (e.g., unions, membership organizations).
• Leveraging advisory board connections to develop authentic relationships with district leaders across the state.
• Articulating the value of using our edtech evidence for edtech decision-making to district leaders such that they felt comfortable endorsing the opportunity to school leaders and teachers. 
• Collaboratively developing a communication plan with district leaders to plan their outreach to school leaders and teachers (see Guideline 3).

Reflection 2: Connect District Needs with the Value of Exchange Evidence

Leaders often crave evidence to inform their decisions and to facilitate tasks such as presenting to school boards, justifying budgets, or communicating with local and state legislatures. We needed to understand the district’s profile and identify their edtech priorities to illustrate this value to leaders. With this information in hand, we drew concrete connections between their landscapes and the value that Exchange Platform evidence provides.

For the Exchange, this included:

• Opening initial district conversations with a discussion of districts’ edtech landscape and pain points.
• Identifying how Exchange Platform evidence can support their existing needs and transparently recognizing where evidence might not meet their needs. 
• Remaining flexible and prioritizing educators’ needs (e.g., reporting data in a variety of formats).
• Articulating how Exchange Platform evidence can support various reporting requirements.

Reflection 3: Co-develop a Custom Communication Plan

Tight schedules dominate educators’ work lives. Every minute of the day is accounted for to meet evolving student needs. Communication must be clear, succinct, and timely. By co-developing a custom communication plan with district and state (and possibly school) leaders, the plan can strategically include communication methods with which educators are familiar and check regularly. The plan should account for educators’ competing priorities and schedules at a time of their choosing. The plan helps leaders stay on top of communication, be prepared to answer teacher questions, and know when and what evidence they can access.

For the Exchange, this included:

• Identifying each school or district’s communication norms.
• Developing custom communication plans with district and school leaders that included scheduled announcements at regular intervals using communication methods familiar to the educators (e.g., emails, newsletter blurbs, one-page flyers, short videos, meeting announcements). 
• Saving leaders time by drafting communications for leaders to send to their educators.
• Tailoring communication drafts to reflect the leaders’ communication styles, using words and phrases familiar to their school or district.

Reflection 4: Emphasize Leaders’ Unique Perspectives

Leaders are used to creating opportunities for their teachers and don’t always realize the value in their insights. But, to completely understand edtech implementation in a specific context, it’s important to tap into leaders’ unique experiences with edtech. To encourage leaders’ participation, we highlighted their expertise and perspective as a key component of their school and district edtech implementation contexts. We also highlighted the value of the evidence to help their edtech decisions. For leaders, this was more effective participation motivation than extrinsic financial incentives. 

For the Exchange, this included:

• Emphasizing leaders’ unique experiences with edtech throughout planning conversations (e.g., leaders often have intimate knowledge of the edtech procurement process, plans for a district-wide edtech pilot, and reasons for replacing a technology).
• Reviewing leader versus teacher data collection tools and displays.
• Requiring leaders to complete the first Exchange Platform survey to access the evidence. (Note. This was very effective for encouraging leader participation.)

Reflection 5: Prioritize Educators’ Intrinsic Motivators

Educators often have a  strong intrinsic motivation to help others, including their colleagues. Educators enjoy collaborating with their peers, leaning on them for advice and recommendations when they have questions about the best way to meet their students’ needs. To motivate educators’ participation, we compensated them with a $20 e-gift card for each survey they completed. However, we also focused communication on the opportunity to share their expertise with others and improve the use of edtech for students. 

For the Exchange, this included:

• Describing educators as professionals with expertise and insights that are valuable for peers and leaders. 
• Describing brief use cases for the evidence educators contribute to the platform.
• Describing the value of evidence for their school or district. 
• Balancing the emphasis on the extrinsic financial rewards we offer (i.e., e-gift cards) with the intrinsic reward of improving edtech use.

Reflection 6: Get Ahead of Those Darn Firewalls 

This one is technical but critical! Ever-present concerns about cybersecurity lead district systems to install firewalls that protect their networks from viruses and scams. In some districts, firewalls automatically sent Exchange Platform emails to educators’ spam folders. Some systems prevented emails from even reaching the spam folders! So, after learning about the opportunity from several trusted sources (see Guideline 2), some educators never received our invitations to share their edtech experiences. To circumvent this, we proactively worked with district IT departments to ensure that their system would recognize our emails as a trusted sender. 

For the Exchange this included:

• Monitoring the backend of the Exchange Platform to check the record of opened and unopened emails. 
• Identifying the appropriate contact in district IT departments after establishing a partnership. 
• Troubleshooting with district IT departments to ensure their network recognizes and accepts our email sender name as a trusted source for emails to educators.

Thinking Ahead

The 2021-2022 school year was a year of growth and learning for the Exchange team. It wasn’t easy, but we took several big steps toward our vision of providing educators with the context-sensitive evidence they need to make edtech selection and implementation decisions. As educators managed competing priorities in ever-changing COVID-19-influenced contexts, we adjusted our processes to accommodate and remain sensitive to educators’ needs. We got creative and ultimately doubled our participation response rates from fall 2021 to spring 2022. 

We’ve learned a lot and will continue to iterate and reflect in the moment, refining our processes as we go. This will provide more educators with the opportunity to share their hard earned lessons with edtech and learn from their peers working in similar contexts.