Introduction

The COVID-19 pandemic compelled schools and other organizations engaged in learning, including museums, to widen their access to digital resources. With museums eager to reach audiences with digital resources and programs, pre-existing and novel challenges of the “digital divide” arose. Disparities in skills and usage of digital resources increased the divide among different categories of income, employment, education, age, and ethnicity usually associated with the ownership of hardware and adequate internet access (Van Dijk and Hacker 2003; Warschauer 2003; Lai and Widmar 2020; Callanan 2021). When considering educators, an exploration of elements that affected their experiences with digital learning also uncovered uncertainties in confidence and open questions about factors that contribute to motivations. By recently surveying more than 1,900 educators nationwide before and after the COVID-19-forced transition to remote schooling, Callanan described significant differences in teachers’ confidence in using digital media, especially among those in public schools, rural areas, and Title I schools, with only 66% of teachers reported being very or extremely confident in using digital media services for teaching after the pandemic-prompted shift to remote learning (Callanan 2021). It’s worth recalling that only a few years ago, scholars were debating if teachers needed to be partially proficient in online teaching, if online learning led to effective instructional practices, and how to best implement online teaching for educators and students to succeed (Lockee 2021). As educators will need to demonstrate their effectiveness in both in-person and online instruction, research is revealing a number of factors that may contribute to motivating them to use or engage with the latter. Among these factors, Callanan and other authors stressed that one of the key motivators for educators to adopt new resources within digital learning was the ability to achieve student learning outcomes (Sugar, Crawley, and Fine 2004; OECD 2015; Callanan 2021). More studies are therefore needed to evaluate the contribution of digital resources to student learning outcomes. While museums advocate for the use of museum learning resources as a means to contribute to student learning outcomes, few data are available to demonstrate their effectiveness to influence educators’ motivations to adopt them, especially when digital resources are used in classroom settings (Hooper –Greenhill 2004; Monaco, Lu, and Wood 2009; Green, Kisida, and Bowen 2013; Zinger, Tate, and Warschauer 2017). Furthermore, research has shown that a lack of adequate metadata (keywords describing resources affecting discoverability and use) associated with digital museum resources decreases their effectiveness (Foulonneau and Riley 2008; Abbott and Cohen 2015; Smithsonian Center for Learning and Digital Access with the School of Education at the University of California, Irvine 2018; Ozden 2019).

The “digital divide” reinforced that educators need additional skills to address different aspects of digital literacy, and several scholars have highlighted the need for variegated and continuous professional development, instructional tools, and support to aid educators in accessing and integrating resources into their practices (Centre for Educational Research and Innovation 2007; Elliott 2017; Callanan 2021; Tran and Gupta 2021). Frameworks were adopted regarding the use of digital technology in the classroom with the skills needed for effective learning. Among them is the TPACK framework, which provides a map for understanding how to integrate technology into the classroom effectively and emphasizes the relationships between technology knowledge (TK), pedagogy knowledge (PK), and content knowledge (CK), and that the purposeful blending of them is key (Mishra and Koehler 2006). By differentiating among these three types of knowledge, the TPACK framework outlines how the content being taught and how the teacher imparts that content (pedagogy) must form the foundation for any effective education technology integration. It also acknowledges that educators are acting within this complex space and has provided useful models for the design of PD experiences (Zinger et al. 2017; Eichelberger and Leong 2019).

This “digital divide” coupled with museums’ eagerness to reach and better serve educators forced them to experiment with new resource development and PD approaches to uncover how best to engage teachers through digital platforms (Baker 2008; Marty 2008; Eagleton et al. 2021; ICOM 2021; Europeana 2021; LessonUp 2021; Gilman 2021). Among others, the European Commission recently presented a digital education action plan that clearly articulated challenges for educators and actionable steps to address them (European Union 2021).

The Smithsonian Learning Lab

Like other museums, libraries, and archives, the Smithsonian Institution (SI) makes its holdings available to the public through digital technology. These holdings include millions of objects, specimens, library volumes, and thousands of cubic feet of archival materials, with less than 1% on display at any given time. To make these resources usable for education, the Smithsonian Office of Educational Technology (OET), formerly known as the Smithsonian Center for Learning and Digital Access, created the Smithsonian Learning Lab (SLL), a research-based digital learning platform focused on a core audience of educators and aligned to their needs and the ways in which they use digital resources in the classroom. Through the SLL, users can discover millions of digitized SI resources: images (e.g., specimens, artworks, and artifacts), recordings, and texts, each with descriptive information (metadata). Users can also create their own collections by choosing and organizing resources, and even upload their own content or that from other providers. To customize their collections based on their students’ needs, educators can use tools to add text, annotations, discussion questions, and quizzes. In addition, they may choose to share what they create through social media, by embedding collections on other websites, or by publishing publicly on the SLL. As of August 2021, users have used the Lab’s tools to create over forty-two thousand collections, and publish more than nine thousand of them, providing new examples for using Smithsonian resources for learning.

To address the need to provide educators with skills to use the SLL, and with extensive experience in implementing educator PD prior to the launch of the SLL, the OET has organized different kinds of PD for educators focusing on the use of the SLL and digital museum resources in classrooms. These offerings include facilitated single-day workshops, year-long programs using design-based research approaches, office hours available to users seeking tips and feedback, and tutorials embedded in the SLL (Zinger et al. 2017). Educators’ PD is defined here as a wide variety of specialized training, formal education, or advanced professional learning intended to help administrators, teachers, and other educators improve their professional knowledge, competence, skill, and effectiveness (Great Schools Partnership 2013). From November 2015 to August 2021, through its in-person and digitally archived workshops, the OET has engaged more than 20,000 educators in the use of the SLL. Over the course of five years, the constant, intentional interaction between the OET, SLL users, and PD participants in need-centred, TPACK-based PD provided valuable insights into the platform development. Originally created through an iterative process that built upon the ways teachers were actually using digital tools in their classrooms, the SLL was modified over time to address the users’ needs and the areas that seemed to be consistent barriers for use. Gradually, the platform evolved to include freely accessible digital sessions for users on the use of the platform and its resources with dedicated staff members, new tools for interactivity and engagement, and updates to the interface to support successful searches. Soon after launching the SLL, in partnership with the School of Education at the University of California, Irvine, the OET conducted research on how educators found digital resources in the Lab, what they did with them, what support they needed, and how the use of digital content affected their students (Smithsonian Center for Learning and Digital Access with the School of Education at the University of California, Irvine 2018). The research uncovered educators’ different patterns and preferences in searching and using digital museum resources and the SLL. It also revealed additional teachers’ needs, which ranged from skills to search for and adapt museum digital resources to their needs to modelled and easy-to-adapt pedagogical strategies and frameworks with digital museum resources to use in classrooms to achieve successful results. Another finding was that museum resources often lacked information that teachers expected and needed. In particular, the study demonstrated the impact of metadata on the use of resources and user-created collections on the SLL. Furthermore, the study indicated the value of educator need-centred PD, which specifically addresses the above-mentioned and adjusts to newly emerging needs (definition adopted throughout this article). By building on this research, this paper provides further evidence that participation in educator need-centred PD, in addition to increasing participants’ awareness of and skills in using digital museum resources in their classrooms, was associated with different patterns in using the SLL, such as creating, using, and sharing content, and increased overall satisfaction with the platform. In addition, while the question of the association between the use of digital museum resources and improvement in student learning outcomes is still open, this paper shows that educators, especially those who participated in PD, agreed with statements about successfully achieving student learning outcomes when using the SLL in classrooms.

Methodology

This paper discusses aggregated data collected through several independent studies that included surveys, interviews, and focus groups with users of the SLL, in-person PD workshop participants, and program partners from 2017 through 2021 (see Table 1). For this article, data were triangulated after collection and analyses of individual studies. Although one of the advantages of this triangulation lies in the ability to find agreement of results through a variety of research methods, limitations include the lack of a uniform methodology for applying triangulation and that it may not be easy to make dependable comparisons between two sets of data due to differences in the ways that the data were collected (Creswell 2009; Harvey 2022).

Table 1

Shows the list of the cited OET research studies.

Title and Description Participants
SLL Registered Users Survey 2020 Survey; n = 623; SLL registered users
Post-workshop Surveys
Survey collected after 36 in-person workshops from 2017 to 2019
Surveys; n = 543; teachers (86%, n = 469)
Teacher Creativity Studios: Asian Pacific American Cultural Presence in the Classroom (2017/2018)
Post-workshop survey with participants and interviews with program partners
Surveys; n = 33 (included in the previously reported post-workshop survey analysis); Teachers (85%, n = 28), School Administrators (9%, n = 3), Curriculum Specialists (6%, n = 2), and Students (6%, n = 2). Results do not equal 100%, as respondents could have selected multiple roles.
Interviews; n = 3; Program partners
Teacher Creativity Studios: Fostering Global Competence in the Classroom (2018/2019)
Post-workshop survey with participants and interviews with program partners
Surveys; n = 44 (included in the previously reported post-workshop survey analysis); Teachers (89%, n = 39), Other/parent (7%, n = 3), Librarian (5%, n = 2), Student (5%, n = 2), Curriculum Specialist (2%, n = 1), and Interested in a specific topic/subject (2%, n = 1)
Interviews; n = 3; Program partners
Teacher Creativity Studios: Latino/Latinx Cultural Presence in the Classroom (2018/2019)
Post-workshop survey with participants and interviews with program partners and workshop participants (for the latter, one year after workshop participation)
Surveys; n = 79 (included in the previously reported post-workshop survey analysis); Teachers (85%, n = 67), Other (15%, n = 12), Curriculum Specialist (9%, n = 7), Librarian, Parent, Student, and Interested in a specific topic/subject (4%, n = 3 for each group), Administrator, Principal, and Museum Staff person (1% each).
Interviews; n = 16; Program partners and workshop participants
Supporting the Innovative Teaching of High School Level Ethnic Studies Courses in Texas (2019/2020)
Post-workshop survey with participants, interviews with participants and program partners, and focus groups with program participants
Surveys; n = 29; curriculum specialists, program directors, and teachers
Interviews; n = 13; Workshop participants and program partners
Focus Groups; n = 3; Workshop participants, ranging from four to seven attendees
Supporting the Innovative Teaching of High School Level Ethnic Studies
Courses in Texas, Year 2 (2020/2021)
Post-workshop survey with participants, interviews with participants, and focus groups with program partners
Surveys; n = 23; teachers
Interviews; n = 15; Workshop participants
Focus groups; n = 2; Program partners, ranging from four to six attendees

While the SLL is free to use, users must register for an account to create customized collections of resources. In late spring of 2020, when most teachers began remote teaching online in the United States, a registered users survey (2020 Survey) was sent to around 44,000 collected email addresses representing approximately 50% of the total registered users at that time. After three rounds of distribution, we collected 623 responses (margins of error: 3.9% with 95% confidence interval, and 5.1% with 99% confidence interval). The data from the survey inform the discussion below regarding differences in the use of the SLL between users who did and did not attend PD.

From November 2015 to August 2021, the OET reached more than 20,000 educators through more than 160 PD sessions that were offered both in-person and digitally, synchronously and asynchronously, about the SLL. While the format of PD offerings, including workshops, may have varied over time and programs, this paper does not address these differences nor their possible contribution to the discussed results. The educator need-centred offerings mostly consisted of single-day workshops aimed at increasing awareness about digital museum resources and increasing skills in using these resources via the SLL, such as creating, copying, and adapting collections for teaching purposes. Workshops were consistently based on the TPACK model that provides a framework to address educators’ skills by highlighting the unique challenges raised by digital technologies and how their introduction complicates the already complex context of teaching. To continue adjusting to the educator’s needs, workshops were based on a design approach explained in Zinger, Naranjo, Gilbertson, and Warschauer (2017) in which, during each PD or each series of PD, participants, facilitators, and researchers worked together to inform the design for successive opportunities, and included pedagogical strategies to effectively use resources with students, such as the Harvard Project Zero Thinking Routines and the National Portrait Gallery’s Reading Portraiture guide, among others, with facilitators trained on these strategies (Tishman and Palmer 2006; National Portrait Gallery 2016). These pedagogical strategies are available and searchable on the SLL. Post-in-person workshop surveys were routinely collected with core questions about participants’ demographic characteristics, and agreeable statements about reaching PD learning outcomes. For this paper, we analyzed 543 aggregated post-workshop surveys collected mostly from teachers (n = 469, 86% of the respondents) after a variety of in-person PD programs from 2017 to 2019. Other research studies cited include the OET projects with local partners that typically targeted specific content (e.g., area-based studies such as Ethnic Studies, Asian Pacific American Studies, Latino Studies, and Mexican American Studies) (Smithsonian Learning Lab 2021). For these projects, the Smithsonian collaborated with local university, museum, and community organization partners within the network of Smithsonian Affiliations, a national outreach program that develops long-term partnerships with museums and other cultural and educational organizations. The partnerships were critical to recruit and train teachers both in-person and virtually on the use of the SLL. Around a month after these programs, we collected participants’ and partners’ feedback through in-depth interviews and focus groups. Lasting between 30 to 60 minutes each, in-depth, semi-structured interviews were conducted with the main aim to understand experiences, barriers, and lessons learned from specific projects (see Table 1). Face-to-face, telephone, and webcam focus group conversations were conducted with program participants and program partners in 2019 and 2021 for the programs, “Supporting the Innovative Teaching of High School Level Ethnic Studies Courses in Texas,” during 2019/2020 and “Supporting the Innovative Teaching of High School Level Ethnic Studies Courses in Texas, Year 2,” during 2020/2021 (see Table 1).

Descriptive quantitative analysis (means, percentages, etc.) and a thematic analysis of qualitative data—a method of analyzing qualitative data that entails searching across a data set to identify, analyze, and report repeated patterns—were implemented (Braun and Clarke 2006). Quotes from interviews were reported to represent larger themes that emerged from the qualitative analyses. All the data reported here were collected after approval by the Smithsonian Institution Internal Review Board.

Results

From its soft launch in November 2015 through August 31, 2021, the SLL has had more than 20 million page views with more than 3.6 million sessions, with an average time per session of 3 minutes and 24 seconds (ranging from 0–10 seconds to 1801+ seconds) (a Google Analytics session [known as “hits”] represents each individual site visit by a user, without a 30-minute period of inactivity). So far, users have created more than 42,100 collections and published more than 9,200 of them on the SLL (see Figure 1).

Figure 1
Figure 1

Shows an image captured of a collection created and published by an educator on the SLL. Arrows point to:

  • # of Times Copied: The number of times this collection has been copied by other SLL users

  • # of Times Favourited: The number of times this collection has been favourited (saved) by other SLL users

  • Publisher & Creator: The publisher and creator of this collection

  • Collection Description: The publisher/creator-written description of this collection

  • Example of a Pedagogical Strategy Resource: This is an example of SLL pedagogical strategy resource (in this case, a Harvard Project Zero Thinking Routine)

  • Example of a Resource: This is an example of an image resource on the SLL

Who used the SLL

The 2020 Survey provided an understanding as to who the SLL registered users are, with the majority of the respondents self-identifying as educators in different capacities. In particular, slightly over half of the respondents identified themselves as teachers (51%, n = 319), with the rest, in descending order, self-identifying as museum staff members, librarian/media specialists, curriculum specialists, topic enthusiasts, parents, home school instructors, and school administrators (see Figure 2). Survey respondents taught a variety of grades from Pre-K to adult education as well as a variety of subject matters from art education to STEM subjects (see Figures 3 and 4).

Figure 2
Figure 2

Shows SLL users’ self-reported usage roles (from the SLL Registered Users Survey 2020).

Figure 3
Figure 3

Shows SLL users’ self-reported grades taught. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have selected multiple grades.)

Figure 4
Figure 4

Shows SLL users’ self-reported subjects taught. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have selected multiple subjects.)

Attending PD was associated with a difference in use of the SLL

The OET provides a variety of PD for educators on the use of the SLL. This paper does not address differences among formats of PD; however, data showed that close to half (42%, n = 263) of the 2020 Survey respondents had attended some form of PD ranging, in descending order, from a workshop facilitated by the Smithsonian, watching tutorial videos, online office hours offered by Smithsonian educators, and workshops facilitated by an entity outside of the Smithsonian (see Figure 5). Fifty-eight percent (n = 360) of the 2020 Survey respondents did not attend PD.

Figure 5
Figure 5

Shows how SLL users self-reported professional development attendance. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have attended multiple forms of professional development.)

Results from the 2020 Survey demonstrated that having attended PD was associated with different patterns of experiences with the SLL. When asked how they heard about the SLL, not surprisingly, respondents who had attended PD more frequently indicated hearing about the platform through Smithsonian in-person training, conference presentations, or professional association communications than respondents who had not attended PD. Respondents who had not attended PD more frequently indicated hearing about the platform through an Internet search, online forum or blog, and social media than those who attended PD (see Figure 6).

Figure 6
Figure 6

Shows how SLL users heard about the SLL. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have selected multiple subjects.)

Our results uncovered different patterns in past frequency and future intent to use the SLL with respondents. For example, respondents who had attended PD utilized the SLL more regularly with students during the academic year 2019–2020 than those who had not. Specifically, they had more frequently used the SLL on a weekly (8%) and monthly (30%) basis than respondents who had not attended (weekly = 6%; monthly = 26%, respectively). Also, participants who had attended PD were less likely to have used the SLL only once: respondents who had not attended PD more frequently used the SLL with students only once (24%) than those who attended (21%) (see Figure 7). As for the users who had attended PD who selected “Never,” as shown in Figure 7 (meaning they had not used the SLL with students during the academic year 2019–2020), a deeper analysis indicated that only one had assigned a collection to her/his students when asked later in the survey. These PD participants who had not used the SLL with their students had primarily searched for resources (71%) and viewed collections (69%). Many of these PD participants selected “Not applicable” to the statements regarding student learning outcomes found in Table 4. Over half (57%) of these PD participants also indicated that they did not share a collection. While further research should be conducted, it can be ascertained that these participants primarily use the SLL to search for resources and view collections rather than directly using the platform with their students. Furthermore, the 2020 Survey results showed that the respondents who had attended PD were more likely to indicate that they would continue using the SLL in the future than those who had not attended PD (see Figure 8).

Figure 7
Figure 7

Shows how SLL users self-reported use of the SLL with students during the academic year 2019-2020 (SLL Registered Users Survey 2020).

Figure 8
Figure 8

Shows SLL users’ self-reported future use of the SLL (SLL Registered Users Survey 2020).

Similar results were achieved through the surveys implemented immediately after PD workshops where data showed that the vast majority of the respondents strongly agreed and agreed to be more likely to use the SLL in their professional lives because of workshop participation (strongly agreed = 64% and agreed = 29%) (see Table 2). The respondents to these surveys also strongly agreed and agreed that the workshop participation increased their confidence with using the SLL (strongly agreed = 71% and agreed = 22%, respectively) and that they gained usable skills to apply in their professional lives (strongly agreed = 74% and agreed = 22%, respectively) (see Table 2).

Table 2

Shows how workshop participants self-reported impacts of PD. Survey respondents graded their agreement to different statements (Post-workshop Surveys).

Strongly Agree Agree Neutral Disagree Strongly Disagree
I am likely to use the Smithsonian Learning Lab in my professional life because of this workshop.
(n = 178)
64%
(n = 114)
29%
(n = 51)
3%
(n = 6)
3%
(n = 5)
1%
(n = 2)
This workshop increased my confidence with using the Smithsonian Learning Lab.
(n = 178)
71%
(n = 126)
22%
(n = 39)
4%
(n = 7)
1%
(n = 1)
3%
(n = 5)
This workshop increased my awareness of the Smithsonian Learning Lab and its capabilities.
(n = 178)
79%
(n = 141)
18%
(n = 33)
0%
(n = 0)
1%
(n = 1)
2%
(n = 3)
I gained usable skills and will be able to apply them to my professional life.
(n = 541)
74%
(n = 399)
22%
(n = 119)
3%
(n = 15)
1%
(n = 3)
1%
(n = 5)

An effective Internet search is considered one of the most important skills in digital literacy and the OET PD specifically addressed how to search and improve searching for resources on the SLL (Abbott and Cohen 2015) (see Figure 9). When asked to rate their experience with searching for a resource on the SLL, respondents who had attended PD rated it as very satisfactory or satisfactory more frequently than those who had not attended PD (see Figure 10). In addition, respondents who had attended PD searched for SLL collections more frequently than those who had not attended (79% vs 51%, respectively) (see Figure 11).

Figure 9
Figure 9

Shows an image captured of the search capabilities of the SLL. Arrows point at:

  • Learning Lab Collections: The tab that leads to collection search results.

  • Search Bar: The bar used to search for resources or collections.

  • Resources: The tab that leads to resource search results.

  • Media Type: The user can filter search resources by image, video, audio, text, and learning resources.

  • Resource Characteristics: The user can filter search resources by public domain and/or included transcription.

  • Resource Provider: The user can filter search resources by the Smithsonian Museum or Department (e.g., the National Museum of American Indian, Smithsonian Magazine, Smithsonian Center for Folklife and Cultural Heritage, etc.).

  • Date Range: The user can filter search resources by date range.

  • Culture: The user can filter search resources by culture (e.g., Native Americans, Indians of North America, Vai [African People], etc.).

  • Place: The user can filter search resources by place (e.g., USA, Africa, Liberia, etc.).

Figure 10
Figure 10

Shows how SLL users self-reported experience searching for resources on the SLL (SLL Registered Users Survey 2020).

Figure 11
Figure 11

Shows how SLL users self-reported use of the SLL. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have indicated multiple answers.)

Different patterns also manifested when addressing experiences with SLL collections. One of the primary features of the SLL is the ability for users to customize digital resources. For instance, users can add new annotations to resources, edit existing annotations, change visible resource metadata, create collections of resources, or copy published collections and use them as is or adapt them to their needs, as previously described. The OET PD specifically addresses how to master these technical functions. The 2020 Survey respondents who had attended PD created and copied collections more frequently than those who had not attended (49% versus 11% and 24% versus 14%, respectively) (see Figure 11).

In addition, our post-workshop survey showed that participants agreed that the PD made them feel more comfortable in their ability to create and adapt SLL collections (see Table 3). Respondents who had attended PD also used collections more frequently for direct instruction and were slightly more likely to assign them to students than those who had not attended PD (36% versus 21% and 17% versus 14%, respectively) (see Figure 11).

Table 3

Shows how workshop participants self-reported impacts of PD. Survey respondents graded their agreement to different statements (Post-workshop Surveys).

Strongly Agree Agree Neutral Disagree Strongly Disagree
I find it valuable to have this museum’s digital resources available to modify in the Smithsonian Learning Lab.
(n = 121)
67%
(n = 81)
26%
(n = 31)
5%
(n = 6)
2%
(n = 2)
1%
(n = 1)
After this workshop, I am comfortable in my ability to adapt a Smithsonian Learning Lab collection.
(n = 122)
42%
(n = 51)
42%
(n = 51)
9%
(n = 11)
5%
(n = 6)
2%
(n = 3)
After this workshop, I am comfortable in my ability to create a Smithsonian Learning Lab collection.
(n = 123)
47%
(n = 58)
36%
(n = 44)
10%
(n = 12)
4%
(n = 5)
3%
(n = 4)
Table 4

Shows how SLL users, if they had used the SLL in teaching or curriculum writing, graded their agreement to different statements related to student learning outcomes. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have not responded to statements.) Definitions of terms within statements (i.e., authoritative resources; observation skills; technology skills; discipline-based thinking; critical thinking skills; project-based learning; and collaborations) were intentionally not provided to the survey respondents as they will be subjects of future studies on the impact of museum digital resources on student learning outcomes.

Strongly Agree Agree Undecided Disagree Strongly Disagree
I wanted students to access authoritative resources.
(n = 532)
42%
(n = 225)
17%
(n = 90)
1%
(n = 6)
1%
(n = 2)
3%
(n = 15)
My students increased observation skills.
(n = 532)
27%
(n = 141)
22%
(n = 118)
6%
(n = 32)
1%
(n = 1)
2%
(n = 8)
My students increased technology skills.
(n = 532)
13%
(n = 70)
22%
(n = 116)
12%
(n = 65)
4%
(n = 19)
1%
(n = 7)
My students increased discipline-based thinking skills (e.g., historical thinking, scientific inquiry process).
(n = 532)
24%
(n = 126)
25%
(n = 134)
7%
(n = 38)
1%
(n = 3)
1%
(n = 5)
My students increased critical thinking skills.
(n = 532)
28%
(n = 147)
23%
(n = 121)
6%
(n = 30)
1%
(n = 6)
1%
(n = 5)
My students engaged in project-based learning.
(n = 532)
23%
(n = 120)
20%
(n = 107)
8%
(n = 44)
2%
(n = 12)
1%
(n = 4)
My students engaged in collaborations. (n = 532) 19%
(n = 102)
19%
(n = 99)
9%
(n = 47)
3%
(n = 15)
1%
(n = 7)

The SLL offers users the opportunity to share their collections by publishing them on the platform; this allows other users to find these collections and then copy, adapt, and use them as needed. By recognizing the difference between the amount of created and published collections on the SLL, we tried to understand if PD impacted the publication of users’ collections. We also investigated if users employed different means to share collections not captured by our metrics. The 2020 Survey showed that respondents who had attended PD were more likely to share collections than those who had not attended PD (63% versus 43%, respectively) and, in particular, to publish collections (27% versus 5%, respectively) (see Figure 12) Furthermore, the survey demonstrated that SLL users indeed used different means to share their collections (e.g., by copying the collection URL and sharing it by email, or social media, or by an app on their mobile phone, or by embedding the collection in a different learning platform) (see Figure 12).

Figure 12
Figure 12

Shows how SLL users self-reported means to share SLL collections. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have used multiple means of sharing.)

Survey respondents who had not attended PD were more likely not aware of the possibility to publish a collection directly on the SLL and unsure of the Smithsonian’s expectations for publishing than those who attended PD (23% versus 10%, respectively) (see Figure 13). Interestingly, those who attended PD were more likely not to share the collections as they wanted to work more on them than those who did not attend PD (22% versus 10%, respectively) (see Figure 13).

Figure 13
Figure 13

Shows SLL users’ self-reported reasons for not publishing collections on the SLL. (SLL Registered Users Survey 2020. Results do not equal 100%, as respondents could have indicated multiple reasons.)

PD participants achieved learning outcomes

The data proved that PD achieved our intended learning outcomes for participants. Data from the Post-workshop Surveys showed that a large percentage of individuals agreed or strongly agreed to statements regarding awareness, confidence, and use of the SLL (see Table 2). When these survey participants were asked to indicate their agreement with the statement, “I gained usable skills and will be able to apply them to my professional life,” 96% (n = 518) agreed or strongly agreed. Respondents to the 2020 Survey participants viewed the SLL as a resource for ideas, primary sources, a basis for educational activities, and students’ assignments. Data from interviews reinforced results from the surveys with PD participants. For example, interviewees described the SLL as follows:

Especially in public education, with funding not available for things like going to museums, it’s an incredible way to get the museum to them. That is something within the last 10 years that many don’t experience at all and for some, this is the way they’re going to see it (Teacher Creativity Studios: Latinx Cultural Presence in the Classroom, Year 2).

It’s possible to do a really deep dive on something and you never run out of places to look. You would look at a Native American image and find more collections, so it’s a great trail. It’s a collaborative effort. A way to get a whole bunch of different perspectives of the same image in different collections and lenses and fitting into different context[s] (Teacher Creativity Studios: Latinx Cultural Presence in the Classroom, Year 2).

They also used it for their personal learning, to satisfy curiosity, and for personal knowledge/continuing education. For instance, when talking about the use of the SLL, a respondent from the 2020 Survey wrote:

For my own learning, I’ve looked at other people’s collections. For example, others presented at workshops and I later looked back at their collections for further understanding. I then did some of my own research using resources/images they had shared in their collections.

And another remarked, “I enjoy the Smithsonian Learning Lab resources for my own continuing education.”

Even partners recommended the use of the Smithsonian Learning Lab in the classroom, with one saying:

You get to curate your own museum by using the SLL, you get to make your own digital museum with your own purpose. If your purpose is social justice, your purpose is to revise dominant narratives that have been told about a particular context or history, then you can be intentional in curating primary documents and you can supersede the intentionality of these different museums. [Teachers aren’t] beholden to a director, board of directors. There are people that control these things and they want to control a narrative. What it does for teachers, is it allows them, in a moment, to be able to control the narrative and I think that that’s something that, if I can use marketing to publicize the SLL to teachers, that is something that’s really useful. This is a moment where you get to rewrite the dominant narratives. You get to use museum stuff and you get to tell the story and you get to remedy propaganda (Teacher Creativity Studios: Latinx Cultural Presence in the Classroom, Year 2).

Participants who attended PD were more likely to be very satisfied with the SLL than those who had not (see Figure 14). In addition to this, those who did not attend PD were more likely to be neither satisfied nor dissatisfied.

Figure 14
Figure 14

Shows how SLL users self-reported satisfaction with the SLL (SLL Registered Users Survey 2020).

Educators, especially those who participated in PD, agreed with statements about successfully achieving student learning outcomes when using SLL in classrooms

A major interest in our study was to shed further light on the use of digital museum resources to achieve student learning outcomes in classroom settings. As researchers are working to identify realistically achievable student learning outcomes when using museum digital resources in classrooms, initial data from ten in-depth interviews uncovered that teachers spoke of students’ increased engagement and demonstrating a higher level of thinking and application of learning when using the SLL. They talked about students’ increased curiosity and how students asked more questions. They described increased student autonomy and inspiration for future learning and felt that the use of the SLL improved students’ ability to make connections and think more deeply. These preliminary findings informed the design of questions about students’ outcomes in the 2020 Survey meant to further explore the subject. Respondents who used the SLL in their teaching or curriculum writing, especially those who had attended PD, agreed that students increased observation skills, technology skills, discipline-based thinking skills, and critical thinking skills (see Table 4). They also reported that students engaged in project-based learning and collaboration.

To reinforce these results, when talking about the use of the SLL, responders from the 2020 Survey wrote, “[I used the SLL to] engage university students in examining images from within your collections, learn about metadata, develop critical thinking skills in relation to their own work with primary source research,” while another remarked, “I used the lab to create activities to help improve student observational and critical thinking skills in science.” A third articulated, “ [I used the SLL to] research and reference for students when looking at systems thinking and cultural implications for sorting and cataloguing. Also evaluating collections for their usefulness for specific research.”

Aspects of impactful professional development

The triangulation of the results of the 2020 Survey with the Post-workshop Surveys and the interviews from specific PD projects added nuances about the importance of incorporating strategies with museum resources in classroom instruction and of tailoring PD to the participants’ specific needs with a learner-centred approach. We uncovered possible explanations of the different patterns between educators who had attended PD and those who had not attended. When discussing these results, it is important to keep in mind the earlier reported limitations of the triangulation. As suggested by findings from the OET’s previous research, workshops about the use of the SLL and digital museum resources in classrooms incorporated pedagogical strategies such as the Harvard Project Zero Thinking Routines and the National Portrait Gallery’s Reading Portraiture guide, among others. The importance of having and discussing examples of using the SLL in classroom settings and the effectiveness of matching digital resources with pedagogical strategies were among the major themes discussed by workshop participants. By talking with teachers who participated in PD projects targeting specific content about the value of using these strategies with their students, we uncovered emerging themes of perception of how the strategies led to greater engagement, increased participation, and a higher level of thinking and learning. For instance, an interviewed PD participant said, “I’m working on a collection where I will use two photographs and ask students to observe and then compare. I will use the Project Zero resource to develop my questions.” A Survey 2020 responder wrote, “I use individual images and the questioning strategies I learned during that workshop on a regular basis when examining visual text.” Workshop participants found pedagogical strategies on the SLL relevant to their subject areas. They agreed that dedicated time to explore suggested museum resources to pair with pedagogical strategies was important or very important and that, as a result of participating in the workshop, they found themselves integrating different teaching strategies when building curricula for or teaching their subject matter (see Table 5).

Table 5

Shows that PD attendees valued pedagogical strategies by indicating their agreement to the several statements (Supporting the Innovative Teaching of High School Level Ethnic Studies Courses in Texas).

Strongly Agree Agree Neutral Disagree Strongly Disagree
I found pedagogical strategies on the Smithsonian Learning Lab that are
relevant to Ethnic Studies.
(n = 21)
17%
(n = 5)
40%
(n = 12)
7%
(n = 2)
7%
(n = 2)
0%
(n = 0)
Because of my participation in the workshop, I found myself suggesting
different teaching strategies to teachers for Ethnic Studies.
(n = 22)
17%
(n = 5)
30%
(n = 9)
20%
(n = 6)
7%
(n = 2)
0%
(n = 0)
Because of my participation in the workshop, I found myself integrating
different teaching strategies when building a curriculum for or teaching
Ethnic Studies.
(n = 22)
20%
(n = 6)
30%
(n = 9)
20%
(n = 6)
3%
(n = 1)
0%
(n = 0)

Our results also pointed to the importance of cultivating long-term, supportive relationships with PD participants who expressed the need for PD offered over multiple, sustained, and variegated sessions. We learned that crucial components of a sustainable and supportive PD were cultivating long-term relationships with program partners (that assist in recruitment, development, and instruction of PD) and the human interaction among the programs’ multiple participants. Project partners were highly complimentary of the staff they collaborated with in planning and implementing the workshops. The following quote by a staff member from a partnering Institution represents the theme:

I am engaged because I am now more interested in using our collections in a different way, not just to study objects but because I need to go and look and search for other objects that we have that I am not aware of or that I don’t see that often because they are in storage. Instead of having them in storage all the time, if I could have access to them and create a digital collection then at least make them available even if you can’t see them physically (Teacher Creativity Studios: Latinx Cultural Presence in the Classroom).

In particular, project partners appreciated the timely, frequent, and helpful communications that led directly to well-run workshops. They also appreciated the OET staff’s flexibility and intention to support the local facilitators to design a workshop appropriate to their audience. The voices of the program participants reiterated the importance of a sustained communication plan with clearly articulated expectations and deadlines and regular updates on new resources and features of the SLL. Clear and consistent communication and assistance throughout the process made the experience a positive one for the organizing partners. This highlights the value of having service-oriented staff organizing PD projects and could suggest that each iteration of the PD will become easier for the partners as they have more examples and experience working on similar projects. Members of a partnering Institution spoke about the OET staff involved in the program, saying:

They were very clear in their communications and expectations. They met all of their deliverables in terms of providing things we needed or things that needed to be completed. I like this group because they are not only planners and visioners, but also do-ers. They are not in an ivory tower, not above setting up and breaking down on [the] day of. They were good listeners (Teacher Creativity Studios: Latinx Cultural Presence in the Classroom).

Even during the shutdown it didn’t feel like any lag in our communication. [OET staff member] made sure to provide us the materials we needed in advance. Before the shutdown happened, she went ahead and shipped everything we would need for the workshop. I have no complaints and all good things to say (Teacher Creativity Studios: Latinx Cultural Presence in the Classroom).

Discussion and lessons learned

Educators represent the majority of the SLL users and this article presents evidence that attending user need-centred PD is associated with different patterns of using the platform and its various functions. Since our results suggest several lessons, we’ll focus on four major concrete findings that could prove useful to institutions that are embarking on providing access to digital resources to educators and making sure that they can be used.

User-centred marketing research and strategies are important

We observed that different users heard about the SLL through different means. In particular, those who had not attended PD more frequently indicated hearing about the platform through an Internet search, online forum or blog, and social media than users who attended PD, while the latter indicated different forms of PD as the preferred means to hear about the platform. As means of advertising are considered crucial elements of marketing campaigns, especially if integrated with pathways to access digital resources and patterns of use, our observation reinforces paying particular attention to user-centred marketing strategies to expand the reach of museum resources. Several scholars suggested that user-centred marketing research and strategies are not always clearly established when distance education programs begin and should be considered from the very onset of the development of learning platforms and programs. These scholars, among others, pointed to how marketing has become an increasingly important skill set to be included in teams embarking on addressing access to and developing digital resources (Stellefson et al. 2008; Yamamoto 2009). Stellefson, Eddy, Chaney, and Chaney, in addition, provided several examples and tips for effective marketing strategies, ranging, among many, from developing a user-centred mindset branding to establishing a customer relationship management system to collect users’ interests and provide pertinent direct information users may need, to optimizing the functionality of distance education websites to market programs by strategically choosing “meta tags” (i.e., title, keywords, descriptions) designated for the website.

Attending an educator need-centred PD that addresses digital literacy skills is associated with increased confidence, skills, and satisfaction using digital museum resources

A recent literature review on factors affecting primary teachers’ use of digital technology pointed out that teachers’ attitudes toward the use of digital technology were found to be related to teachers’ confidence, beliefs, and self-efficacy (Spiteri and Chang Rundgren 2020). Our findings showed that PD attendees reported feeling more aware of, more comfortable with using, and more satisfied with the platform’s functions than non-PD attendees, which may have led to more frequent use and, correspondingly, to increased confidence as described by other authors (Pongsakdi, Kortelainen, and Veermans 2021). Furthermore, our results showed that attending PD that targeted skills customized to the users’ needs on the use of the digital platform was associated with different uses. For this paper, we focused on searching, creating, copying, and sharing digital resources. As many scholars consider search to be a key element of digital literacy, and our initial studies found that searching required both discrete skills and accumulating experience in when and how to use them, the OET PD specifically addressed how to search for a particular resource or collection on the SLL. Survey respondents who had attended PD more frequently rated their search experiences as very satisfactory or satisfactory than those who had not attended PD.

Educators can be confronted with a plethora of difficult-to-use digital resources leaving them with the need to quickly sort out resources for usability and/or adapt them to their students’ needs. Digital learning platforms now often offer the possibility to adapt resources, but literature on digital literacy, while emphasizing the importance of educators’ skill of flexibility, has not frequently addressed how educators adapt digital resources (American Association of Colleges of Teacher Education and the Partnership for 21st Century Skills 2010; Kelentrić, Helland, and Arstorp 2020). Our findings show that attendees of PD more frequently created their collections and copied and adapted others’ collections. These results point to the need for further research on how resources are adapted, what set of skills educators need to adapt resources to achieve effective instruction and students’ learning outcomes, and to assess this effectiveness.

Additional skills within digital literacy are a collaboration among peers and sharing content and practices that contribute to cooperative learning models. Among other scholars, McCann indicated that communities of practice may foster best practices, knowledge sharing, and innovation to the field, as well as assistance to those educators who lack access to such resources and technology (McCann 2009). From its conception, the SLL offered the possibility for users to share their collections by publishing them on the platform so that they become available to all users and discoverable through search engines based on the quality of the metadata added (Smithsonian Center for Learning and Digital Access with the School of Education at the University of California, Irvine 2018). Over time, we noticed a difference in the number of created and published collections. Initial findings demonstrated that users who attended PD were more likely to publish collections on the SLL as they were more aware of the function. This study confirmed the initial finding and uncovered uncertainty of the SI expectations in publishing on the platform as a barrier. Our data revealed other means to share collections such as by copying the collection URL and sharing it by email or social media or by an app on a mobile phone, or by embedding the collection in a different learning platform. This lesson can prove particularly useful when addressing educators’ needs of intercommunicating digital learning platforms, as school districts are rapidly moving in adopting them, and structuring and assessing marketing strategies (Rhode et al. 2017).

Educators, especially those who participated in PD, agreed with statements about successfully achieving student learning outcomes when using SLL in classrooms

As literature evidenced that teachers seek educational technology because it “can have considerable positive impacts on student performance,” museums are eager to demonstrate that their digital learning resources contribute to achieving student learning outcomes, especially when they are used in classroom settings (Terada 2020). Our findings showed that educators, particularly those who participated in PD, agreed with statements about successfully achieving student learning outcomes when using the SLL in classrooms. Student learning outcomes varied from increased observation skills, technology skills, discipline-based thinking skills, and critical thinking skills. SLL users also reported that students engaged in project-based learning and collaboration. Widely accepted by the museum world and considered among the outcomes that promote lifelong learning, documenting these skill achievements contributes to advancing the field of digital learning, especially if further assessed (Savva 2013). Our findings begin to articulate achievable learning outcomes when using museum digital resources in the classroom and prepare for further research needed to advance the field of digital learning and promote the use of digital museum resources among educators. Even though it is not the focus of this paper, a particular mention is dedicated to addressing pedagogical strategies to effectively use digital museum resources in classrooms. Following its initial findings, the OET introduced pedagogical strategies as resources in the SLL, and now constantly addresses their use through PD opportunities. Our preliminary findings indicated that PD workshop participants found the pedagogical strategies pertinent to their subject matter on the platform and integrated them into classroom practices. The use of research-based pedagogical strategies to achieve student learning outcomes has been amply reported in the literature and our study points to the need for further studies to specifically link the use of digital museum resources paired with pedagogical strategies to student learning outcomes.

Long-term relationships with constituencies and consistently accessible support from museum staff are key components of digital learning ecosystems

Additionally, we would like to consider the importance of long-term relationships with constituencies, and consistently available support with museum staff, within the limits of our methodology. In an increasingly digital world in which many organizations are striving to develop effective education ecosystems, our findings reinforce the importance of the role of human interaction in building strong learning communities (European Union 2021). Our qualitative findings showed how partners and PD participants appreciated the OET staff’s competence, knowledge, flexibility, and availability to accommodate their needs. A clear communication plan and strategy and the human interaction among the programs’ multiple participants were considered crucial elements to ensure a sustainable and supportive PD by partners and PD participants. Prior research on online versus face-to-face instruction demonstrated that positive outcomes were comparable across both formats, and e-learning courses in workplace and organizational settings have begun to include interaction and dialogue among participants to facilitate sharing their personal experiences and establish personal connections (Githens 2007; Russell et al. 2009; Kwaske and McLennan 2021). Our qualitative research confirmed how PD participants not only appreciated the personal connections with PD partners and the OET staff, but also with their peers. Aligned with the existing literature, our findings showed that educators need and ask for a PD offered over multiple, sustained, and varied sessions (Office of Education Technology 2017). To accommodate this need, the OET now regularly offers freely accessible digital sessions on the use of the platform and its resources with dedicated staff members. This “just in time” form of support complements other forms of PD, is accessible to all SLL users, and allows participants and staff members to share experiences and tips to leverage digital resources and tools in meaningful and impactful ways.

Future directions

The data presented through this article pave the way to further research on the effectiveness of specific formats of PD in increasing educators’ confidence and skills related to digital literacy (e.g., office hours, embedded videos, or different workshop formats like progressive sessions over a period of time) and on the length of the duration of these effects after completing PD. While more studies are needed on the impact of museum digital resources on student learning outcomes, our results can provide a foundation to begin the identification of achievable outcomes. The future of digital access to Smithsonian digital resources and tools intended for teachers should continue to be developed iteratively in direct consultation with teachers. The insights summarized in this paper capture responses to observed and recorded needs, but we understand too that from COVID and beyond, the classroom and the needs of those who inhabit it will continue to change and expand. Our approaches to the tools and services we provide, and the PD that informs and supports it, will need to keep pace with that reality. “Online Schools Are Here to Stay, Even After the Pandemic” voiced The New York Times on April 11, 2021, and, as we all work to meet the increasing demand to engage educators digitally, these practical lessons learned can be adapted and applied to provide access to digital resources for the classroom (Singer 2021).

Abbreviations

TPACK Technology knowledge, Pedagogy knowledge, and Content knowledge

OET Smithsonian Office of Educational Technology

SLL Smithsonian Learning Lab

PD Professional Development

STEM Science, Technology, Engineering, Math

Acknowledgements

Author acknowledgements

The authors would like to express their appreciation to Stephanie Norby, former Director of the Smithsonian Center for Learning and Digital Access (SCLDA), for directing the creation of the Smithsonian Learning Lab, to Melissa Wadman, former Manager of Program Evaluation at SCLDA, and Zihao Quin, former intern at SCLDA, for the development and the analysis of some of the projects here reported, respectively. A special thank you is extended to Dr. Monique Chism and Rebecca Boutwell Kobberod, Under Secretary for Education and Head of Digital Transformation at the Smithsonian Institution, respectively, and Gemma Cantlow, Smithsonian Fellow, for their thoughtful feedback on the paper. We wish to express our gratitude to the Smithsonian educator community and Affiliate museum partners for their contributions to professional development offered at the OET. The list of these Affiliate museum partners includes (in alphabetical order):

  • City of Austin Parks and Recreation Department, in Austin, Texas, including:

    • The Asian American Resource Center

    • Emma S. Barrientos Mexican American Cultural Center

    • George Washington Carver Museum, Cultural and Genealogy Center

  • The Frost Art Museum, in Miami, Florida

  • The Irving Arts Center, in Irving, Texas

  • The Japanese American National Museum, in Los Angeles, California

  • LA Plaza de Cultura y Artes, in Los Angeles, California

  • The Senator John Heinz History Center, in Pittsburgh, Pennsylvania

  • The Tsongas Industrial History Center, Lowell National Historical Park, in Lowell, Massachusetts

  • The UTSA Institute of Texan Cultures, in San Antonio, Texas

  • The Western Reserve Historical Society, in Cleveland, Ohio

  • The Wing Luke Museum of the Asian Pacific American Experience, in Seattle, Washington

Last, but not least, the authors would like to acknowledge the support of the Smithsonian Asian Pacific American Initiatives Pool, the Smithsonian Latino Initiatives Pool, the Montgomery College–Smithsonian Faculty Fellowships Program, and the Grable Foundation for many of this article’s cited projects.

Guest Editor acknowledgements

The Editors-in-Chief of this collection wish to thank the Social Sciences and Humanities Research Council (SSHRC/CRSH) for its financial support.

Competing Interests

Christina Talbert completed paid consultancy work from the Office of Educational Technology as part of some of the data acquisition for this study. The other authors have no competing interests to declare.

Contributions

Authorial contributions

Authorship is alphabetical after the drafting author and principal technical lead. Author contributions, described using the CASRAI CredIT typology, are as follows:

Christina Talbert, Consultant to the Office of Educational Technology, Smithsonian, United States, ChristinaLTalbert@gmail.com – ct

Giuseppe Monaco, Office of the Under Secretary for Education, Smithsonian, United States, MonacoP@si.edu – gm

Darren Milligan, Office of Educational Technology, Smithsonian, United States, MilliganD@si.edu – dm

Ashley Naranjo, Office of the Under Secretary for Education, Smithsonian, United States, NaranjoA@si.edu – an

Philippa Rappoport, Office of Educational Technology, Smithsonian, United States, RappoPh@si.edu – pr

Tess Porter, Office of Educational Technology, Smithsonian, United States, PorterT@si.edu – tp

Authors are listed in descending order by significance of contribution. The corresponding author is Giuseppe Monaco, MonacoP@si.edu

Conceptualization: gm, ct, dm, pr, tp, an

Methodology: gm, an, pr, tp

Formal Analysis and Triangulation of the Data: gm, ct

Writing – Original Draft Preparation: ct, gm

Writing – Review and Editing: ct, gm, dm, an, pr, tp

Visualization: ct, dm

Supervision: ct, gm, dm

Project Administration: dm, an, pr, tp

Funding Acquisition: dm, an, pr, tp

Editorial contributions

Special collection editors

Co-Editor-in-Chief:

Marie-Claude Larouche, Full Professor, Department of Educational Sciences, and Co-director (2018-present) of the Research Lab on Culture and Its Publics (lrpc.ca), Université du Québec à Trois-Rivières, Canada

Co-Editor-in-Chief:

Hervé Guay, Full Professor and Director of the Department of Literature and Communication, and Co-director (2015–2021) of the Research Lab on Culture and its Publics (lrpc.ca), Université du Québec à Trois-Rivières, Canada

DSCN editorial team contributors

Section Editor:

Iftekhar Khalid, The Journal Incubator, University of Lethbridge, Canada

Text Editor:

Émilie Rouillard-Beauchesne, The Journal Incubator, Université de Montréal, Canada

Bibliography Editor:

Morgan Pearce, The Journal Incubator, University of Lethbridge, Canada

Layout Editor:

Chista Avram, The Journal Incubator, University of Lethbridge, Canada

References

Abbott, Franky, and Dan Cohen. 2015. Using Large Digital Collections in Education: Meeting the Needs of Teachers and Students. Digital Public Library of America. Accessed September 30, 2021. https://www.amaga.org.au/sites/default/files/uploaded-content/website-content/ACT_Branch/Events/2017/using-large-collections-in-education-dpla-paper-4-9-15-2.pdf.

American Association of Colleges of Teacher Education (AACTE) and the Partnership for 21st Century Skills. 2010. 21st Century Knowledge and Skills in Educator Preparation. London: Pearson. Accessed September 30, 2021. https://files.eric.ed.gov/fulltext/ED519336.pdf.

Baker, Judy. 2008. Introduction to Open Educational Resources. Connexions. Accessed September 30, 2021. https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/OpenResources.pdf.

Braun, Virginia, and Victoria Clarke. 2006. “Using Thematic Analysis in Psychology.” Qualitative Research in Psychology 3(2): 77–101. DOI:  http://doi.org/10.1191/1478088706qp063oa.

Callanan, Erin. 2021. “New Study: Teachers Also Suffering from Digital Divide in Pandemic: Remote Learning Challenged Educators, Exacerbated Inequities Among Students.” Accessed September 30. https://www.wgbh.org/foundation/press/new-study-teachers-also-suffering-from-digital-divide-in-pandemic-remote-learning-challenged-educators-exacerbated-inequities-among-students.

Centre for Educational Research and Innovation. 2007. Giving Knowledge for Free: The Emergence of Open Educational Resources. OECD Publishing. Accessed September 30, 2021. https://www.oecd.org/education/ceri/38654317.pdf.

Creswell, John W. 2009. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 3rd ed. Thousand Oaks, CA: Sage Publishing. Accessed September 30, 2021. https://www.ucg.ac.me/skladiste/blog_609332/objava_105202/fajlovi/Creswell.pdf.

Eagleton, Catherine, Kamila Oles, Maria Economou, Neil Curtis, Lisa Collinson, and Susannah Waters. 2021. “Turning a Pivot Into a ‘New Normal’? Online Teaching and Learning with Digitised Collections in Higher Education Contexts.” ICOM. Accessed September 30. https://icom.museum/en/news/online-learning-teaching-digitised-collections/.

Eichelberger, Ariana, and Peter Leong. 2019. “Using TPACK as a Framework to Study the Influence of College Faculty’s Beliefs on Online Teaching.” Educational Media International 56(2): 116–133, DOI:  http://doi.org/10.1080/09523987.2019.1614246.

Elliott, Joshua C. 2017. “The Evolution from Traditional to Online Professional Development: A Review.” Journal of Digital Learning in Teacher Education 33(3): 114–25. DOI:  http://doi.org/10.1080/21532974.2017.1305304.

European Union. 2021. Digital Education Action Plan. Accessed January 29, 2022. https://education.ec.europa.eu/document/digital-education-action-plan.

Europeana. 2021. Europeana Foundation. Accessed September 30. https://www.europeana.eu/en/about-us.

Foulonneau, Muriel, and Jean Riley. 2008. Metadata for Digital Resources: Implementation, Systems Design and Interoperability. Oxford: Chandos Publishing. Accessed September 30, 2021. https://www.sciencedirect.com/book/9781843343011/metadata-for-digital-resources. DOI:  http://doi.org/10.1533/9781780631257

Gilman, Amy. 2021. “The Era of the Visionary Museum Director Is Over … or It Should Be.” Hyperallergic. Accessed September 30. https://hyperallergic.com/665251/era-of-visionary-museum-director-is-over-or-it-should-be/?mc_cid=f79cc55d31&mc_eid=e2fa2ef79c.

Githens, Rod P. 2007. “Understanding Interpersonal Interaction in an Online Professional Development Course.” Human Resources Development Quarterly 18(2): 235–274. DOI:  http://doi.org/10.1002/hrdq.1202

Great Schools Partnership. 2013. “Professional Development.” The Glossary of Education Reform. Accessed September 30, 2021. https://www.edglossary.org/professional-development/.

Green, Jay P., Brian Kisida, and Daniel H. Bowen. 2013. “The Educational Value of Field Trips. Taking Students to an Art Museum Improves Critical Thinking Skills, and More.” Education Next 14(1). https://www.educationnext.org/the-educational-value-of-field-trips/.

Harvey, Lee. 2022. “Researching the Real World.” Quality Research International. Accessed January 25. https://www.qualityresearchinternational.com/methodology/RRW1pt15Triangulation.php.

Hooper-Greenhill, Eilean. 2004. “Measuring Learning Outcomes in Museums, Archives and Libraries: The Learning Impact Research Project (LIRP).” International Journal of Heritage Studies 10(2): 151–174. DOI:  http://doi.org/10.1080/13527250410001692877.

ICOM. 2021. Museums, Museum Professionals and Covid-19: Third Survey. ICOM. Accessed September 30. https://icom.museum/wp-content/uploads/2021/07/Museums-and-Covid-19_third-ICOM-report.pdf.

Kelentrić, Marijana, Karianne Helland, and Ann-Thérèse Arstorp. 2020. “Professional Digital Competence Framework for Teachers.” The Norwegian Centre for ICT in Education. Accessed September 30, 2021. https://www.researchgate.net/publication/321796285_Professional_Digital_Competence_Framework_for_Teachers_in_Norway.

Kwaske, Ilianna, and Kay McLennan. 2021. “Why Social Interaction Is Important in Online Learning.” Tulane University School of Professional Advancement. Accessed January 29, 2022. https://sopa.tulane.edu/blog/why-social-interaction-important-online-learning.

Lai, John, and Nicole Widmar. 2020. “Revisiting the Digital Divide in the COVID-19 Era.” Applied Economic Perspectives and Policy 43(1): 458–464. DOI:  http://doi.org/10.1002/aepp.13104.

LessonUp. 2021. Accessed September 30. https://www.lessonup.com/site/en.

Lockee, Barbara B. 2021. “Shifting Digital, Shifting Context: (Re)Considering Teacher Professional Development for Online and Blended Learning in the COVID-19 Era.” Educational Technology Research and Development 69(1): 17–20. DOI:  http://doi.org/10.1007/s11423-020-09836-8.

Marty, Paul F. 2008. “Museum Websites and Museum Visitors: Digital Museum Resources and Their Use.” Museum Management and Curatorship 23(1): 81–99. DOI:  http://doi.org/10.1080/09647770701865410.

McCann, Kimble. 2009. “Virtual Communities for Educators: An Overview of Supports and Best Practices.” In Proceedings of TCC, 137–142. TCCHawaii. Accessed September 30, 2021. https://www.learntechlib.org/p/43789/.

Mishra, Punya, and Matthew J. Koehler. 2006. “Technological Pedagogical Content Knowledge: A New Framework for Teacher Knowledge.” Teachers College Record 108(6): 1017–1054. Accessed March 9, 2022. https://asu.pure.elsevier.com/en/publications/technological-pedagogical-content-knowledge-a-framework-for-teach

Monaco, Giuseppe, Bo Lu, and Megan Wood. 2009. “Impact of the National History Day in Ohio Program on Students’ Performances: Pilot Evaluation Project.” The Journal of Museum Education 34(1): 79–96. Accessed September 30, 2021. http://www.jstor.org/stable/40479717. DOI:  http://doi.org/10.1080/10598650.2009.11510621

National Portrait Gallery. 2016. “‘Reading Portraiture’ Guide for Educators.” Smithsonian Institution. Accessed September 30, 2021. https://npg.si.edu/learn/classroom-resource/reading-portraiture-guide-educators.

OECD (Organisation for Economic Co-operation and Development). 2015. Students, Computers and Learning: Making the Connection. Paris: OECD Publishing. DOI:  http://doi.org/10.1787/9789264239555-en.

Office of Education Technology. 2017. Reimagining the Role of Technology in Education: 2017 National Education Technology Plan Update. U.S. Department of Education. Accessed September 30, 2021. https://tech.ed.gov/files/2017/01/NETP17.pdf.

Ozden, Demirkan. 2019. “Pre-Service Teachers’ Views about Digital Teaching Materials.” Educational Policy Analysis and Strategic Research 14(1): 40–60. Accessed September 30, 2021. https://files.eric.ed.gov/fulltext/EJ1212874.pdf.

Pongsakdi, Nonmanut, Arto Kortelainen, and Marjaana Veermans. 2021. “The Impact of Digital Pedagogy Training on In-Service Teachers’ Attitudes Towards Digital Technologies.” Education and Information Technologies 26: 5041–5054. DOI:  http://doi.org/10.1007/s10639-021-10439-w.

Rhode, Jason, Stephanie Richter, Peter Gowen, Tracy Miller, and Cameron Wills. 2017. “Understanding Faculty Use of the Learning Management System.” Online Learning 21(3): 68–86. Accessed September 30, 2021. https://files.eric.ed.gov/fulltext/EJ1154161.pdf. DOI:  http://doi.org/10.24059/olj.v21i3.1217

Russell, Michael, Rebecca Carey, Glenn Kleiman, and Joanne Douglas Venable. 2009. “Face-to-Face and Online Professional Development for Mathematics Teachers: A Comparative Study.” Journal of Asynchronous Learning Networks 13(2): 71–87. Accessed September 30, 2021. https://eric.ed.gov/?id=EJ862349.

Savva, Stefania. 2013. “Museum-Based Multiliteracies and Learning for 21st Century Skills: A Preliminary Study.” The International Journal of the Inclusive Museum 6(2): 117–130. Accessed September 30, 2021. https://www.researchgate.net/profile/Stefania-Savva/publication/258831293_Museum-based_Multiliteracies_and_Learning_for_21st_Century_Skills_a_preliminary_study/links/576f8b7c08ae842225a88970/Museum-based-Multiliteracies-and-Learning-for-21st-Century-Skills-a-preliminary-study.pdf. DOI:  http://doi.org/10.18848/1835-2014/CGP/v06i02/44444

Singer, Natasha. 2021. “Online Schools Are Here to Stay, Even After the Pandemic.” The New York Times. April 11. Accessed September 30, 2021. https://www.nytimes.com/2021/04/11/technology/remote-learning-online-school.html.

Smithsonian Center for Learning and Digital Access with the School of Education at the University of California, Irvine. 2018. Curation of Digital Museum Content: Teachers Discover, Create, and Share in the Smithsonian Learning Lab. Smithsonian Learning Lab. Accessed September 30, 2021. https://learninglab.si.edu/cabinet/file/fd2f72f5-89bb-4a01-aa0d-ddf17d0f8adf/Smithsonian-SCLDA_CurationofDigitalMuseumContent.pdf.

Smithsonian Learning Lab. 2021. “Learning Lab Research.” Accessed October 25. https://learninglab.si.edu/about/research.

Spiteri, Marthese, and Shu-Nu Chang Rundgren. 2020. “Literature Review on the Factors Affecting Primary Teachers’ Use of Digital Technology.” Tech Know Learn 25: 115–128. DOI:  http://doi.org/10.1007/s10758-018-9376-x.

Stellefson, Michael S., James M. Eddy, Don Chaney, and Elizabeth H. Chaney. 2008. “The Marketing of Distance Education.” International Electronic Journal of Health Education 11: 85–96. Accessed September 30, 2021. https://www.researchgate.net/publication/265425325_The_Marketing_of_Distance_Education.

Sugar, William, Frank Crawley, and Bethann Fine. 2004. “Examining Teachers’ Decisions to Adopt New Technology.” Educational Technology and Society 7(4): 201–213. Accessed March 9, 2022. https://www.researchgate.net/publication/220374696_Examining_Teachers’_Decisions_To_Adopt_New_Technology

Terada, Youki. 2020. “A Powerful Model for Understanding Good Tech Integration.” Edutopia. George Lucas Educational Foundation. Accessed September 30, 2021. https://www.edutopia.org/article/powerful-model-understanding-good-tech-integration.

Tishman, Shari, and Patricia Palmer. 2006. Final Report: Artful Thinking. Harvard Graduate School of Education. Accessed September 30, 2021. http://www.pz.harvard.edu/sites/default/files/ArtfulThinkingFinalReport-1.pdf.

Tran, Lynn Uyen, and Preeti Gupta. 2021. “Rebuilding Our Teams to Be Critically Conscious in Our Educational Work.” Journal of Science and Environmental Learning 1(1.6): 41–62.

Van Dijk, Jan, and Kenneth Hacker. 2003. “The Digital Divide as a Complex and Dynamic Phenomenon.” The Information Society 19(4): 315–326. DOI:  http://doi.org/10.1080/01972240309487.

Warschauer, Mark. 2003. “Demystifying the Digital Divide.” Scientific American 289(2), 42–47. https://www.scientificamerican.com/article/demystifying-the-digital/. DOI:  http://doi.org/10.1038/scientificamerican0803-42

Yamamoto, Gonca Telli. 2009. “E-Learning: Marketing in a Global Context.” In The Challenges for Marketing Distance Education in Online Environment: An Integrated Approach, edited by Ugur Demiray and N. Serdar Sever, 407–427. Eskisehir: Anadolu University. Accessed September 30, 2021. https://www.researchgate.net/publication/289251353_E-Learning_Marketing_In_A_Global_Context.

Zinger, Doron, Ashley Naranjo, Nicole Gilbertson, and Mark Warschauer. 2017. “A Design-Based Research Approach to Improving Professional Development and Teacher Knowledge: The Case of the Smithsonian Learning Lab.” Contemporary Issues in Technology and Teacher Education 17(3): 388–410. Accessed September 30, 2021. https://citejournal.org/volume-17/issue-3-17/general/a-design-based-research-approach-to-improving-professional-development-and-teacher-knowledge-the-case-of-the-smithsonian-learning-lab/.

Zinger, Doron, Tamara Tate, and Mark Warschauer. 2017. “Learning and Teaching with Technology: Technological Pedagogy and Teacher Practice.” In The SAGE Handbook of Research on Teacher Education, edited by D. Jean Clandinin and Jukka Husu, 577–593. London: SAGE Publishing. DOI:  http://doi.org/10.4135/9781526402042.n33