Nations in Neighborhoods-  Folk Arts Instruction and Student Learning -AEMDD Final Data Report:

Dennie Palmer Wolf^1, Steven J. Holochwost^2, Tal Bar-Zemer^3, Anika Selhorst^3, Amanda Dargan³

 ¹ WolfBrown² Department of Psychology, Georgetown University & WolfBrown ³ City Lore

Abstract: 

The authors investigated the association between participation in a program of folk and traditional arts instruction and achievement in English language arts in a sample of low-income elementary school students, many of whom were recent immigrants and English language learners.  The program drew on the core practices of traditional and folk arts – socio-critical literacies that bridge home and school, multi-modal instruction, apprenticeship learning, and communal effort – to provide students with the confidence and strategies of accomplished learners. English language arts achievement was assessed using a standardized state proficiency exam. Students who participated in the program received significantly higher overall scores on the exam after controlling for gender, ethnicity, English language learner and special education classifications.  These findings suggest that an arts education program featuring folk and traditional arts may help students to engage as learners.

Collaboratively painting a Tibetan Thangka

Keywords: art education, folk arts, traditional arts, English language learners, English language arts, urban schools, at risk students

 Executive Summary

In the last quarter century, budget constraints and mandated testing have combined to force many schools across the country to curtail or eliminate arts education from their curricula (Keiper, Sandene, Persky, & Kuang, 2009). These pressures are particularly acute in large, urban school districts serving disproportionate numbers of low-income students, many of whom are recent immigrants or classified as English language learners. These students, placed at risk for educational failure, have inherent strengths – multiple languages, bi-cultural identities, and determination to succeed in new settings – that formal education rarely draws on (Moll, 2000; Guiéttierez & Rogoff, 2003). For them, the loss of arts instruction is the loss of a subject that builds bridges across cultures and languages, links concrete and conceptual learning, and provides a setting to acquire and demonstrate understanding, even as fluency in academic English is still developing. Moreover, where arts education persists, instruction in folk and traditional arts is rarely offered. Even in the most recently formulated national educational standards, traditional and folk arts are subsumed into a framework that continues to stress individual artistry, the tools, skills, and techniques of the formal arts practices, and the separation of arts from daily life (NCCAS, 2013). By contrast, in this paper, we present evidence that folk and traditional arts, with their emphasis on language that bridges home and school, multimodal forms of communication, apprenticeship learning, and communal effort, can serve a valuable role in the instruction of English language learners. We argue that values and practices fostered in folk and traditional arts help students to build on the knowledge and linguistic strengths that they bring and encourage educators to acknowledge and challenge those strengths (Jimerson, 2004).

practicing embroidery

NYC Community School District 28 has collaborated with the cultural organization, City Lore, to bring community folk and fine artists into a set of Queens public elementary and junior high schools to work with students and teachers to study the historical and cultural contexts in which the arts are made and to develop the research, discussion, and literacy skills associated with this kind of inquiry. In the project teaching artists are paired with classroom teachers and arts specialists to engage their students in oral history, community study, and the development of artistic expertise. Working with students, the teaching teams produce arts-integrated units of study that incorporate instruction in the traditional arts, student inquiry, and primary research, as well as individual and collaborative artistic expression, in order to increase students’ arts skills, language learning, and to foster students’ awareness of the many traditions that make up the multicultural landscape of New York City.

There are three major results from this project that can inform future work in the field.

• First, the findings from the Nations in Neighborhoods project provide evidence that folk and traditional arts, with their emphasis on language that bridges home and school, multimodal forms of communication, apprenticeship learning, and communal effort, can serve a valuable role in the strengthening literacy development of students attending urban public schools. In addition, the values and practices fostered in folk and traditional arts encourage educators to acknowledge and build upon the strengths that their students have acquired as they master the languages and dialects of their home, neighborhood, and school cultures.
• Second, the analyses presented here argue for the importance of acknowledging the nested nature of data from projects such as those sponsored by the AEMDD program:  students are nested within grades, and grades within schools.  Accounting for these relations is an important step in obtaining robust data even under circumstances that do not permit for randomized, control studies.
• Finally, the data reported here describe the increasing effectiveness of the Nations in Neighborhoods program throughout the multi-year course of the project.  Practices such as sustained participation by a cadre of teachers, discussions of student learning, and the collaborative scoring student work, may have played a role in focusing the project implementation. This suggests how important participatory research designs, in which teachers and teaching artists understand and contribute to the measurement, can be in assuring strong implementation.

Introduction

The Folk and Traditional Arts: Key Aspects of Pedagogical Practice

The American Folklore Society (2013) defines folklore as the traditional art, literature, knowledge, and practice that is disseminated largely through oral communication and behavioral example. As such, folk and traditional arts are rooted in and expressive of the cultural life and heritage of communities and groups who share a common identity. Often learned informally, these art forms are embedded in the daily lives and cultural traditions of families and communities and constitute an important part of the ‘funds of knowledge’ (Moll, Amanti, Neff & Gonzalez, 1992, p. 133) that many students bring to schools. As repositories of cultural values, symbolic meanings, shared history and aesthetic principles, they are a rich source of complex cultural meanings on which all students – independent of families’ education, language, wealth, or status – can draw as thinkers and creators (Campbell 2004; Green 2008; Dargan 2011; Dewhurst et al., 2013). For example, think of how many five year-olds – long before they read – are fluent in storytelling, riddles, teasing, and game rhymes. Far from being ‘just’ handicrafts or daily rituals, folk and traditional arts can introduce and teach complex concepts like tradition, diaspora, hybridity, and worldview (Campbell, 2004, p. 26).

Speaking of passing on the practices of Hawaiian lau hala weavers, Dewhurst writes that arts educators “are not just teaching students how to make the art at hand but also how to be in the world” (Dewhurst et al., 2013, p. 5). In addition, folk and traditional arts may provide an unusually transparent set of practices for developing an identity as a learner. Children entering school flourish if they can acquire the technologies of self that will allow them to observe, apprentice, take the risk of performing, learn to recover from errors, ask questions, compete, and eventually to mentor or model in this new setting. In short, to be successful students need more than raw knowledge, they need to acquire the concrete actions, strategies, and ways of behaving, interacting, talking, and learning necessary to enter different communities (Besley, 2005; Foucault, 1988)

Collaborative painting of Indian Mithila scrolls

This process is even more critical for young people who are constantly translating between the expectations of another culture and the expectations of schooling in the contemporary United States. Historically, this process has been subtractive, requiring students to discard or deny earlier identities and languages in favor of Americanization and English. In this context, folk and traditional arts may provide contexts in which students can add the technologies of self appropriate to Queens, Phoenix, or Cincinnati to those they bring from home or community. At its best, these practices result in cosmopolitan, culturally- and linguistically-nimble students who have learned how to inhabit an increasingly global world (Appiah, 2006; Dewhurst, 2013; Hansen, Burdick-Shepherd, Marrarano, & Obelleiro, 2009). But for folk and traditional arts to contribute to these valued outcomes, artist-teachers and their classroom partners have to introduce more than the topics or contents of these art forms. They must also convey the underlying values and practices of their field (Dargan, 2011) in ways that also draw on and respect students’ individual circumstances (Echevarria, Vogt, & Short, 2004).

Four such practices – bridging literacy, multimodal instruction, apprenticeship learning, and communal effort – are at the root of folk and traditional arts. Through a set of short  examples, drawn from City Lore’s Nations in Neighborhoods (NiN) program, we describe how these practices foster the development of a set of critical technologies of the self among English language learners.

Sociocultural Literacy.

Performing Bomba and Plena with accompanying drums

Sociocritical literacy “historicizes everyday and institutional literacy practices and texts and reframes them as powerful tools oriented toward critical social thought” (Gutiérrez, 2008, p. 1). Translated into the practices of NiN residencies, teaching artists encourage students to bring elements of their own lived cultures, and the cultures they are studying into the classroom, resulting in a two-way exchange. For example, during a residency about Puerto Rican music and dance, 5th grade students listened to, played, and improvised characteristic rhythms from the island. In addition, they listened to African and Spanish music as part of understanding how music reflects world diaspora patterns. Building on their understanding of the evolution of bomba and plena rhythms, they discussed how popular Latin music in New York is the contemporary expression of a long hybrid tradition. Many students with family backgrounds from the Dominican Republic and Puerto Rico identified a connection between the music and dance they participated in at home and at family gatherings, and the sounds and steps they were learning in class. In addition, one student with a family background from Nigeria also realized the connection to African music. In her first journal entry she writes “I enjoyed this very much because it reminded me of home in Nigeria and here in New York when my family dance and sing. I was able to make a connection to the type of cultural music the people danced to, because it was very similar to the music and style of dances we practice back home and back in Nigeria.” Later, she makes an even more immediate connection. She writes “Some things in my house have a pulse and a downbeat. When I was looking around I found my hula hoop and jump rope and when I used them to check for a clave [rhythm], I found that my jump rope made a swishing noise when I turned it as I jumped as it hit the ground I heard another beat that kept repeating as I jumped. Then I tested my hula hoop and when I was spinning it I heard swish woop woop woop noise, and I found the clave or pulse and it kept repeating. When I was walking, too, I heard two downbeats. One from me walking – it went click click click and I heard rolling book bags which made a whirring noise. I now realize many things around you have a downbeat or pulse.” These examples point to another two-way connection: the popular music that students hum and dance to is no longer something they have to ‘leave at the door.’ Instead, this music becomes the occasion for key literacy practices like research, critical listening, and evidence-based writing and discussion. The music and those strategies for deepening understanding are mutually illuminating, creating what Guitiérrez (2008) refers to as a ‘third space’ where students’ social and linguistic understandings merge with – and often enliven – academic learning. In this space, created by the study of folk and traditional arts, supported by the practice of journal-keeping, this student presents herself as an original thinker and a cosmopolitan learner. [see supplementary material at http://youtu.be/n-s_a8b_Q18]

Multimodal Instruction:

Performing the Brave Little Parrot with Mithilla Paintings in the Background

Animal observational drawing at the Bronx Zoo.

Since many folk arts are traditionally embedded in rich multi-arts contexts, NiN residencies are often co-taught by two artists from different, but culturally-related, arts disciplines. In fourth grade, students read and retold a Jakata tale from Northeast India, in a residency taught Malini Srinivasan, an Indian Bharatanatyam dancer, and Jenna Bonistalli, a visual artist. Students created a series of painted panels inspired by Mithila painting, a style of narrative folk painting also from Northeast India, enriched by observational drawing sessions of animals at the Bronx Zoo.

 Apprenticeship Learning and Communal Effort: 

In many folk and traditional arts, beginners – even children – are treated as serious learners who can be gradually introduced to the values and practices of art forms handed down by family members, elders, or master artists. The techniques demonstrated are not diminished or infantilized, and young learners use the same or similar materials and tools as their teachers, with the teacher carefully scaffolding the learning until the ‘apprentice’ can manage to drum, dance, cut, or stamp cloth skillfully (Rogoff, 1990). As part of drawing on traditional pedagogies, NiN artists often engage students in extended periods of observation and listening (“no questions asked”) as they demonstrate a movement, (Echevarria, Vogt & Short, rhythm or brush stroke, insisting that the eye and the hand are instruments of understanding Sennett, 2008). After this initial period of withholding questions and verbal instructions, teaching artists attach words to actions and materials, offer and demonstrate suggestions, and respond to questions. In this process, learners first witness and practice embodied learning, gradually acquiring the vocabulary to articulate their learning process and to justify the artistic choices they made. 

For nearly all students, the opportunity to become a participant in the fullest possible practice of an art form is engaging. For English language learners, the gradual movement from nonverbal learning to spoken exchange may afford the supports for acquiring new vocabulary, asking questions, offering observations, and eventually articulating the artistic choices they make. A printmaking residency taught by Alan Calpe to 8th grade students at the Academy of New Americans illustrates elements of the apprenticeship model. Calpe began the residency by showing students examples of his own work and discussing the techniques and ideas he explores in his work. He introduced them to printmaking terminology, tools, materials, and techniques. Students then explored the work of Latin American printmakers, including Olga Blinder, Livio Abramo, Antonio Henrique Amaral and Jose Guadalupe Posada, in order to understand both the diversity of their approaches and printmaking techniques and the ways they explore themes of self and cultural identity. Inspired by the work of these artists, students created their own design motifs as a template for their collagraph prints. They created a collagraph plate through collaged papers and did rubbings to understand how imagery can be created through relief. Students learned to work with professional printmaking tools and materials, even though some of the tools had sharp edges. Working in pairs they explored the printmaking process, using brayers and ink o create an edition of their printing plate. They also learned about subtractive printmaking by drawing on and gouging a rubbercut plate. Then they took a fieldtrip to a professional printing studio, K. Caraccio Printing Studio, where they learned about and used other printmaking techniques, tools, and materials. They made monotypes with a resident artist there, and each student created and pulled their own print. They also toured an exhibit and artist’s studio at the Elizabeth Foundation. At the conclusion of the residency, students assembled their prints into an exhibition, where the whole was greater than the sum of its parts.

Finally, as a part of scaffolding, or supporting individual learners, folk and traditional arts often feature joint or communal effort. For example, Puerto Rican bomba presents a dialogue between the improvised movements of the dancer and the responses of the accompanying drummer: neither is a soloist. Similarly, in many NiN residencies, students work in ensembles: performing dance theater and folk tales, jointly building huge mosaic patterns, creating murals and doing musical performances. In these settings, students who are in the process of acquiring academic English typically work as the drummer and the dancer do, frequently translating directions, concepts, and questions from English to Spanish (or Farsi or Mandarin).

Multimodal Instruction: 

Since many folk arts are traditionally embedded in rich multi-arts contexts, NiN residencies are often co-taught by two artists from different, but culturally-related, arts disciplines. In fourth grade, students read and retold a Jakata tale from Northeast India, in a residency taught Malini Srinivasan, an Indian Bharatanatyam dancer, and Jenna Bonistalli, a visual artist. Students created a series of painted panels inspired by Mithila painting, a style of narrative folk painting also from Northeast India, enriched by observational drawing sessions of animals at the Bronx Zoo. For the final performance, students retold the story through spoken narration and dance, with the painted panels projected as a backdrop. Reading, speaking, dancing and painting the story reinforced students’ understanding of the characters and the narrative line, as well as an understanding of cultural patterns in Indian folk and classical arts. While the written work of many students was several years below grade level, their dance and drawing experiences built a complex understanding of the animals and plants that populate the Jakata tale.

In a video interview one student narrates her picture, explaining how it captures the moment when the parrot flies over the burning forest on his way to get help. Showing how she drew the hazy smoke with a charcoal pencil, she explains that, “We are working on the leaves right now, and we want the leaves to pop out because they are really small” She points to a pencil “I’m using this because I want the leaves to look more better.” Based on having her drawings at hand, this student can tell a complex narrative, explain different artistic techniques, and discuss specific artistic choices (see supplementary material at link to the CARTS interview). In another example, when teaching artist Lu Yu engages students in a dance theater residency, he draws on his training in Chinese opera and acrobatics to show how simple props can imply entire scenes: He turns a bolt of fabric into a river, then engages students in transforming that same cloth into a boat, then a storm, then the rope that travelers cling to, using evocative visual clues to support students’ use of figurative language and comprehending complex meanings. As Echevarria explains, “Authentic, meaningful experiences are especially important for ELLs because they are learning to attach labels and terms to things already familiar to them. Their learning becomes situated rather than abstract when they are provided with the opportunity to actually experience what they are learning about.” 2004, p. 29).

Description of Activities

Through its activities, NiN seeks to:

• Create a replicable model for integrating social studies, community resources, and the arts
• Build teachers’ capacity to use community resources in their classrooms
• Improve students’ literacy knowledge and skills, as well as their skill in creating, performing, and responding to the arts.

The NiN curriculum is explicitly designed to engage young people in a wide range of activities designed to support the development of their speaking, listening, reading and writing skills:

• Ongoing classroom discussion and small group work focus on complex objects, texts, and concepts
• Journal writing allows students to formulate and record their individual thoughts
• Interviews provide practice in expressing concepts and experiences for external audiences
• The opportunity to represent experiences and ideas in a range of media (visual arts, performance, written language, and oral presentations) offers students many experiences and modalities for developing English language fluency.

Each year of Nations in Neighborhoods (NIN) focused on the arts, culture, and history of a specific region of the world: Asia, Latin America, and finally, in the third year of implementation, Africa.

Each program year commenced with a three-day Summer Institute.

During the Fall semester, artists and teachers collaborated with their students to develop arts-integrated units of study investigating the history, cultures, and artistic traditions of the region;

• These units of study were designed to support core curricula learning in English Language and Social Studies.
• Teachers and teaching artists also made local connections to relevant nations via NYC-based cultural institutions
• Residencies ran 12-16 sessions, investigated performing and visual art forms, and included a community connections component of a field trip and/or visit by a community member to share his or her expertise with the students.
  

  Classroom teacher at Bell Academy working with students on illuminated maps of the silk road.

Key Measurable Project Outcomes

Consequently, the focus of this report is the impact of NiN experiences on students’ literacy development as measured by:

Scores on New York state tests of English Language Arts proficiency,

These scores were supplemented by qualitative data on oral language data from interviews and writing development collected in classrooms, scored by teachers and artists, in Year 3 and used to inform and add rigor to the implementation of residencies in Years 3 and 4.

To define these effects rigorously, we employed a multilevel statistical model that accounts for the nested structure of the data. Children’s literacy scores are likely to be more similar in the same class and school. Therefore, these classroom and school effects must be accounted as a part of identifying whether there are any additional effects attributable to children’s participation in the NiN program itself. While this model introduces a level of methodological complexity not normally encountered in analyses of arts programs, it is one solution to moving beyond simple correlational evidence and regression models for identifying the effects of arts integration experiences.

Treatment

Participating schools have been part of the project for four years, plus a planning year with a core of volunteer teachers who have remained 80% stable. In each participating school:

• Teachers attend a three-day summer institute, followed by three full-day professional development sessions spread throughout the school year.
• A teaching artist brings a residency based in the arts of a specific region of the globe to each participating classroom. Each residency includes:
• Planning with the classroom teacher
• Fourteen sessions of classroom work with an artist or artists.
• A field trip to a related cultural site (museum, business, community center) or visit to the classroom by a guest artist or community expert
• A final performance or exhibition of student work

In implementation Year 3, between 11 and 36% of students attending these four schools were in classrooms that participated in the NiN program.  However, not every grade within each school had classrooms participating in NiN: at IS 235, students in 6^th and 8^th grade participated in the program, while at IS 294 6^th graders participated.  Moreover, within a given grade at any school, selected classrooms participated in the program while others did not.  For this reason, data analysis focused on a comparison of participating and non-participating students both across and within specific grades.

Peace tower installation opening

Methods

The research coordinators and principals of each of the schools supplied the data (with the one exception noted above).  These data were taken from the Achievement Reporting and Innovation System (ARIS), and included students’ grade in school, classroom, gender, ethnicity, English Language Learner (ELL) status (yes/no), Special Education (SPED) classification (yes/no), and scaled scores on the English Language Arts (ELA) exam, used by New York City (and state) to test student’s reading and writing achievement.

The data are inherently hierarchical in their structure, with students nested within classrooms, which are in turn nested within schools.

As such, a multilevel model was used to estimate ELA scores as a function of NiN participation while accounting for the nested structure of the data.

Failing to account for this nested structure will yield inaccurate results.

Preliminary analyses were conducted to determine if ELA exam scores were systematically missing as a function of school, classroom, gender, ethnicity, SPED or ELL status.  These variables were entered in all subsequent analyses.

As mentioned earlier, these more sophisticated analyses are one solution to developing a next round of more rigorous evidence for the effects of arts integration programs such as NiN.

Results 

An Overall Effect on English Language Arts Scores

Given the program’s emphasis on English language development, the measure of primary interest was students’ scaled scores on the ELA exam.  Table 1 reports mean ELA scores for the students in the sample taking the ELA exam in each year of the program.

Table 1

A series of random-effects ANOVAs were performed within the multilevel modeling framework to assess the contribution of students, classrooms, and schools to the pooled variance in scaled ELA scores for each year.  These models revealed that across years student factors exerted the strongest influence on ELA scores, but the school and classroom within the school to which the student is assigned are also associated with these scores (see Table 2).

Table 2  

For the 2009-2010, 2010-2011, and 2011-2012 program years a three-level model was run (presented in note 9) to assess the effects of NiN after controlling for gender, ethnicity ELL and SPED status (entered into the model as covariates).  For the 2008-2009-program year, a two-level model was adequate, given that students were enrolled at a single school.  For the 2008-2009 and 2009-2010 program years, no relationship between participation in the program and ELA scale score was observed.  However, for the 2010-2011-program year, children who participated in NiN scored 6.51 points higher on average on their ELA scale scores than children who did not participate in the program, a difference that approached statistical significance (t (36.3) = 1.73, p = .091).  For the 2011-2012-program year, this difference was significant, with children who participated in NiN scoring 6.43 points higher on their ELA scale scores (t (40.6) = 2.29, p = .027) than students who did not receive the program.

This effect is roughly equivalent in size to one-quarter of the effect for being classified ELL.  In other words, participation in NiN is associated with an increase in ELA scores that is 25% as large as the decrease associated with ELL status.

This pattern of increasing effectiveness across the three years of implementation is addressed in detail in the discussion section below.

Within-School Findings 

Inside these overall findings for the impact of NiN, there are important differences associated with specific schools, grades, and specific circumstances in the 2011-12 school year. These variations point to the considerable importance of digging beneath overall, or even school-level, findings to identify the classroom conditions and the design features of specific arts integrations projects that support high-level implementation with significant consequences for students.  Already in examining the pattern of findings, City Lore staff have been able to identify to some of the practices and circumstances that support or limit the effects of the program. (See the concluding sections of this report.)

PS 11

In the aggregate, NiN was associated with significantly higher ELA scale scores for students attending PS 11 (t = 2.69, p = .007), with students receiving NiN scoring, on average, 8.03 points higher on ELA scores.  But the pattern varied by grade:

For 4^th graders taking part in NiN this association was evident at the trend level, but did not reach statistical significance (t = 1.52, p = .129).

For 5^th graders, a more complex pattern of results was observed, in which both a main effect (t = 2.43, p = .016) and an interaction effect with ELL status (t = -2.48, p = .014) were found for NiN participation (see Appendix, table 3).  An interaction was found between ELL status and NiN participation. Probing this interaction effect revealed an unexpected finding.  Among native speakers, ELA scores were higher for students in NiN than for those who did not participate in NiN. But for ELL students this pattern of results was reversed: ELL students scored higher if they did not participate in NiN (see Appendix, figure 2).

PS 69

All students at PS 69 received NiN. Thus, while there are overall gains, no within-school or grade-level analyses were possible.

PS 117

In the aggregate, students participating in NiN did not exhibit higher ELA scores than non-participants (t = 1.52, p = .130).

Participating 5th and 6th graders did not display a main effect on their ELA performance. Classroom observations point out that in both of these classes there were significant disruptions to the continuity of NiN instruction: a classroom teacher’s chronic illness in one case; the substitution of a new teacher for a teacher with experience in the project, in the second case.

In contrast, for 4th graders, participating in NiN was associated with significantly higher ELA scale scores (t = 2.20, p = .030).  On average, students in classrooms participating in NiN received scale scores that were 5.72 points higher, after controlling for gender, ethnicity, ELL classification, and SPED classification.

IS 235

Students participating in NiN did exhibit significantly higher ELA scores (t = 2.32, p = .023); on average, their scores that were 8.69 points higher than their peers after controlling for grade, classroom, gender, ethnicity, ELL status, and SPED status.

This effect held for 8^th graders (t = 1.98, p = .051).

It did not hold for 6^th graders (t = .428, p = .690).

Note that no classrooms of 7^th graders received NiN in 2011-2012.

Additional information on classrooms indicates that the 6th grade class was comprised almost entirely of newcomer students with little to no academic English. By comparison the 8th grade class had many more nearly fluent ELL students.

IS 294

NiN was not associated with higher ELA scales at IS 294 in the aggregate or for any grade.  Classroom observations point out that in this year scheduling difficulties prevented the artist from working the full amount of time with students with the result that the planned large-scale group learning project devolved to shorter, more individualized assignments. As a result the implementation of the residencies at this school were more nearly an exposure program than an in-depth art making experience.

Supplementary Qualitative Data on Classroom Measures of Oral Language and Literacy Development

During Year 3, as part of strengthening the residencies and ensuring that teaching artists and classroom teachers shared a common definition of program goals, City Lore designed a seminar in which both groups joined in scoring samples of student interviews and written assignments. During three professional development days the evaluators, teachers, and teaching artists identified and defined a set of authentic literacy practices that City Lore programs support.  These included:

• Discussion and oral presentation of work-in-progress
• Oral reflection
• Journaling, drawing, and diagramming to capture current observations and emerging ideas
• Written reflections, labels, etc. for final work

Subsequently, a group of teachers and artists collaborated in a multi-session workshop to develop a set of pre-, during, and post-measures of student learning, aligned to City Lore’s practices and to the new Common Core Standards particularly in the area of informational writing.

The goal was to pilot a set of measures that could index whether there was immediate, short-term literacy growth that occurred over the course of the residencies (which lasted 12-14 weeks).  These measures were piloted and then applied to a small structured sample of students in participating classrooms (2 students performing below, at, and beyond grade level, or, in the case of Special Education classrooms, 2 students performing at the highest level for that group.) Two raters from the group established inter-rater reliability and then scored that portion of the structured sample for which there were the full complement samples.

Student Writing

The initial findings from this work are reported below. One measure looked at the improvement in students’ writing in response to a prompt asking them to describe an object or performance that was related to a specific residency they were experiencing. The overall writing measure was comprised of 9 dimensions of writing (e.g., use of relevant art form and cultural content, sentence complexity, conventions of written English, etc.) each scored on the 1 – 4 scale (with three being grade-level proficient) that New York City teachers are accustomed to applying to student writing.  Figure 3 shows the changes in performance for the early elementary and the middle school samples from pre- to post- writing task.

Student Interviews and Discussion 

The overall interview and oral discussion measure was comprised of 8 dimensions of speaking (e.g. command of relevant cultural and art form content, personal engagement, complexity, etc.) each scored on the 1 – 4 scale that New York City teachers are accustomed to applying to students’ work.  Figure 4 shows the changes in performance for the early elementary and the middle school samples from pre- to post- writing task.

Figure 3: Pre – post Changes in Student Writing Scores (N = 44)

Figure 4: Pre –post Changes in Student Oral Interview Scores (out of 44)

While the Year 3 sample of students is small (N= 44 across both levels), these data are promising. They include all 6 schools, including IS 235, where students do not take standardized tests. The project now has reliable measures for capturing short-term growth in expanded forms of literacy. Moreover, having seen the data, and understood it purposes, more teachers in Year 4 are motivated to collect the requisite sample of pre-, during, and post- student work.

As with the whole school data, it is important to interpret the residency-level literacy measures cautiously. Again, many factors, in addition to NiN practices may be influencing how well students speak and write in the context of the residency. These data are drawn across many grade levels, media, and times of the year, so there is considerable “noise” in the samples.  Nevertheless, the scoring student work seminar had a focusing effect on the design and conduct of the residencies, helping both teachers and teaching artists to provide increased opportunities for speaking and writing and to set high expectations for the kinds of information and reflection that students engage in.  Potentially the scoring seminar played a role in the increased quality of the implementation in Year 4 and the resulting higher levels of student performance on the standardized tests.

Discussion of Findings 

These analyses point to an overall significant effect of NiN participation on students’ literacy scores for the 2011-12 academic year, with a similar trend observed in the 2010-2011 academic year.  This suggests that programs like Nations in Neighborhoods are one strategy for boosting literacy achievement in urban schools where many elementary school students are acquiring academic English as a language that will enable them to participate, contribute, and succeed.  However, no such effects were found for the 2008-2009 or 2009-2010 academic years.

When results for 2011-2012 year were probed at the school and classroom level, the data show substantial variability, with some schools and grades showing larger gains, while others showed little to no effect.  This points to the considerable importance of the project’s qualitative data (e. g., classroom observations, interviews, and samples of student work). This classroom-level information points out that the program is most successful when:

• Classroom teachers participate in professional development consistently
• Classroom teachers actively participate in the residency workshops and extend the residency lessons between workshops with the teaching artist
• Principals and teachers work together to schedule the program effectively
• Students have enough English proficiency to benefit from the classroom discussions, interviews, and writing assignments
• Teacher-artist collaboration is supported by face-to-face planning and designing for integration
• In class experiences are enriched by dynamic out-of-classroom experiences (field trips, performances, and guest visits) that are engaging and memorable, and which re-enforce connections between the students’ own lives and the artistic/cultural content they are exploring

The findings outlined in this report underscore the importance of putting effective data collection procedures in place at the time that studies are beginning, rather than attempting to use data to retrospectively evaluate a program.  Participation in Nations in Neighborhoods (NiN) was clearly associated with higher ELA scores in 2011-2012 and this relationship held for the previous 2010-2011 academic year at the trend level.  Prior to that, this association was not observed.  Did something change about the program prior to fall 2010 that year that might account for this change, or did the sample of children included in those years of the student vary in some manner not captured by our demographic data?  Because our data were drawn from extant school records, we cannot answer this question.

In addition, the pattern in these findings suggests how programs may mature across the several years of an AEMDD grant. Interviews with NiN staff suggest that the increasingly rigorous and effective implementation of the NiN program may be the cumulative result of:

• A consistent teacher and teaching artist cohort. The majority of the educators in the project had repeated opportunities to practice implementing the project approach, and I think they also felt as though they had ownership of the project
• Classroom teacher buy-in to an authentic documentation and E&A approach. The teacher study group/collaboration in developing the performance tasks was instrumental in ratcheting up our data collection efforts and producing a common language around student arts and cultural learning, and goals for artist instruction.
• Program direction improvements over time: better participant orientations and PD’s, more opportunities for paid planning meetings, more TA meetings about E&A, clearer protocols, and increasingly focused implementation
• A growing number of students with multiple years of experience in the program – the “village elders” of the project.

The findings also hint at how valuable a truly longitudinal approach would be in assessing the efficacy of the NiN program.  The composition of the data set is tantalizingly close to a longitudinal set – we know some children exist in data for multiple years, but we do not know which children.  If we did, we could examine trajectories in ELA scores over time as a function of NiN participation.  This would allow us to investigate how improvement in ELA scores varies as a function of depth of participation, or ‘dosage’.  We may discover that this relationship is not linear: perhaps a student must receive the program for multiple years before sizable gains in language development are observed, or perhaps those gains are compounded by some multiple of each year’s experience.

Rehearsing Japanese Immigration Stories

Students who participated in the Nations in Neighborhood program earned significantly higher ELA scores than students who did not participate in the program. Thus, consistent with our hypothesis, participation in a program of folk and traditional arts instruction incorporating socio-critical literacy, multimodal instruction, apprenticeship learning, and communal effort was associated with better ELA skills.  The size of the effect was considerable, equal in magnitude to approximately one-quarter of the decrement in scores associated with ELL status. Given these results, and the fact that many districts across the country also serve large numbers of ELL students, we focus this discussion on why the featured practices may be supportive of literacy learning. Consider this example:

In a dance theater residency focused on stories of Chinese immigration to the US, teaching artist, Lu Yu draws on his experience in Chinese opera, acrobatics, and contemporary theater – as well as his years of practice. He models how a single gesture can symbolize the essence of a character, putting his entire face, body, and voice to work to become a villain, a frightened villager, or a storm-tossed traveler. He questions each child closely to develop an understanding of his or her character’s intentions, nature, and wishes. Once a student arrives at that essential posture and movement, Yu urges him or her to speak from that embodied understanding, improvising towards increasingly articulate body movements, facial expressions, and speeches that evolve across each rehearsal. Because there is no memorized text, there are no fatal mistakes or forgetting. With each performance a student practices translating complex meanings into words. The student is the playwright as well as the performer. Because every student witnesses how each character is built, all students command the narrative communally; when someone forgets or gets confused, anyone can jump in at any moment to fill a gap or amplify the tale.

On one hand, Lu acknowledges the resources students bring from their personal experiences to development and performance of the narrative, creating a third space that places novel concepts and ideas in a familiar context. At the same time, he relies on observation, movement to support the development of character speech and narration. Since there is no fixed script, students can add to their parts as their understanding grows and as they learn from one another. Because every student witnesses how each character is built, all students command the narrative communally; any one of them can jump in at any moment to fill a gap or amplify the tale. A student recently arrived from Bangladesh  lead a scene where Chinese railroad workers remember that  it is Chinese New Year back home. Lu instructs him to say loudly “They must be celebrating Chinese New Year back home” and then directs the students to think about how they would react to thinking of their families celebrating back home. Some choose wistful gestures, some choose exclamations of chagrin, excitement, and wonder. The student practiced a few different ways to say the line to incite the most varied reactions from the rest of his scene partners. As he collaborated on the gestures and tones with suggestions from Lu they built the scene allowing for both an English Language learner to feel the group responding to his spoken line, and to understand the power that a facial gesture or a wordless exclamation can summon to convey a range of feelings, memories and experiences.

Limitations and Future Directions

Some caution is in order. This study shares the same limitations as any quasi-experimental study. Although there is a comparison group, it is not a true control group, in that children were not randomly assigned to it [Confirm this in reference to question about how classrooms were selected].  The children included in the sample were chosen based on convenience – they happened to attend one of the five schools in which the program was implemented. Therefore we can’t generalize our findings to broader populations of children. Finally, we cannot say that participation in the program led to higher ELA scores, only that participation was associated with them.

Despite these limitations, the study raises some interesting questions for future research.  Perhaps the most intriguing among these has to do with causal mechanisms or processes: how, exactly, does instruction in folk and traditional arts come to be associated with improved English language outcomes? We have posited that key practices – sociocultural literacy, multi-modal learning, apprenticeship learning, and communal effort – support increased literacy development. But what changes in children as they experience these practices? If, as we propose, NiN students are developing key technologies of self, is this improvement confined to ELA, or is it more global in its effects? Does students’ interest in school increase? Do students’ theories about who can learn or excel shift? Alternatively, does the creation of  ‘third spaces’ lead to settings and interactions where larger numbers of students ‘shine’, thereby changing teachers’ perceptions of students’ strengths and resources?  Only by understanding what underlying factors, or mediators, fuel the observed growth in literacy we can further refine a practice where the distinctive attributes of folk and traditional arts support learning in culturally and linguistically diverse classrooms.

Our findings indicate that instruction in the folk and traditional arts may benefit students, including a subset of those who are often at increased risk for educational failure, despite the experiences and resources they bring to learning. Indeed, these findings suggest that eliminating arts education to devote more time to language arts instruction may ultimately be counter-productive. More broadly, these results underscore the role of the arts in a complete education and suggest how an arts education that includes folk traditions may be particularly effective.