ORE Planned Research Agenda, 2019-20

This page lists planned 2019-2020 research and evaluation projects by topic. Expand each project description to learn more about the questions we’re asking.

Case Studies of Schools with Improvements in K-3 Literacy

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • What are the contexts and conditions in place in elementary schools in that have made large improvements in 3rd grade literacy?
  • What do principals and other school leaders identify as the supports, practices, or actions school that have contributed to their improvement?
  • In what ways have school and district leaders contributed to school level changes?

Study of Four-Year Changes in K-3 Student Literacy Outcomes

This study is a grant requirement and major district initiative.  The question we’re asking is:

  • To what extent have schools and students demonstrated changes in literacy outcomes (PSSA, AIMSweb, Minimum Growth) between SY2015-16 and SY2018-19?

Study of K-3 Summer Learning Loss

This is a knowledge building study of a major district initiative.  The questions we’re asking include:

  • What changes (if any) in independent reading levels did K-2 students experience between the Quarter 4 (June) assessment and Quarter 1 (November) assessment across three years?
  • Did changes vary based on grade level?
  • Did changes vary based on student demographics?

Evaluation of 4-5 Summer Literacy Institute

This is a knowledge building study of a major district initiative.  The questions we’re asking include:

  • Were participants satisfied with the quality of the professional development offered at Institute?
  • How did teacher knowledge of literacy practices change after participating in the Institute?

Evaluation of K-3 Online Institute Modules

This study is a grant requirement and major district initiative.  The questions we’re asking include:

  • To what extent were the online sessions implemented as intended?
  • To what extent were teachers satisfied with the quality of the online sessions?
  • How did teacher satisfaction and knowledge gains differ by Institute format (online or in person)?

Evaluation of the Early Literacy Specialist Coaching Model

This study is a grant requirement and major district initiative.  The questions we’re asking include:

  • Were coaches, teachers, and principals satisfied with the ELS coaching initiative?
  • Was ELS coaching implemented as intended?
  • What were the primary successes and challenges of implementing ELS coaching?
  • What do teachers and principals perceive to be the benefits of coaching?
  • In what ways did schools improve their literacy instruction as a result of coaching?
  • In what ways did teacher retention change?

Evaluation of K-5 Literacy Coaching

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • How was K-5 coaching implemented across elementary schools?
  • What were the primary successes and challenges of implementing coaching?
  • What do teachers and principals perceive to be the benefits of coaching?
  • In what ways did schools improve their literacy instruction as a result of coaching?

Evaluation of Innovative Approaches to Literacy (Building Bridges with Books)

This evaluation is a grant requirement.  The questions we’re asking include:

  • To what extent is the program accomplishing activities to improve curriculum and instruction, student learning environment, and parent and family engagement?
  • How many teachers and librarians were trained?
  • To what extent is the program plan and/or components meeting teacher/librarian/school needs?
  • To what extent are parents satisfied with program activities?
  • Are teachers/ librarians better able to identify and remediate students reading below grade level? How have teacher practices changed as a result of the program?
  • Do students in the program demonstrate increased literacy/ ability to read at grade level?
  • At the grade and school level, are there increases in reading performance?

Evaluation of Reading Specialists Support for Struggling Readers

This evaluation is a grant requirement.  The questions we’re asking include:

  • How many students received support from a Reading Specialist? How often?
  • How many Reading Specialists complete the trainings for various interventions?
  • What is the perception of the Reading Specialists in terms of their ability to implement the grant-funded interventions?
  • Did students that received support from a Reading Specialist demonstrate improvements in literacy outcomes? What lessons can be learned from Reading Specialists with particularly positive results?

Study of Implementation and Outcomes of Blended Learning Approaches

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • How often do students use online adaptive programs in the classroom?
  • How do teachers and coaches perceive the implementation of blended learning?
  • How did teacher survey results and student usage data change from previous years?

Study of Implementation of Common Planning Time Protocols

Study of Implementation of Renovated Literacy and Learning Centers in PreK-2 Classrooms

This study is a grant requirement. The questions we’re asking include:

  • To what extent have the renovations and professional development been implemented as intended?
  • To what extent are classroom teachers satisfied with the renovations, professional development, and program supports?
  • To what extent has the program improved teacher practice?
  • Did students in the treatment classrooms show growth in literacy and attendance rates? If so, to what degree?

Study of Implementation of Teacher Academy

This is a knowledge-building study.  The questions we’re asking include:

  • What are the coaching needs of the Teacher Leader Academy participants?
  • What are the perceptions of the participants on the ongoing PD?
  • To what degree do the Teacher Leader Academy participants agree that the PD as increased their knowledge of the six leadership competencies, their confidence in their ability to implement them, and their motivation to do so in their schools?

Descriptive Study of New Teacher Characteristics

This is a knowledge-building study.  The question we’re asking is:

  • What are the educational and professional backgrounds of new teacher hires in SDP?

Case Studies of Schools with High and Low Teacher Retention

This is a knowledge-building study. The questions we’re asking include:

  • What are the contexts and conditions in place in elementary schools in SDP that have made large improvements in 3rd grade literacy?
  • What do principals and other school leaders identify as the supports, practices, or actions school that have contributed to their improvement?
  • In what ways have school and district leaders contributed to school level changes?

Study of Teacher Perceptions of Site Selection Process

This is a knowledge building study.  The question we’re asking is:

  • What are teachers perceptions of the site selection process?

Evaluation of Teacher Residency Programs

This is a knowledge-building study of a major district initiative. The questions we’re asking include:

  • What are the characteristics and key components of teacher residency programs operating in SDP?
  • Are residents, mentor teacher, and principals satisfied with their experiences?
  • What are their perceptions of the effectiveness of their program?

Study of Differences in PSSA Outcomes between Dual Language and Non-DL Students

This is a knowledge-building study.  The question we’re asking is:

  • Are there differences in the PSSA outcomes of ELs and non-ELs by Dual Langauage program participation?

Study of K-5 EL ACCESS Growth

This is a knowledge-building study. The questions we’re asking are:

  • To what extent are K-5 students in SDP meeting their ACCESS growth targets? Are their differences in ACCESS growth by student demographics?
  • Is there a relationship between school characteristics and ACCESS growth (number of ESL teachers, size of EL population, school size, DWS ratings, etc)?

Evaluation of the Quality Teaching for English Learners (QTEL) approach

This evaluation is required by a grant. The questions we’re asking are:

  • To what extent is each component of the QTEL initiative implemented as designed? To what extent are teachers and coaches satisfied with the quality of the QTEL initiative activities (institute, coaching, PD)?
  • How does participating in QTEL Institute change pedagogical knowledge and self-reported beliefs about English learners?
  • To what extent does participating in the QTEL initiative change teacher practice?
  • To what extent do schools that implement the QTEL initiative experience changes in English Proficiency levels as measured by ACCESS scores?

Evaluation of the Implementation and Outcomes of the "Together is Better" Approach to Supporting English Learners

This evaluation is required by a grant. The questions we’re asking are:

  • To what extent has the program been implemented as intended?
  • To what extent are stakeholders satisfied with the co-teaching model and related PD, coaching, and support?
  • To what extent has the program influenced teachers’ instructional strategies and cultural competence?
  • To what extent are students participating in TIB schools experiencing academic growth and other desirable school outcomes?

Study of Education for Children and Youth Experiencing Homelessness (ECYEH) Grant-Funded Activities

This study is required by a grant.  Questions we’re asking include:

  • Did ECYEH approaches in 2018-19 result in an increase in the identification of students experiencing homelessness?
  • How many students received the following support: enrollement assistance, uniforms, school supplies, transportation assistance, and tutoring? Is there a decrease in the percent of students identified as truant in the 2018-19 SY compared to the previous year?
  • Do families identified by ECYEH feel knowledgeable about their rights and are satisfied with the supports provided?
  • To what extent do students identified as homeless improve on PSSA Reading and Math assessments, and are promoted to the next grade (high school students)?

Study of Implementation of PA SMART Grant-Funded Activities to Increase Computer Science Course Offerings

This study is required by a grant.  Questions we’re asking include:

  • How many students have access to computer science courses?
  • Are more students enrolling and passing computer science courses as a result of the PA SMART grant?

Study of School Selection Participation

This is a knowledge-building study related to a major district initiative.  Questions we’re asking include:

  • Which students participate in the school selection process, which students are offered admission, and which students ultimately enroll in programs with competitive criteria?
  • Do these patterns differ by sending school, by receiving school, or by student demographics (race/ethnicity, gender, economic disadvantage, special education status, English learner status)?

Study of Suspension Trends and Patterns

This is a knowledge-building study.  Questions we’re asking include:

  • What is the trend in the percentage of students receiving zero out of school suspensions?
  • How do the trends differ by gender, grade, disability status, and English Language Learner status?
  • At the District level, were certain groups of students – based on student characteristics including gender, grade, disability status, English Language Learner (ELL) status, and race/ethnicity – suspended disproportionally to their representation in the overall student body?

Study of Teacher and Student Demographics

This is a knowledge-building study.  Questions we’re asking include:

  • What are the demographic characteristics of SDP teachers
  • How do teacher demographics compare to the demographics of SDP students?

Study of Special Education Student Reports of Belonging in School

This is a knowledge-building study summarizing District-wide survey data.  The question we’re asking is:

  • What do students with IEPs say about how welcome they feel in school?

Study of Implementation of Academic Parent Teacher Teams (APTT) approach

This study is a grant requirement. The questions we’re asking include:

  • How many parents attended each of the APTT meetings?
  • What are parent and teacher perceptions of APTT?
  • Do students demonstrate increases on their core assessment performance from Fall to Spring?
  • What are essential factors to the sustainability of APTT?

Study of School Health Index Results

This is a knowledge-building study.  The questions we’re asking include:

  • What common challenges do schools face in implementing health and wellness activities?
  • To what extent do school responses on the SHI indicate alignment with the District Wellness Policy?
  • In which areas do schools need the most support in meeting guidelines?

Study of Breakfast Model Implementation and Outcomes

This is a knowledge-building study.  The questions we’re asking include:

  • What barriers exist in expanding school breakfast participation?
  • How can Eat Right Philly and other community partners support and assist Food Service to increase breakfast participation rates?

Evaluation of Eat Right Philly (ERP) Grant-Funded Programming

This evaluation is a grant requirement.  The questions we’re asking include:

  • What challenges and successes do ERP community partners encounter in their schools?
  • What opportunities exist for ERP community partners to align and coordinate programming?
  • What are successful and sustainable food access and assistance models in schools?
  • How do these programs engage parents and caregivers?

Evaluation of Promoting Adolescent Student Health (PASH) Grant-Funded Programming

This evaluation is a grant requirement.  The primary question we’re asking is:

  • To what extent do PASH priority schools provide: (1) effective sexual health education to students; (2) access to key sexual health services for students; and (3) safe and supportive environments for students?

Study of Naviance Use

This is a knowledge-building study.  The questions we’re asking include:

  • Which Naviance tasks are being completed students in grades 6-12?
  • Are there differences in completion rates for students based on their enrolled school, or on their demographic characteristics?

Study of Implementation of Ninth Grade Academies

This is a knowledge-building study of a major district initative.  The questions we’re asking include:

  • How are schools implementing ninth grade academies?
  • What are the challenges and barriers to implementing a ninth grade academy?
  • What supports do Assistant Principals feel they need in order to successfully implement a ninth grade academy?

Study of Implementation of Open to Opportunities in Career and Technical Education

This is a grant required study.  The questions we’re asking include:

  • How many students enrolled at two District schools for adjudicated youth received 5 days of CTE instruction?
  • How many received certifications or industry credentials?
  • How do students rate classroom instruction and climate?
  • What is the recidivism rate for program participants?

Study of Alignment of Course Marks and Standardized Test Scores

This is a knowledge-building study.  The primary question is:

  • How well are student grades aligned with their performance on standardized assessments?

Study of Credit Recovery Trends (with PERC)

This is a knowledge-building study.  The questions we’re asking include:

  • How many students begin and complete a credit recovery course?
  • How do students perform in courses after completing a credit recovery course?

Study of High School Graduate Performance in Postsecondary Settings (with PERC)

This is a knowledge-building study.  The question we’re asking is:

  • What are the patterns of postsecondary course enrollment, placement in developmental courses, and outcomes of SDP graduates attending CCP and Temple?

Study of Trajectories of Categories of 9th Grade Students

This is a knowledge-building study.  The question we’re asking is:

  • What are their academic trajectories and Keystone performance of subgroups of ninth grade students through their remaining high school years?

Study to Determine College Ready Metric for Graduates

This is a knowledge-building study.  The question we’re asking is:

  • What high school characteristics are related to college success for District students?

Study of CTE Program Participation and Student Outcomes

This study is a grant requirement.  The questions we’re asking include:

  • What are the background characteristics of students that do and do not choose to take CTE courses?
  • What are the educational outcomes of these two groups?

Descriptive Findings from Graduate Follow-Up Survey

This is a knowledge-building project.  The questions we’re asking include:

  • What are the career and education activities and experiences of students that graduated 1-2 years ago?
  • Are their current plans consistent with their plans at the time of graduation?
  • What obstacles have they experienced in meeting their goals?

Descriptive Information from Senior Exit Survey

Administering the senior exit survey is a state requirement.  The question we’re asking is:

  • What are the post-graduation career and education plans of 12th graders?

Descriptive Information from CTE Graduate Follow Up Survey

This is a knowledge-building study.  The question we’re asking is:

  • What are the career and education activities of recent graduates that took CTE courses?

Evaluation of High School Summer Institute

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • Were participants satisfied with the quality of the professional development offered at Institute?
  • How did teacher knowledge change after participating in the Institute?

Study of Implementation of School-wide Positive Behavioral Interventions and Supports (SWPBIS)

This is a knowledge-building study of a major district initiative.  The question we’re asking is:

  • What are the outcomes (attendance, suspensions, achievement, and climate) in schools implementing SWPBIS?

Study of Implementation of STEP Program

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • How has the hiring process for STEP teams been implemented?
  • What types of services has the STEP team provided at each school?
  • How many students have been served by STEP?
  • What are some of the challenges and barriers to STEP implementation?
  • What types of support does the STEP team feel necessary to successfully implement this model?

Study of Implementation of Youth Courts

This is a knowledge-building study.  The questions we’re asking include:

  • How is Youth Court implemented at each school?
  • How many students have participated in Youth Court at each school?
  • What are the challenges and barriers to implementation of Youth Court?

Study of School Climate and 9th Grade On Track Rates

This is a knowledge-building study that uses District-wide survey data.  The question we’re asking is:

  • What is the relationship between school climate ratings and 9th grade On Track Rates?

Study of KEI Trends over Time

This is a knowledge-building study. The question we’re asking is:

  • How have kindergarten students scored on the Kindergarten Entry Inventory over time?

Descriptive Information about School Partners/Vendors Supporting School Improvement

This is a knowledge-building project. The questions we’re asking include:

  • How many and which partners/vendors are currently supporting schools?
  • What are areas of need where partners/vendors could provide additional support at schools?

Study of City Year Activities and Student Outcomes

This is a knowledge-building project. The questions we’re asking include:

  • How are City Year supports being implemented in schools?
  • How many students are served?
  • How do outcomes for students receiving City Year supports compare to similar students who do not receive supports?

Study of Schools Supported by System of Great Schools (SGS) Activities

This is a knowledge-building study of a major district initiative. The questions we’re asking include:

  • What supports do schools receive when part of the System of Great Schools (SGS) process?
  • To what extent are the goals of the supports reached – for each school and for the system of schools receiving supports?

Evaluation of Second Step Social-Emotional Learning Curriculum Implementation

This is a knowledge-building study of a major district initiative. The questions we’re asking include:

  • Are teachers implementing Second Step with fidelity?
  • Are teachers adequately trained, prepared, and supported to implement Second Step?
  • What benefits and challenges have teachers experienced with Second Step?
  • What additional supports do teachers need to implement Second Step successfully?

Description of DHS-involved Student Outcomes

This is a knowledge building project.  The questions we’re asking include:

  • What are the demographic characteristics (i.e. gender, race/ethnicity, English Learners, etc.) of students being served by DHS that attend District schools?
  • What are the academic and behavioral outcomes for students served by DHS?

Study of Parent/Guardian-Community Ratings and School Outcomes

This is a knowledge-building study that summarizes District-wide survey data.  The question we’re asking is:

  • What is the relationship between parent/guardian perceptions of parent/guardian-community ties and student academic outcomes?

Study of PSSA Trends Over Time

This is a knowledge-building study.  The question we’re asking is:

  • To what degree have students moved between proficiency levels on the math and ELS-PSSA between 2015 and 2018?

Study of Teacher Retention and Turnover (in partnership with REL-MA)

This is a knowledge-building study.  The questions we’re asking include:

  • What are the characteristics of the School District of Philadelphia teacher workforce?
  • What are the annual mobility rates for teachers in SDP, and how do rates vary among different subgroups of teachers or schools?
  • What teacher and school characteristics are associated with an increased likelihood of changing schools within the district or leaving the district?

Study of Trends in School Climate Over Time

This is a knowledge-building study.  The question we’re asking is:

  • What are the trends in school climate indicators over time?

Study of Relationship between Gains in School Climate and Gains in School Outcomes

This is a knowledge-building study.  The question we’re asking is:

  • What is the relationship between school-level change in school climate indicators and school-level achievement over time?

Study of Graduation Trends Over Time

This is a knowledge-building study.  The question we’re asking is:

  • What are the trends in school graduation rates over time?

Evaluation of Renaissance/Turnaround Schools that Joined in 2016-17 Cohort (contracted with Mathematica)

This is a 3-year contracted evaluation of a major district initiative. The questions being asked are:

  • Year 1:  What are the context and supports for the 7 Renaissance Initiative schools that were identified in 2016-2017.  To what extent do the efforts align with literature?
  • Year 2:  What is the impact of entering the Renaissance Initiative in 2016-2017 (two Renaissance Charters and five District-run turnaround schools) on school- and student-level outcomes after two years.
  • Year 3:  What is the relationship between the costs of the Renaissance Initiative and student and school outcomes?

In addition to these research projects, ORE plans to publish data briefs summarizing key performance indicators, create topical summaries of District-Wide Survey findings, disseminate key findings from the Youth Risk Behavior Survey, and develop other materials that support the use of research, data, and evidence. For a summary of major projects across the Evaluation, Research, and Accountability (ERA) division in 2018-19, download this report from August 2019.