ORE Planned Research Agenda 2022-23

Our planned 2022-23 research and evaluation projects are listed here by topic. If a project spans multiple topics, we have organized it by its primary topic.

Expand each project description to learn more about the questions we’re asking.

Assessments

Descriptive study of participation on District-wide assessments administered in 2021-22 (includes College and Career assessments)

This is a knowledge-building study. The questions we’re asking include:

  • Which tests were administered by the Office of Assessment in 2021-22 school year?
  • What were the participation rates for these tests?

View the published brief:
A Summary of District-Wide Assessments Administered to SDP Students During the 2021-22 School Year

Descriptive study of 2022-23 PSSA results

This is a knowledge-building study. The questions we’re asking include:

  • What percentage of students scored proficient or advanced on the PSSAs this year, and how does it compare to prior years?
  • What percentage of students scored proficient or advanced on the PSSAs this year, and how does it compare to prior years, by grade level?

View the published brief:
PSSA Performance Trends in SDP: 2018-19 to 2022-23

Descriptive study of 2022-23 Keystone results

This is a knowledge-building study. The questions we’re asking include:

  • What percentage of students scored proficient or advanced on the Keystone exams this year, and how does it compare to prior years?
  • What percentage of students scored proficient or advanced on the Keystone exams this year, and how does it compare to prior years, by grade level?

Descriptive study of Star end-of-year participation and performance

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • What were the participation patterns in the three Star assessment windows throughout the year?
  • What percentage of students performed At/Above Benchmark in the three Star assessment windows?

View the published report:
End of Year Analysis of Participation and Performance on the Star Assessments at the School District of Philadelphia: 2022-23

Study of the relationship between student performance on Star and PSSA assessments

This is a knowledge-building study. The questions we’re asking include:

  • What is the correlation between Star and 2022-23 PSSA performance
  • What is the classification accuracy of Star when predicting 2022-23 PSSA performance?

Study of the relationship between student performance on Star and Keystone assessments

This is a knowledge-building study. The questions we’re asking include:

  • What is the correlation between Star and 2022-23 Keystone performance?
  • What is the classification accuracy of Star when predicting 2022-23 Keystone performance?

English Learners

Descriptive study of past trends in transition to high school outcomes for English Learners

This is a knowledge-building study. The question we’re asking is:

  • How did English Learners perform on high school outcomes such as 9th grade On-Track and graduation based on middle school factors such as PSSAs and course grades? (Tentative)

This is a Philadelphia Education Research Consortium (PERC) project. PERC is currently a partnership between Research for Action and ORE.

Descriptive study of performance on the Star computer-adaptive tests by ACCESS levels

This is a knowledge-building study. The questions we’re asking include:

  • Who took the ACCESS English proficiency test?
  • What were the patterns in ACCESS and Star performance for tested SDP students?
  • What did Star performance look like by ACCESS level?

View the published report:
English Learner Performance on the 2021-22 ACCESS, Star Reading, and Star Math Assessments

Evaluation of Quality Teaching for English Learners (QTEL) Summer Institute, professional development, and coaching

This evaluation is required by a grant. The questions we’re asking include:

  • To what extent were participants satisfied with the QTEL Institute, and did they perceive participation as being useful and beneficial to their practice?
  • How did participant knowledge of best practices and attitudes about teaching ELs change as a result of the QTEL Institute?
  • Did participant perceptions of their knowledge of best practices and attitudes about teaching ELs differ based on participant subgroup?

View the published report:
Quality Teaching for English Learners (QTEL): 2022-23 Implementation Report

Evaluation of 2022-23 Together is Better (TIB) professional development and coaching

This evaluation is required by a grant. The questions we’re asking include:

  • To what extent have teachers’ perceptions of their instructional and collaborative practices changed after participation in 2022-23 TIB programming?
  • To what extent have teachers’ opinions on the instruction of ELs in their school changed after participation in 2022-23 TIB programming?

Family Engagement

Final evaluation report of Academic Parent Teacher Teams (APTT)

This evaluation is required by a grant. The questions we’re asking include:

  • To what extent did families participate in, and were satisfied with, APTT meetings?
  • How complete was implementation of the APTT model at schools where ORE conducted observations?
Read the APTT final evaluation report published in September 2022

Health and Nutrition

Descriptive information from Eat Right Philly Alternate School Health Index (SHI) survey of health and PE teachers

This study is required by a grant. The questions we’re asking include:

  • What common challenges do schools face in implementing health and wellness activities?
  • To what extent do school responses on the SHI indicate alignment with the District Wellness Policy?
  • In which areas do schools need the most support in meeting guidelines?

Descriptive information from 2021-22 District-Wide Survey food insecurity data

This is a knowledge-building study.  The questions we’re asking include:

  • To what extent do SDP students and families report experiencing food insecurity?
  • To what extend do SDP principals report that food insecurity is a challenge to student learning?

View the published brief:
Household Food Insecurity in the School District of Philadelphia: An Analysis of District-Wide Survey Results, 2021-22

Descriptive study of 2021 Youth Risk Behavior Survey (YRBS) findings (Philadelphia sample)

This study is required by a grant. The question we’re asking is:

  • What trends exist in Philadelphia youth risk behavior topics overall and across student subgroups?

View the published report:
Youth Risk Behavior Survey: Selected Results from the School District of Philadelphia (2021)

Study of 2021 Youth Risk Behavior Survey (YRBS) mental health report (Philadelphia sample)

This study is required by a grant. The questions we’re asking include:

  • What are the year-to-year trends in suicidal ideation among Philadelphia youth, overall and across student subgroups?
  • Are experiences of Adverse Childhood Events (ACEs) associated with suicidal thoughts and attempts?
  • What protective factors are related to lower rates of suicidal behavior?

Study of 2019-20 and 2021-22 school breakfast models, participation rates, and responses to District-Wide Survey questions

This is a knowledge-building study.  The question we’re asking is:

  • How did school breakfast models and breakfast participation rates change from 2019-2020 to 2021-22?

High School Graduation and Beyond

Descriptive study of graduation pathways and student Keystone performance

This is a knowledge-building study.  The question we’re asking is:

  • What percent of high school students are on track for each graduation pathway based on 2022-23 and earlier Keystones performance? (Tentative)

This is a Philadelphia Education Research Consortium (PERC) project. PERC is currently a partnership between Research for Action and ORE.

Study of the relationship between 9th grade On-Track and graduation rates for recent cohorts

This is a knowledge-building study.  The questions we’re asking include:

  • What is the relationship between 9th grade On-Track status and on-time graduation for recent cohorts?
  • Have historic relationships between these factors persisted during District efforts to increase 9th grade On-Track rates?

View the published brief:
Ninth Grade On-Track Rates and Their Relationship to On-Time Graduation for First-Time SDP Ninth Graders in 2016-17 through 2018-19

Descriptive information from the 2021-22 Senior Exit Survey

This is a knowledge-building study.  The questions we’re asking include:

  • What post-secondary education plans do students have within one year of graduation?
  • Did disruptions due to the COVID-19 pandemic impact students’ post-secondary education plans?
  • What support systems did students have to help them navigate career and post-secondary education options?

View the published slide deck:
2021-22 Senior Exit Survey: District-Level Report

Descriptive information from the 2022-23 Senior Exit Survey

This is a knowledge-building study.  The questions we’re asking include:

  • What post-secondary education plans do students have within one year of graduation?
  • What support systems did students have to help them navigate career and post-secondary education options?

Descriptive information from the 2021-22 Career & Technical Education (CTE) follow-up survey

This is a knowledge-building study.  The question we’re asking is:

  • What do recent CTE graduates report about their post-graduation experiences, and the extent to which they are related to their CTE pathway?

View the published slide deck:
Results from the Spring 2022 CTE Graduate Follow-up Survey

Descriptive information from the 2022-23 Career & Technical Education (CTE) follow-up survey

This is a knowledge-building study.  The question we’re asking is:

  • What do recent CTE graduates report about their post-graduation experiences, and the extent to which they are related to their CTE pathway?

Descriptive information from the 2022-23 graduate follow-up survey

This is a knowledge-building study.  The question we’re asking is:

  • What do recent District graduates report about their post-graduation experiences, and the barriers and facilitators to enacting their postsecondary plans?

Evaluation of Power Skills Academy

This evaluation is a grant requirement.  The questions we’re asking include:

  • To what extent was the Power Skills Academy implemented as intended?
  • To what extent did the program support high school students in meeting the PA Department of Education’s new graduation requirements?
  • What were the participation and completion rates for the program?

Evaluation of Science Olympiad

This evaluation is a grant requirement.  The questions we’re asking include:

  • To what extent did students participate in the program?
  • What implementation barriers and facilitators were identified by coaches?
  • What feedback did students have about the program and their attitudes about STEM?

Remediation and Enrichment

Evaluation of 2023 Summer Programs

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • What were summer program enrollment/attendance rates?
  • What were staff, student, and family experiences of summer programs?

View the published report:
2023 Summer Programs in SDP: Offerings, Attendance, Survey Results, and Recommendations

School Climate

Descriptive study of monthly student attendance trends

This is a knowledge-building study.  The question we’re asking is:

  • What were the trends in 90%+ monthly student attendance in 2021-22 and 2022-23?

View the published brief (2021-22):
Student Attendance Patterns in Philadelphia, 2017-18 to 2021-22

Study of School Safety and Climate (Guardrail 1) using a mixed methods approach

This is a knowledge-building study.  The questions we’re asking include:

  • What are student responses on the key school climate issues covered in the District-Wide Survey, including school safety, relationships with staff and peers, and physical school environments?
  • How do student experiences in school safety and climate differ by school?
  • To what extent do qualitative findings from student focus groups corroborate or offer further insights on findings from quantitative data from the District-Wide Survey?

Evaluation of Schoolyard Redesign Project

This evaluation is required by a grant. The questions we’re asking include:

  • What student activities are happening on the playground during school hours and how have these activities varied before and after redesign?
  • What school staff activities are happening on the playground during school hours and how have these activities varied before and after redesign?
  • Are there any changes to school climate or achievement after playground installation?

System Trends

Descriptive information from the October 2022 enrollment snapshot

This is a knowledge-building study.  The questions we’re asking include:

  • What are the enrollment numbers for different public school sectors in Philadelphia in 2022-23 school year?
  • What are the year-over-year and long-term trends in enrollment?

View the published brief:
Philadelphia Public School Enrollment, 2022-23

Descriptive information from the 2021-22 School Progress Report on Education and Equity (SPREE)

This is a knowledge-building study.  The questions we’re asking include:

  • What is SPREE and what metrics does it include?
  • What were the patterns of school performance in 2021-22 SPREE reports?

Descriptive information from 2021-22 School Selection (final fall 2022 enrollment)

This is a knowledge-building study of a major district initiative.  The questions we’re asking include:

  • What are the enrollment patterns at criteria-based high schools over the last four years; overall and by student groups?
  • What were the enrollment patterns at the schools that used zip code priorities in 2021-22?

View the published report:
School Selection: Enrollment of First-Time 9th Graders at District High Schools, 2019-20 through 2022-23

Descriptive information from 2022-23 School Selection

This is a knowledge-building study of a major district initiative.  The question we’re asking is:

  • What were the patterns in student qualifications, application submission, admission offers received, and student acceptances of admission offers?

View the published report:
A Summary of the 2022-23 School Selection Process: Qualifications, Applications, Admission Offers, and Student Responses

Study of relationships between School Selection admission criteria and 9th grade On-Track rates

This is a knowledge-building study.  The questions we’re asking include:

  • To what degree do data points used in past cycles of School Selection predict 9th grade On-Track status at criteria-based schools?
  • Are there specific data points, combinations of data points, and/or qualification thresholds that best predict student success at criteria-based schools, or subsets of criteria-based schools?

Study of graduation rate trends over time

This is a knowledge-building study.  The questions we’re asking include:

  • What are the graduation rates for the 2018-19 9th grade cohort for different public school sectors in Philadelphia?
  • What are the longitudinal trends in graduation rates?

View the published brief:
2021-22 Four-Year High School Graduation Rates in Philadelphia

Study of District-Wide Survey School Support Staff results

This is a knowledge-building study.  The questions we’re asking include:

  • What are District support staff perceptions of school climate, school leadership, equity and inclusion, and their professional capacity?
  • To what extent do these perceptions vary by school?

Study of Education for Children and Youth Experiencing Homelessness (ECYEH) grant funded activities during 2021-22 school year

This study is required by a grant. The questions we’re asking include:

  • During the 2021-22 school year, did ECYEH approaches result in the identification of school-age students as experiencing homelessness at some point during the year?
  • How many students did ECYEH support in terms of the following services: enrollment assistance, uniforms, school supplies, and transportation assistance?
  • Was ECYEH able to implement the TEEN program for students in addition to professional development opportunities for educators, school personnel, and others?
  • How do students experiencing homeless compare to other District students on measures of attendance and achievement?

View the published brief:
Education of Children and Youth Experiencing Homelessness (Analysis of 2021-22 Data)

2023-24 and Beyond

Here are some projects we are looking forward to completing in future years!

Study of the relationship between student grades and Keystone performance

This is a knowledge-building study.  The questions we’re interested in include:

  • What is the relationship between Keystone performance and student grades for Algebra I, Biology, and Literature?
  • Does how soon after course completion the Keystone test is taken have an effect on the strength of the relationship?

Study of the relationship between Star computer-adaptive tests (CATs) and Star curriculum-based measures (CBMs)

This is a knowledge-building study.  The questions we’re interested in include:

  • What is the relationship between Star CATs and Star CBMs?
  • How accurately do the CBMs predict performance on the Star CATs?