Skip to main content
SpringerLink
Log in

Using a Regression Discontinuity Design to Estimate the Impact of Placement Decisions in Developmental Math

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

This study evaluates the effectiveness of math placement policies for entering community college students on these students’ academic success in math. We estimate the impact of placement decisions by using a discrete-time survival model within a regression discontinuity framework. The primary conclusion that emerges is that initial placement in a lower-level course increases the time until a student at the margin completes the higher-level course they were not assigned to by about a year on average but in most cases, after this time period, the penalty was small and not statistically significant. We found minor differences in terms of degree applicable and degree transferable credit accumulation between students placed initially in the lowerlevel course.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The terms basic skills, developmental, remedial and preparatory math education are frequently used interchangeably though they have different conceptual origins, imply differing educational strategies and carry some political baggage. Our preferred terms are either developmental or preparatory math.

  2. The extent to which these math courses are “remedial” depends on students’ subsequent course of study and degree objectives. Arithmetic is never credit bearing and does not count toward any degree or certificate. Pre-algebra is credit bearing but does not meet the math requirement of most certificates or AA degrees and is not transferable to a four-year institution. Elementary algebra is also credit bearing and was sufficient to meet the math requirement for an AA and for most certificates. Its credits are not transferable to a four-year institution. Intermediate algebra credits are transferable and intermediate algebra was required for some technical certificates. Recently, the minimum math requirement for an AA degree was raised to intermediate algebra. However, this change happened after the analysis period for this paper.

  3. Note that compositional changes in the sample over time only affect the validity of the RDD estimates if they change the placement decision through mechanisms other than changes in underlying ability as measured with the placement test score (since the latter is explicitly controlled in the RDD analysis).

  4. As a sensitivity check we also produced traditional cross-sectional estimates (at specific time intervals such as 12, 24, 36, and 48 months after placement) of the impacts presented in this paper. These estimates, which did not differ substantially from those presented here, are summarized in the Online Appendix; the full results are available from the authors.

  5. For a detailed presentation of the use of this test see the Online Appendix.

  6. It is worth noting that specific analyses presented in this paper used subsamples of this overall sample because not all of the students fit within the optimal bandwidths used in the RDD analyses.

  7. Pooling data across colleges was not possible due to the fact that course sequences, placement score cut points, multiple measures, and other major analytical and policy variables varied across the colleges.

References

  • Allison, P. D. (2005). Fixed effects regression methods for longitudinal data using SAS. Cary, NC: SAS Institute.

    Google Scholar 

  • Angrist, J. D., & Lavy, V. (1999). Using Maimonides’ rule to estimate the effect of class size on scholastic achievement. Quarterly Journal of Economics, 114(2), 533–575.

    Article  Google Scholar 

  • Bailey, T. (2009). Challenge and opportunity: Rethinking developmental education in community college. New Directions for Community Colleges, 145, 11–30.

    Article  Google Scholar 

  • Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29, 255–270.

    Article  Google Scholar 

  • Bettinger, E. P., Boatman, A., & Long, B. T. (2013). Student supports: Developmental education and other academic programs. The Future of Children, 23(1), 93–115.

    Article  Google Scholar 

  • Bettinger, E., & Long, B. T. (2009). Addressing the needs of under-prepared students in higher education: Does college remediation work? Journal of Human Resources, 44, 736–771.

    Article  Google Scholar 

  • Boatman, A. & Long, B. T. (2010). Does remediation work for all students? How the effects of postsecondary remedial and developmental courses vary by level of academic preparation. National Center for Postsecondary Research Working Paper.

  • Boylan, H., Bliss, L., & Bonham, D. (1994). Characteristics components of developmental programs. Review of Research in Developmental Education, 11(1), 1–4.

    Google Scholar 

  • Boylan, H., Bliss, L., & Bonham, D. (1997). Program components and their relationship to students’ performance. Journal of Developmental Education, 20(3), 1–8.

    Google Scholar 

  • Calcagno, J. C., & Long, B. T. (2008). The impact of remediation using a regression discontinuity approach: Addressing endogenous sorting and noncompliance. Working Paper 14194. Cambridge, MA: National Bureau of Economic Research.

  • Caliendo, M., Tatsiramos, T., & Uhlendorff, A. (2013). Benefit duration, unemployment duration, and job match quality: A regression-discontinuity approach. Journal of Applied Econometrics, 28, 604–627.

    Article  Google Scholar 

  • California Community College Chancellor’s Office. (2011). Matriculation Handbook. Retrieved from http://www.cccco.edu/Portals/4/SS/Matric/Matriculation%20Handbook%20(REV.%2009-2011).pdf.

  • Clotfelter, C. T., Ladd, H. F., Muschkin, C. G., & Vigdor, J. L. (2014). Developmental Education in North Carolina Community Colleges. Educational Evaluation and Policy Analysis,. doi:10.3102/0162373714547267.

    Google Scholar 

  • Fan, J., & Gijbels, I. (1996). Local polynomial modeling and its applications. London: Chapman & Hall.

    Google Scholar 

  • Fong, K., Melguizo, T., Bos, H., & Prather, G. (2013). A different view on how we understand progression through the developmental math trajectory. Policy Brief 3. California Community College Collaborative. Rossier School of Education, University of Southern California. http://www.uscrossier.org/pullias/research/projects/sc-community-college/.

  • Fong, K., Melguizo, T., & Prather, G. (2015). Increasing success rates in developmental math: The complementary role of individual and institutional characteristics. Research in Higher Education. doi:10.1007/s11162-015-9368-9.

    Google Scholar 

  • Gamse, B. C., Jacob, R. T., Horst, M., Boulay, B., Unlu, F. (2008). Reading First impact study: Final report. NCEE 2009-4038. Washington DC: National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

  • Greenwood, M. (1926). The natural duration of cancer (Vol. 33, pp. 1–26)., Reports on Public Health and Medical Subjects London: Her Majesty’s Stationery Office.

    Google Scholar 

  • Ham, J., & LaLonde, R. (1996). The effect of sample selection and initial conditions in duration models: Evidence from experimental data on training. Econometrica, 64(1), 175–205.

    Article  Google Scholar 

  • Imbens, G., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635.

    Article  Google Scholar 

  • Lazarick, L. (1997). Back to the basics: Remedial education. Community College Journal, 68, 11–15.

    Google Scholar 

  • Lesik, S. A. (2007). Do developmental mathematics programs have a causal impact on student retention? An application of discrete-time survival and regression-discontinuity analysis. Research in Higher Education, 48(5), 583–608.

    Article  Google Scholar 

  • Martorell, P., & McFarlin, I, Jr. (2011). Help or hindrance? The effects of college remediation on academic and labor market outcomes. The Review of Economics and Statistics, 93(2), 436–454.

    Article  Google Scholar 

  • McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: a density test. Journal of Econometrics, 142(2), 698–714.

    Article  Google Scholar 

  • Melguizo, T., Bos, H., & Prather, G. (2011). Is developmental education helping community college students persist? A critical review of the literature. American Behavioral Scientist, 55(2), 173–184.

    Article  Google Scholar 

  • Melguizo, T., Bos, H., & Prather, G. (2013). Are community colleges making good placement decisions in their math trajectories? Policy Brief 1. California Community College Collaborative. Rossier School of Education, University of Southern California. http://www.uscrossier.org/pullias/research/projects/sc-community-college/.

  • Melguizo, T., Hagedorn, L. S., & Cypers, S. (2008). The need for remedial/developmental education and the cost of community college transfer: Calculations from a sample of California community college transfers. The Review of Higher Education, 31(4), 401–431.

    Article  Google Scholar 

  • Melguizo, T., Kosiewicz, H., Prather, G., & Bos, J. (2014). How are community college students assessed and placed in developmental math? Grounding our understanding in reality. Journal of Higher Education, 85(5), 691–722.

    Article  Google Scholar 

  • Moss, B., & Yeaton, W. (2006). Shaping policies related to developmental education: An evaluation using the regression-discontinuity design. Educational Evaluation & Policy Analysis, 28(3), 215–229.

    Article  Google Scholar 

  • Moss, B., Yeaton, W., & Lloyd, J. E. (2014). Evaluating the effectiveness of developmental math by embedding a randomized experiment within a regression discontinuity design. Educational Evaluation & Policy Analysis, 36(2), 170–185.

    Article  Google Scholar 

  • Murnane, R. J., & Willett, J. B. (2011). Methods matter: Improving causal inference in education and social science research. Oxford: Oxford University Press.

    Google Scholar 

  • National Center for Public Policy and Higher Education & Southern Regional Education Board. (2010). Beyond the Rhetoric: Improving College Readiness through Coherent State Policy. Washington, D.C.

  • Ngo, F., Kwon, F., Melguizo, T., Bos, H., & Prather, G. (2013). Course placement in developmental mathematics: Do multiple measures work? Policy Brief 4. California Community College Collaborative. Rossier School of Education, University of Southern California. http://www.uscrossier.org/pullias/research/projects/sc-community-college/.

  • Ngo, F., & Kwon, W. (2015). Using multiple measures to make math placement decisions: Implications for access and success in community colleges. Research in Higher Education, 56(5), 442–470.

    Article  Google Scholar 

  • Robinson, J. P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33(3), 267–292.

    Article  Google Scholar 

  • Schneider, M., & Yin, L. (2011). The hidden costs of community colleges. Washington, DC: American Institutes for Research. Retrieved from http://www.air.org/files/AIR_Hidden_Costs_of_Community_Colleges_Oct2011.pdf.

  • Schochet, P. A. (2006). Regression discontinuity design case study: National evaluation of early reading first. Princeton, NJ: Mathematica Policy Research.

    Google Scholar 

  • Scott-Clayton, J., & Rodriguez, O. (2015). Development, discouragement, or diversion? New evidence on the effects of college remediation. Education Finance and Policy, 10(1), 4–45.

    Article  Google Scholar 

  • Singer, J., & Willett, J. (1993). It’s about time: Using discrete-time survival analysis to study duration and the timing of events. Journal of Educational Statistics, 18(2), 155–195.

    Article  Google Scholar 

  • Singer, J., & Willett, J. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.

    Book  Google Scholar 

  • Strong American Schools. (2008). Diploma to nowhere. Retrieved from http://www.edin08.com.

  • Urquiola, M., & Verhoogen, E. (2009). Class-size caps, sorting, and the regression-discontinuity design. American Economic Review, 99(1), 179–215.

    Article  Google Scholar 

  • Zuckerman, I. H., Lee, E., Wutoh, A. K., Xue, Z., & Stuart, B. (2006). Application of regression-discontinuity analysis in pharmaceutical health services research. Health Services Research, 41, 550–563.

    Article  Google Scholar 

Download references

Acknowledgments

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A100381 to the University of Southern California. Additional support was received from an internal grant from the Advancing Scholarship in the Humanities and Social Sciences (ASHSS) Initiative of the University of Southern California, Office of the Provost. We would first like to thank Bo Kim for exceptional research assistance. Special thanks to Will Kwon and Kristen Fong for providing support in replicating the results for other colleges, and to Holly Kosiewicz for insightful feedback. The manuscript benefited substantially from the comments of the following members of the advisory committee to this project: Paco Martorell, Sarah Reber, Lucrecia Santibanez, Juan Esteban Saavedra, and Gary Painter. Lastly, we want to thank the Los Angeles Community College District, its research department, its math faculty, and its students, for their active participation in this research project.

Disclaimer

The views contained herein are not necessary those of the Institute of Education Sciences.

Author information

Authors and Affiliations

  1. Rossier School of Education, University of Southern California, Los Angeles, CA, USA

    Tatiana Melguizo & Federick Ngo

  2. American Institutes for Research, San Mateo, CA, USA

    Johannes M. Bos & Nicholas Mills

  3. Los Angeles Community College District, Los Angeles, CA, USA

    George Prather

Authors
  1. Tatiana Melguizo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johannes M. Bos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Federick Ngo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicholas Mills
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. George Prather
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tatiana Melguizo.

Additional information

George Prather—retired from Los Angeles Community College District.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 3100 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Melguizo, T., Bos, J.M., Ngo, F. et al. Using a Regression Discontinuity Design to Estimate the Impact of Placement Decisions in Developmental Math. Res High Educ 57, 123–151 (2016). https://doi.org/10.1007/s11162-015-9382-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-015-9382-y

Keywords

Search

Navigation