Findings on virtual courses offered by large districts in Brookings’ educational choice research

The Brookings Institution recently released its 2014 Education Choice and Competition Index, which scores each of the 100 largest school districts (plus seven others based on their choice policies) in the United States based on a variety of factors related to choice and competition. One of the elements is “accessibility of Virtual Courses.” The rating is based on three factors:

A) Does the district have publicly available policies allowing students to enroll in a variety of virtual courses that count towards graduation or matriculation;

B) Is at least 2%* of the total student population enrolled in at least one virtual course; and

C) Are no substantial costs borne by the student or family.

Districts are rated on their yes/no answers to the questions. Each question is equally weighted and worth 1/3 of a point. The district gets a 1 if it answers all questions affirmatively, a 0 if all answers are negative, or a .33 or .67 for one or two positive answers. The full explanation of ratings is here. The research was conducted via a combination of surveys/interviews of school districts, and web searches of district sites to find online course options and policies. The researchers defined online courses as having no required onsite component, and with the teacher and student remote from one another. Therefore credit recovery classes that use online content and are taken by students in a scheduled period in a computer lab would not count.

Of the 107 districts:

  • 30 (28%) said yes to all three questions. In these districts, a variety of online courses are available to students, without costs borne by the student or family, and at least 2% of the students in the district are taking online courses.
  • 47 (44%) said yes to two questions. Of these, all districts but one have online courses available and students do not have to pay, but the district reported less than 2% of students taking online courses.
  • 19 (18%) said yes to one question. In all of these cases the responding district has online courses available, but some costs are borne by students/families. Perhaps not surprisingly, in all these districts fewer than 2% of students are taking online courses.
  • 11 (10%) said no to all three questions.

These numbers are interesting in several ways, and to me the most salient point is that nearly three-quarters of all school districts have fewer than 2% of their students taking online courses. As a point of comparison, Florida—the state with the most students taking online courses that meet the Brookings report definition—has about 10% of students taking an online course each year.** As Florida is a single case, we must be cautious about ascribing too much significance to it. It provides one data point, however, that suggests that when students are given the option to take a publicly-funded online course, and are aware of the option, many more than 2% of students will take advantage of that option.

Why, then, are the numbers so much lower in these districts than in Florida? We don’t know, but possible factors include the following:

  • Florida has a long history of online learning, so students there are more familiar with online learning than students in most other states.
  • Florida has an online learning graduation requirement, and although the requirement has only recently come into force, it has likely raised awareness of online courses.
  • Perhaps students in large districts such as those in the Brookings study take online courses at lower rates than the general population of all students.
  • The large districts surveyed may have courses available but not be communicating their availability, and students don’t know they are an option.

Students should not be forced into taking online courses (with the possible exception of a single online course being required for graduation), and it’s not clear if there is an optimum percentage of students who are choosing online courses. Still, the discrepancy between the numbers in the Brookings study, and the numbers in Florida, suggest that many students still don’t have access to, and knowledge of, online course opportunities.


* It’s not clear why the 2% threshold was chosen as the level at which a district gets credit for online courses, and an argument could be made for putting the threshold at any of a variety of levels. But for a ratings system like this a threshold has to be created, and 2% is certainly defensible.


** This is a back-of-the-envelope calculation based on the number of course enrollments in online courses, and the student population in Florida, both of which are known, and an estimate of how many students take more than one online course, which is not known.

Report on virtual learning reminds us that well-respected outlets sometimes publish uninformed articles

The Atlantic is a respected magazine that often publishes well-researched, thoughtful, influential articles on a wide variety of topics. Those positive characteristics make the publication of a poorly researched article on virtual learning—as occurred late last week—much worse than if the same article had been in a less-respected journal. The publication of such an article also serves as a reminder to digital learning practitioners, researchers, and advocates about the importance of communicating the basic facts and information clearly and often. We may think that some of these issues have been clear for years, but other people appear to be just discovering them—and sometimes getting the facts wrong.

The recent Atlantic article, Virtual Education: Genuine Benefits or Real-Time Demerits, by Jen Karetnick, contains numerous factual errors that, because of the readership, are likely to misinform a wide audience. Perhaps more disappointing is that the article is also poorly written such that in several cases logical conclusions that a reasonable reader might draw would be incorrect.

The story begins with an anecdote about a student who took multiple online courses from Florida Virtual School (FLVS) and then moved to New York, where she found that the school she entered initially would not accept the credits for the online courses. The writer suggests that the lack of transfer credits is an issue of online course quality. However, in reality, different states and districts have varying graduation requirements and discretion as to the credits that they accept even from brick-and-mortar schools. That’s the status quo everywhere for all students who move between states and has nothing to do with online schools or courses. In most cases the school enrolling the new student decides on what courses are accepted. For example, I recently heard a similar story from a colleague who this year moved with her high school-aged son from Ohio to Oregon. In Ohio he had been on track to graduate. In Oregon he was not, because of the different state requirements. In a twist relative to The Atlantic article, my colleague’s son has made up the difference in credit requirements by taking online courses and has now gotten himself back on track to graduate.

In the anecdote from The Atlantic, the student was told that the New York City Department of Education wouldn’t accept online course credits from FLVS. This appeared to be a blanket policy about all online courses, not just the ones in Florida, because FLVS is among the largest, oldest, and most-studied course providers in the country. Florida students have completed more than 2 million course credits at FLVS, and several studies have shown that those students have done well on Advanced Placement exams, state assessments, and other measures.

Karetnick further misleads readers when she notes much later – in the 19th paragraph – that in the case of the student who moved to New York, her situation was partially resolved because the student “discovered that it is up to the principal’s discretion whether to accept the online credits,” and the principal did indeed grant her credit for most courses. It was not an ideal resolution for the student, because those courses were counted as credits but not as grades (which might help with her GPA), but it was far less of an issue than the author originally suggested. Although it’s hard to believe coming from a respected magazine, it appears as if Karetnick heard the story about the credits not being accepted and thought that sounded like such a good indictment of online learning that she had to lead with it. Undoubtedly there are readers who didn’t make it down that far in the article and left believing that the student lost her online course credits.

Karetnick writes extensively on several other subjects with few citations, and she conflates known problems in education generally with problems in online education specifically. For example, in a long discussion about cheating, she seems unaware that the techniques students use to cheat in online courses include many of the same ones used by students in physical schools and classes. Karetnick notes that students say “it’s far too easy to find the answers to their tests and homework assignments by accessing Google on other devices. They’ve seen peers in other schools copy projects from the Internet, or reproduce those produced by friends who’ve taken the class already.” As all experienced educators know, these are not issues that are specific to online courses. In fact, many teachers and college professors in traditional physical schools run student essays through plagiarism software and other checks for this exact reason. If Karetnick had taken the time to talk with a single experienced online teacher, she would have found the many ways in which experienced online teachers address issues of academic integrity.

Regarding state online learning graduation requirements, Karetnick suggests that several states are backtracking and lessening such requirements—but she is wrong about the details. She writes: “New Mexico’s latest requirements, for example, state that an online- or distance-learning class can be substituted with advanced or community-college coursework.” But New Mexico has never had an online learning graduation requirement. For several years it has had a graduation requirement that could be fulfilled in one of several ways, including an advanced course or an online course. Karetnick’s suggestion that New Mexico is backtracking on an online learning requirement is factually incorrect and suggests that either her research and reporting is slipshod, or that she began the article with her ideas well formed and then sought examples to make her argument—and bent those examples to her conclusions when necessary.

Here are some other examples of this article’s poor reporting:

  • During her discussion of FLVS, Karetnick says that many virtual schools “are not accredited in the same manner that traditional public and private institutions are.” A reasonable reader might conclude that this is true of FLVS, but it is not. FLVS is accredited by AdvancED, which accredits thousands of schools, most of which are traditional physical schools.
  • Detailing the history of online learning, she writes “in 2007, Florida established a state-run online school.” Karetnick is off by about a decade as FLVS was established in the late 1990s.
  • Of online courses, she writes that “The best [online classrooms] use blended-learning techniques, where groups of students actually Skype with a live teacher and participate in group chats.” There is no doubt that student-teacher interaction and student-student interaction are positive attributes of some online courses. However, she appears to believe that these are considered “blended-learning techniques,” which exposes her lack of familiarity with basic digital learning terminology.

The above list is only partial; additional errors and misleading statements abound.

Karetnick is correct about some issues. Of online schools, she writes, “enrollment numbers are ever-changing because many of their customers are part-timers.” That is correct; we know that high rates of student mobility present challenges to schools and policymakers. Comparing online schools that have high levels of student-teacher interaction with those that don’t, she writes “The worst [online] courses, on the other hand, appear to be those that simply provide reading materials online, test the students on that content, and ask them to complete projects.” This criticism is undoubtedly true as well.

The article is accurate in several other areas, too, but about 75% of it is misleading or factually wrong. That it was published in The Atlantic means that it is likely to be read by some influential people. This unfortunate occurrence is an excellent reminder to researchers, practitioners, and all who value the spread of new ideas that we must be ever-vigilant about communicating accurate information clearly and consistently.

Valuable and accurate critiques of some digital learning efforts and policy certainly exist. The Atlantic article is not among them.

Disclosure: Florida Virtual School has been a Keeping Pace sponsor and client of Evergreen Education Group.



Digital Learning Snapshot: District of Columbia Public Schools

This is the third in a series of blog posts about digital learning innovation in school districts. See our first post, introducing the series, and the second post on Clark County, Nevada.

District of Columbia Public Schools (DCPS) has implemented digital instruction in three main areas:

  • School redesigns of a total of thirteen schools using extensive digital instruction.
  • A dozen initiatives using digital content in many subjects, across all grades levels, in all district schools.
  • Credit recovery that is offered in a blended format in which content is delivered online and students meet with teachers two to three times a week.

Beginning in fall 2012, DCPS began blended learning in eight schools using a feeder pattern (4 elementary, 3 middle, and 1 high school). The district brought in a group of outside experts to help design digital instruction in these schools, and because of the mix of schools (each making some autonomous decisions) and different grade levels, the implementations vary in significant ways.

In two of the elementary schools, students move through three instructional stations in literacy and math. One station is small group direct instruction with the teacher, one block involves working with digital content, and the third block is often focused on independent practice, such as reading or writing, or collaboration with other students. In these schools, teachers help select digital content and receive professional development from a Technology Instructional Coach who is shared by the schools. Professional development focuses on helping teachers analyze and use the data being produced by the digital content.

The redesigned middle school is focused on math instruction and is using a different approach than that used by the elementary schools. All students have a laptop that allows them to move through the content at their own pace, with support from a team of teachers. The middle school uses different digital content providers than the elementary schools, a different outside expert who advised on the instructional design, and a staffing model that is unlike any other school in the district. All students are assessed at the end of each day to determine what math topics they will focus on the next day, and how. Instruction is done directly with a teacher, with digital content used independently or with peers.

In addition to the blended school redesigns, students in grades 6–12 use online courses for remediation, credit recovery, and acceleration.

Digital content and tools

In addition to the digital content and tools used in the blended schools, a variety of initiatives across the district use digital content in math, literacy, language development, science, social studies, and world languages. Content selections are made primarily at the district level. Students access digital content from schools, homes, libraries, and other locations. Digital content includes the following:

  • Three math providers are collectively used in grades K–9 in most elementary and middle schools, for 11.5 hours per week in sessions that range from 10–45 minutes each.
  • Two English language arts content providers are used. One is used in 17 schools in grades preK–5, for up to 1.5 hours per week in 20 or 30 minute sessions. Content from the other provider is used in 27 schools in grades preK–8 for one hour per week in 20 or 30 minute sessions.
  • A world language supplement is used in grades 4–8 for 1.5 hours per week.
  • Digital textbooks are being piloted in social studies and science courses for grades 6–9 in five schools. The district has set aside funds to expand the use of digital textbooks to 30 additional schools, pending results of the pilot.
  • A learning management system is used in grades 6–12 in four schools.

Significant technology infrastructure upgrades were needed throughout the district, and especially in the blended schools, including installation of wireless network capabilities and increased bandwidth. Schools across the district, including the redesigned schools, use a variety of devices including tablets, desktop computers, and laptop computers. The elementary-level redesigned schools have students rotating through instructional models and therefore do not need devices for all students at the same time. The redesigned math program at the middle school requires laptops for all students. A total of $10 million has been spent in the past year to purchase new devices and implement a four-year refresh cycle for technology, including upgrading labs, teacher devices, and any other outdated computers across the district.

Although the district is focused on school redesigns and the use of digital content and tools for instructional reasons, in many cases technology requirements for the new national online assessments—DCPS is using The Partnership for Assessment of Readiness for College and Careers (PARCC)—are driving the device decision-making process for devices and infrastructure. The digital learning changes are able to build on, and take advantage of, those technology improvements.

Teaching and staffing

Existing classroom teachers are implementing blended learning in the redesigned schools, and 13 instructional technology coaches were hired in 2014 to support the blended classroom teachers. The instructional technology coaches work with teachers in their respective DCPS buildings to help them integrate digital content and technology tools. DCPS has a blended learning manager who is located in the Office of Teaching and Learning. DCPS believes that technology is a core part of education, and the merger of instruction and technology is essential to allowing students to have ownership in their learning.

Teacher evaluators in blended learning schools had to be retrained on evaluation techniques that were applicable to the blended learning classroom. Teachers and administrators were concerned about what impact blended learning might have on the existing evaluation system, and as a result the system was updated to address the concerns.

Budget and funding

DCPS supports its blended learning rollout through a variety of grants, including $2 million from Next Generation Learning Challenges and the CityBridge Foundation, which is working to create a system of high-performing schools in Washington, DC. Funds were allocated in the annual budget to support blended learning; no additional tax revenue was needed to implement the program. DCPS also receives a significant amount of fiscal and in-kind support from the local government and businesses that have helped to bring blended learning to life.


The redesign of the first set of blended schools is part of the larger district strategy to implement blended learning in more schools across the district. DCPS has found that it needs to add additional professional development time for teachers, to allow time for them to understand the instructional approach using digital content and tools, and also to select the content they wish to use, and become comfortable with it. In addition the district is considering how to establish routines for students transitioning between instructional modes in classrooms. As with other blended schools, moving from a traditional classroom in which students are being instructed by the teacher for most of the class period, to one in which they are partially self-directed and moving between stations, requires different classroom management techniques as well as varied pedagogical methods.


School accountability in the digital age

We have just released the second in our series of policy briefs, titled School Accountability in the Digital Age. (These policy briefs are in addition to the Keeping Pace annual report; the first policy brief examined Teaching Across State Lines.)

The new report explores the ways in which the high rates of student mobility in online schools, and the high proportion of students who enter these schools behind in accumulating credits, challenge state accountability systems. As discussed in a recent blog post, the Department of Education in Arizona has recognized these and related issues and has created a proposal to the Arizona State Board of Education to adjust some accountability measures for Arizona Online Instruction schools.

Some of the issues pertain to all schools, but are exacerbated for online schools because of their non-typical student populations. For example, among the shortcomings of the current graduation calculations for all schools are:

  • Schools receive no recognition towards graduation rate calculations for a student who is on track while at the school, but leaves prior to graduating.
  • If a student starts high school elsewhere and enters the school behind on credits, the new school gets no additional recognition for helping the student catch up.
  • A school’s four-year graduation rate will be decreased by enrolling a student who is far behind, even if the school helps that student catch up and graduate in five or six years.

Policy changes that would help address some of the current accountability system issues include the following:

Credit schools with graduating students in five or six years.

Schools should receive some points towards their state report card for graduating students in five and six years. How the credit is allotted would depend on the specifics of each state’s school performance framework.

Measure students’ progress towards graduation, especially for situations in which students switch schools.

The specific calculation would vary by state. Key factors in this approach include defining how a student is determined to be behind, on track, or ahead of pace, at time of enrollment; determining what is appropriate progress towards graduation; and determining incentives to schools for working with students who were behind at time of enrollment and getting them on track to graduate.

Change funding mechanisms to systems that minimize the impact of high student mobility.

Although funding is often perceived as an issue separate from accountability, funding and accountability are tied together in many ways. In particular, issues of student mobility are exacerbated when funding is based on a limited number of count days.

Publish data on student mobility for all schools, and consider creating a designation specific to schools with high rates of student mobility, regardless of other student demographic factors.

Student mobility data should include the number and percentage of students who enroll in the school after the start of the school year, as well as the number of students who start beyond certain dates based on a standard methodology. States that create this designation would need to consider whether they would change how the performance rating would be calculated for such schools.

Require separate reporting on online programs so that online student outcomes can be tracked.

An educational program that is serving a significant number of students with a distinct mode of instruction—as with an online school—should report on online student programs and usage separately from any other school.

Calibrate performance penalties for schools that miss targets for the percentage of students who take state assessments.

States have a compelling interest in ensuring that a high percentage of students take state assessments, but penalties should be calibrated to different levels of participation, instead of being based on missing a single target.

When students change schools, require that the sending school transfer complete student information to the receiving school quickly.

When a student changes schools, the new school should quickly have access to data such as attendance, state test results, and other important academic information about the individual student.

End counting by cohorts, and determine where students go after leaving an online school.

State reporting is too often based on total student counts; instead they should be based on the educational trajectory of each individual student. For example, states should determine where students go after leaving an online school, in categories including students who transfer into another school, those who earn a GED, those who drop out, and those who move out of state, and then make sure those designations are appropriately reflected in the calculation of graduation rate.

State accountability systems are meant to measure and report on how well schools are serving students, in ways that can be understood by policymakers, parents, students, and other stakeholders. These accountability systems tend not to measure schools well when certain factors are present, including when the school has a student population with a high rate of mobility, and when the school has a student population that has entered the school behind in terms of credits accumulated towards graduation. Situations in which students maintain progress towards their academic goals should be seen as successful, and should be distinguished from situations where students fall behind.

Implementation of the policy changes outlined above would help address these shortcomings. It would also help ensure that all schools are held accountable for advancing students during their time at the school, whether that time comprises the student’s entire education, or is a very short stop along their educational path.


Determining appropriate accountability measures for online schools: the Arizona example

In the next few days Keeping Pace is going to release a report titled School Accountability in the Digital Age: A closer look at state accountability systems and online schools, with a focus on student mobility and graduation rates. Once the report is out I’ll write about the main findings and recommendations.

Although we didn’t realize it while we were writing the report, the Arizona Department of Education (ADE) has recognized many issues similar to the ones our policy brief will explore. In a draft proposal to the Arizona State Board of Education titled Accountability Determinations for Arizona Online Instruction Schools, the ADE discusses these issues and makes several recommendations for changes to the accountability system. Arizona has online charter schools and district schools, both of which can serve full-time and part-time students statewide if they are approved under the state’s Arizona Online Instruction (AOI) program. Arizona is among the states with the most students attending online schools, with nearly 50,000 unique students attending online schools as either part-time or full-time students as of SY 2012-13.

Among the issues that ADE has identified, and its proposed solutions for accountability for districts and charter schools with virtual school programs, are:

  • The current system leaves too many AOI schools as unrated. This situation violates the intent of state education code and the conditions of the waiver the state is seeking from NCLB requirements under the federal Department of Education’s flexibility rules. The changes would increase the number of schools receiving a rating.
  • AOI schools are required to have a 95% rate at which students participate in state testing. Despite ramifications for not reaching this level, some schools do not, and they are penalized even in situations when a student has taken the state test but is associated with a physical school. ADE proposes counting all students as having taken the test for AOI accountability purposes even if the student took the test while associated with a physical school.
  • Particular attention is on increasing the important of student growth relative to proficiency, and also on graduation rate calculations. It also recognizes that virtual schools serve a mobile population with diverse set of academic goals. The proposed system would reward schools for retaining under-credited students, and also reward schools for students’ growth towards graduation. It would also reward schools for graduating students in 5, 6, and 7 years.

The ADE proposal recognizes that most AOI schools do not fit the criteria of alternative schools, but that they also have student populations significantly different than most traditional schools. As such, the ADE seeks to create a new accountability system that would apply to AOI schools that accounts for the differences in student mobility rates, under-credited students, and other factors.

An upcoming blog post will discuss the findings of our research, which looked at several other states that are at various stages of determining how to address current accountability measurement shortcomings.

WordPress SEO fine-tune by Meta SEO Pack from Poradnik Webmastera