Featured Research

Posts from 2015

Leading with Race in the Pacific Northwest

Last summer, I interned with Home Forward, the Portland area housing authority. The project I was hired on to help with was ambitious: I was tasked with creating an “Equity Platform” that defined equity for the agency and included suggestions for how to make equity more central to the agency’s work. Home Forward aspired to be more than a housing authority; in their words, they wanted to provide their residents with not just a home, but also “a path to success.” Home Forward already had a range of institutional practices in place aimed at equity: a committee for inclusive hiring practices and a fifteen-dollar starting wage, among others. But the agency wanted to expand these efforts and formalize them based on a specific platform.

In order to make recommendations to Home Forward, I interviewed employees of several other public agencies in the region. I found that over the last ten years, many had launched their own platforms that focus on equity through a specifically racial lens. Some of the activities that these agencies pursued include racial bias trainings for new hires, analyses of the ways in which agency policies affect people of color, and the inclusion of minority communities in decision-making processes. To these agencies, focusing their approach to equity on race was justifiable simply by reviewing the numbers. People of color are disproportionately represented in statistics on child poverty rates, income insecurity, low educational achievement, and unemployment rates, among many others.

A distinction soon became apparent between this race-focused approach and the approach that Home Forward appeared to be pursuing. My host agency envisioned a plan that ensured they could provide “paths to success” without leaving anybody behind. The issue of race was important to them, but rather than declare racial equity as an explicit goal, or use it as a framework for their practices, they appeared to see it as a byproduct.

Much of the inspiration and guidance for the focus on race in developing equity policies and platforms comes from the Government Alliance on Race and Equity (GARE), a project run out of the Haas Institute for a Fair and Inclusive Society at the University of California, Berkeley.

Julie Nelson, GARE’s Director, has been the point-person for many of the projects in the Pacific Northwest, including the City of Seattle’s Race and Social Justice Initiative (RSJI). Started in 2004, RSJI has ensured that city employees receive training on racism and bias and has begun a “Community Roundtable” that convenes grass-roots organizations for input on city policies. RSJI’s work has led to several citywide changes, including increases in contracts with women and minority-owned businesses, and civic engagement in under-served minority neighborhoods.

Nelson describes GARE’s approach as “leading with race.” The motivation for this emphasis is GARE’s view that “colorblind” social policies put in place decades ago have not succeeded in eliminating institutional racism. On the contrary, people of color have historically been marginalized by these policies. Examples include race-neutral mortgage-lending practices, which have resulted in anti-Black discrimination, and race-neutral rules for patients with Medicare and Medicaid, which place disproportionate burden on people of color.[1]

GARE operates on the assumption that leading with race is the best way to achieve better results for society as a whole.

“When institutions are broken, they are broken for everyone,” Nelson said. “Institutional problems hurt everyone, but people of color are hurt first and hardest.” Issues faced by these communities of color, therefore, function as a sort of early warning that institutions are failing.

john a. powell, the Haas Institute’s Director, uses the term “targeted universalism” to describe this framework. The term refers to policies that are ultimately intended to improve broad outcomes for society as a whole, but which are designed with more immediate focus on obstacles preventing minority populations from realizing their goals. As powell says in his 2009 piece The Importance of Targeted Universalism:

Targeting within universalism means identifying a problem, particularly one suffered by marginalized people, proposing a solution, and then broadening its scope to cover as many people as possible. It sees marginalized populations in American society as the canary in the coal mine...It recognizes that problems faced by particular segments of American society are problems that could spill over into the lives of everyone, just as the Lower Ninth Ward was not the only part of New Orleans to suffer in the wake of Katrina. Likewise, the subprime credit crisis did not end in poor, urban communities, but has spread far beyond and has been felt throughout the global economy.

The “Ban the Box” initiative is a good example of how a campaign that began with a focus on racial justice can lead to positive outcomes for a broader population. The initiative calls for employers to eliminate questions about a candidate’s criminal history from initial job applications; defenders of racial justice and formerly incarcerated individuals started the movement in 2003, and it spread widely in West Coast governments and around the country. Critically, despite its focus on communities of color, the ban the box movement benefits all formerly incarcerated individuals, whether they are white or of color.

As the summer progressed, I began to understand GARE’s implementation processes and how Home Forward might apply them in crafting their own platform. GARE provides trainings on individual, institutional and structural racism, as well as concepts of implicit and explicit bias. While working with GARE, many agencies create racial equity tools that obligate policy makers to not only examine a policy in terms of the costs and benefits to people of color, but also to be aware of who is involved in the decision-making process.

For instance, in 2012, Multnomah County produced a toolkit for implementing racial equity practice called the “Equity and Empowerment Lens.” The document is meant to guide an agency through the process of assessing, analyzing, and ultimately improving equity practices. Examples of specific activities include questions prompting policy makers to consider the impact their policies have on communities of color; planning guides on how best to include community partners in their work; and suggestions for defining, measuring and reporting equity goals and their results.

When asked how their Equity and Empowerment Lens is put to use, Multnomah County employees point to the Cully Weatherization 2.0 project as an example. For this project, the County, the City of Portland, and several environmental organizations partnered with NGOs representing communities of color in the low- and moderate-income neighborhoods of northeast Portland. The project provided job training and employment to local residents, while “weatherizing” one hundred local homes. Weatherization means preserving affordable housing, while making homes more resilient to climate change, and healthier places to live. The project was designed within a conscious equity frame, in which project decisions were made with community input, and the power structure of the project’s implementation were discussed throughout.

GARE’s involvement in the region has also led to new departments, offices, and bureaus being created: the Office of Equity and Human Rights at the City of Portland; the Multnomah County Office of Diversity and Equity; the Seattle Race and Social Justice Initiative; Metro’s Diversity, Equity, and Inclusion wing. To different degrees, these offices hold their host institution accountable for equity practices in human resources, procurement, policy design, and policy implementation.

As I learned more about these kinds of ways in which other agencies implemented equity practices with a focus on race, it was clear that my supervisors remained noncommittal. At meetings and in emails, they were careful to stress the “social” part of our equity planning while shying away from emphasis on race. The majority of Home Forward residents and voucher holders are white, so Home Forward did not want to be seen as abandoning them.

Home Forward employees were quick to recognize GARE’s success, and completely comfortable acknowledging both the region’s gaping racial disparities and the historical roots of racism in Oregon. Nonetheless, the idea of emphasizing race in the development of equity practices was met with discomfort; it appeared that, rather than seeing the ways in which a focus on race can generate policy solutions for a broader population, Home Forward saw instead the risk that they might simply alienate white residents and constituents.

Throughout the Portland area, GARE continues to seek out new institutions for partnerships. There is a sense of momentum and excitement around the new initiatives in the city’s agencies – even those whose initiatives have stalled or fallen behind. The bureaucratic process can be slow to create large-scale changes in communities and most of the visible successes so far have been small. It is true that small-scale change in the way a region is governed can add up over time, but it remains to be seen whether these agencies can approach the problem with the urgency it demands, and whether Home Forward will choose to lead with race.

__________

Nereida Heller is a 2016 Master of Public Policy candidate at the Goldman School of Public Policy. This is her first article for Policy Matters Journal, where this was originally posted.

The Class Size Debate: What the Evidence Means for Education Policy

“The days in which lawmakers support schools that are somehow good enough for someone else’s children, but not for their own – those days must be over.” - Arne Duncan, U.S. Education Secretary, January 12, 2015

Why class sizes?

The more students a teacher is responsible for, the harder it is to teach. That’s hard to argue with. Yet the research and policy around this idea is surprisingly controversial. We can say with confidence that smaller class sizes improve grades for younger learners. But the answers to deeper questions are less clear.

The issue of class sizes is a live policy debate. Teachers unions from Oakland, California to Auckland, New Zealand are advocating for smaller class sizes. A call for smaller class sizes was second only to higher salaries for the LA teachers union at recent protests. The president of the National Education Association, Lily Eskelsen García, attributes fighting for smaller class sizes as her motivation for becoming an education activist. 

The idea of smaller class sizes is strongly supported by teachers and the general public. A national survey of 50,000 Americans found that reducing class sizes was perceived to be the best way to reform schools. And yet when public budgets are tight, class sizes are quick to grow. California’s class size cap increased substantially following the Great Recession.

While there is evidence that smaller class sizes improve student learning, the magnitude of the impact must be weighed against the impact of other reforms. For example, would paying higher salaries to retain a high-quality teacher workforce be money better spent? 

The question of which students would benefit is also important: would smaller classes improve high achievers’ learning more than, say, students who are still catching up? Or would the gains in learning be spread out broadly among the children in the class? 

Jessica Tyson, a history and English teacher at Oakland Technical High School, said that the issue is more than just the right size of an individual classroom.  It’s also about making room for better lesson preparation. 

“Teaching fewer hours and having that extra hour in the day to collaborate, plan, grade and reflect on what students are doing would be very helpful to me,” Tyson said. 

“U.S. teachers spend more time in front of students by a significant margin than in other developed countries,” she said. “I was struck by how wonderful [it] would be … if I could just teach slightly less, how much more I could do.”

The Evidence on Class sizes

To answer the question of how much difference smaller classrooms make for student learning, we need good data and strong analytic methods that can isolate the effect of class sizes. Researchers do not want to accidentally pick up other effects such as differences in school resources, for example, or difference in students’ characteristics or parental involvement. 

Two of best studies we have on this issue – Project STAR and a 1999 analysis of Israeli schools by economists Joshua Angrist and Victor Lavy – address these problems. They both show a positive effect from smaller school sizes in the first few years of school. I discuss this research in greater detail below. 

Placing children in random class sizes: Lessons from Project STAR

Much of what we know about class sizes comes from an experiment called Project STAR (also known as the Tennessee Study). From 1985 to 1989 11,600 Tennessee students from kindergarten to third grade were randomly assigned to three class-size categories. The three class sizes were 13–17 students, 22–25 students and over 25 students. 

The results were strong. An average student assigned to the smallest classes had a reading score nearly 8 percent higher than students in the medium-sized classes. The smaller-class students, on average, achieved 9 percent higher math scores. (See the paper here.) 

Students in smaller classes who completed high school were more likely to take college-entrance exams than students assigned to medium or large classes. The effects are even stronger for minority and less affluent students. 

Education economists Alan Krueger and Diane Schanzenbach calculate that, based on Project STAR’s results, reducing class sizes from 22 to 15 students has a 5.5 percent return in annual benefits. This takes into account students’ increased lifetime earnings.

Project STAR generated the best data we have on class sizes, but findings should be interpreted carefully. First, the experiment was conducted in larger urban schools, so results may not be as strong for smaller suburban schools. Second, it is hard to believe that the students were completely randomly assigned; it’s easy to imagine some motivated parents lobbied to move their children to the smaller classes. Finally, Tennessee is a much below-average state for education. This means the Project STAR students may have benefited from the class-size reductions more than students in higher-performing states might. 

This study also only looks at the effects on students who were in small classes in their first few years of school. Stanford economist Eric A. Hanushek and his colleague, Steven Rivkin, downplay the conclusion that many researchers draw from Project STAR – that smaller classes are a cost-effective policy to improve learning. In a 2006 paper emphasizing teacher quality over class sizes they are skeptical. “In only 40 out of 79 schools,” they write, “did the kindergarten performance in the small classroom exceed that in the regular classroom.” This is better than random (which would be an improvement in about 26 out of the 79 schools). Nevertheless, they argue, the results show too weak an effect to justify wholesale change to class sizes.

Despite Hunushek and Rivkin’s reservations, Project STAR indicates room, on average, for improving test scores for children ages 5 to 8 in class sizes larger than 17 students. If real-world results replicate those found in the STAR project, we can expect higher student achievement by moving towards smaller classes. 

Maimonides’ Rule: What a 12th Century rabbinic scholar can tell us about class sizes

Because the random allocation of students found in Project STAR is so rare, education researchers have found creative ways to mimic a randomized experiment. MIT professor Joshua Angrist and University of Warwick professor Victor Lavy use the unique education policy of modern Israel to study class sizes. The researchers harness Israel’s rule that public school classrooms must not have more than 40 students, inspired by 12th Century rabbinic scholar Maimonides who advocated class sizes smaller than 40 students. 

After 41 students are enrolled, the class splits into two. This continues after each multiple of 40. Class size is therefore related to the random chance of how close school enrollment is to a multiple of 40 and less related to confounding variables. Socioeconomic characteristics schools are controlled for.  

For 5th grade students, Angrist and Lavy find a 10-student increase in class size explains a 6.5 percent drop in the average student’s reading comprehension scores and 4.5 percent drop in math scores. Angrist and Lavy do not find statistically significant effects of class size for third graders, possibly due to test-score training that year. The 5th grade effects of class size on achievement are strong, giving further evidence that class sizes matter. The non-effects and weak effects of class size on 4th and 5th graders highlights some uncertainty in the literature and data.

The heart of the academic debate on class sizes  

Significant and insignificant results pepper the class size literature. Karen Akerhielm, a graduate of Yale’s economics PhD program, finds a 10-student reduction in class sizes explains approximately a 5 percent improvement in science and history test scores. Stanford economist Caroline Hoxby, in a 2000 paper, cannot find a relationship between school districts’ test scores and estimated class sizes. She also cannot find any evidence that the effect is stronger for schools with a higher proportion of low-income or African American students. This is in contrast to the strong effects found in the STAR project. 

Conflicting results such as these in the literature – and differing opinions on how to weight the findings of each paper in meta-analyses – have meant that two of the most prominent researchers in the field, Hanushek and Krueger, draw different conclusions. In the 2002 book The Class Size Debate Hanushek writes, “despite the political popularity of overall class size reduction, the scientific support of such policies is weak to nonexistent.” In the same book Krueger (2002) writes, “the strongest available evidence suggests a connection” and that reducing class sizes from 22 to 15 students per teacher has a positive benefit-cost-ratio at a real discount rate lower than 6%.  

While this academic debate over the methods and results of experiments on class size reduction continues, the issue has lost much of its momentum in education policy circles. Much of this goes back to the related debate about what kind of reforms produce the biggest impact for the money spent. 

Eric A. Hanushek, for example, argues that focusing on teacher quality over class sizes yields a better return on one’s investment. In an influential 2005 paper on factors that influence student achievement, Hanushek compares reducing a class by ten students to switching the teacher with a moderately better replacement.

“The effects of a costly ten-student reduction in class size are smaller,” Hanushek says.

Hanushek is not claiming that smaller classes are ineffective at improving achievement, just that it may not be the right priority. The verdict is still out whether an increase in ‘teacher quality’ is an appropriate comparison in terms of cost and amenability to policy changes. When we talk about improving teacher quality rather than decreasing class sizes, are we talking about higher salaries to attract and retain better teachers? Are we talking about higher credential requirements for new teachers (which has been shown to be ineffective)? Are we talking about more on-the-job training? Teacher quality means different things to different people; this suggests caution should be taken when comparing teacher quality with class sizes.   

In short…

Despite strong arguments on both sides of the debate, there are reasons why we should take small class sizes seriously. The methodologically strongest experiment, the STAR experiment, shows strong results. Positive results are commonly found in the literature. Private schools reveal parents’ and teachers’ preferences for smaller student-teacher ratios. School sizes are large in many U.S. school districts, and the problem may be even worse than the official statistics.

“Most of the class sizes I’ve had have been around 28 people,” Oakland teacher, James Malamut, said. “When I worked for the School District of Philadelphia it was 31 on paper, but there would be times when it was closer to 35. They’d sneak a few kids in there unofficially.” 

When teachers repeatedly raise an issue, policymakers should listen. Teachers close to the actual practice of teaching strongly believe in reducing the student-to-teacher ratio. A 2007 survey found that 81 percent of American teachers would prefer smaller class sizes over higher salaries.

“I think it would have a really significant impact,” Malamut said. “I’d be able to give more individualized attention in the class. I’d have more time for preparation. And when grading I’d be able to give more individualized feedback.”

The evidence on the relationship between class size and achievement is not conclusive but there is good reason to believe that smaller classes could be beneficial to students in the early years of primary school and especially for minorities and low-achieving students. Whether it follows that class-size reductions are poor value-for-money is less certain. Reducing class sizes, particularly for 6- to 14-year-olds where the evidence is strongest, should be considered – alongside teacher training, salaries, accreditation, and collaboration initiatives ­– as a serious albeit expensive policy option to improve educational outcomes and reduce educational outcome disparities. Does class size matter? It certainly does.

__________

Darian Woods is a Master of Public Policy candidate 2016 at the Goldman School of Public Policy at University of California, Berkeley.

This article was originally posted on PolicyMatters Journal, a publication put forth by the graduate students of GSPP.

Journalism for Social Change

An analysis of the first Massive Open Online Course dedicated to the practice of solution-based journalism.

INTRODUCTION

In March 2015, Journalism for Social Change transitioned from an on-the-ground class – teaching graduate students how to use solution-based journalism to positively impact the United States child welfare system – into a MOOC, or Massive Open Online Course.

Results after two offerings of the class suggest that the Journalism for Social Change (J4SC) MOOC was successful in delivering journalism instruction; informing a disparate range of students about U.S. domestic policy focused on vulnerable children; and in leading students to produce high-quality, solution-focused journalism.

They also point to barriers inherent in education delivered in a Massive Open Online Format; shortcomings in the design of learning elements; and issues in retaining engagement over the seven weeks that each online class was offered.  

But, before launching into a discussion of the outcomes, it is important to discuss J4SC’s history as a brick-and-mortar class, and how its learning goals were translated for the eventual MOOC.

HISTORY

In the spring of 2012, Fostering Media Connections, a journalism non-profit based in San Francisco, launched a class called Journalism for Social Change [1] at the University of California, Berkeley.

The course, which is currently offered at the Goldman School of Public Policy, attracted graduate students from Goldman, the School of Journalism and the School of Social Welfare. Students were trained on what solution-based journalism is, how it can be applied to the field of child welfare, and were then tasked with producing solution-based news stories in the medium of their choice for publication in an online news site published by Fostering Media Connections called The Chronicle of Social Change.

Every week students heard lectures by top experts in journalism, public policy and child welfare research. The course was immediately followed by a paid summer fellowship for top students from the respective schools – who spent 10 weeks covering the foster care system on state and federal levels.

In spring 2013, Journalism for Social Change came back with a broader and more ambitious focus. The course and subsequent fellowship examined child maltreatment as a public health issue and examined ways journalism can be used to hold those charged with protecting children accountable.

In addition to new subject matter, the program expanded to the University of Southern California’s Sol Price School of Public Policy with graduate students who hailed from Price, the Annenberg School for Journalism and the School of Social Work.

In fall 2013, a variant of the program was taught to undergraduates in San Francisco State University’s Department of Journalism.

By 2014, J4SC had trained nearly 150 students in the classroom, and its parent organization, Fostering Media Connections, had hosted more than a dozen summer fellows.

Those students and fellows produced hundreds of stories that appeared in The Chronicle of Social Change to then be followed by top-tier media outlets. This sustained coverage has repeatedly influenced public policy.

Student coverage that appeared in The Chronicle of Social Change sparked follow-up stories and segments in The National Journal, The Associated Press Roll Call, The San Francisco Chronicle, The Contra Costa Times, San Jose Mercury News, KPIX TV, KGO Radio News, KFBK Radio, The Sacramento Press, EdSource and Witness LA among others.

In California, student coverage has been instrumental in preserving educational provisions for foster youth in the state budget, and strengthening services available to transition-aged youth as California rolled out extension of care to age 21. On the federal level, J4SC fellows detailed the unintended barriers to the educational achievement of foster youth created by the Federal Educational Rights and Privacy Act (FERPA), which coincided with the introduction of legislation to amend the law. In 2013, President Barrack Obama signed that legislation into law.

And most recently, student coverage on the application of predictive risk modeling in assessment of the risk of child abuse has garnered follow up coverage in far-flung publications including Forbes.com and Bloomberg News among others. This coverage has helped drive a high-level policy conversation about the use of “Big Data” in child abuse prevention.

In the winter of 2015, the designers of J4SC would go on to launch an online variant of the course. It was quite possibly the first MOOC dedicated to solution-based journalism in existence.

J4SC BECOMES A MOOC

In June 2013, U.C. Berkeley’s provost sent out a request for proposals for a new project, initially bankrolled by the school and Google, called MOOCLab.

Berkeley had signed a contract with a consortium of elite universities partnering to provide courses on the edX [2] online learning platform, and was intent on attracting a handful of classes to compete for grants to build online variants of brick-and-mortar offered by MOOCLab. Journalism for Social Change class was one of three to be awarded a grant to become a MOOC.

In January 2014, the J4SC team began working alongside the Berkeley Resource Center for Online Education (BRCOE) and the California Social Work Education Center (CalSWEC) [3] to build learning modules and map out a plan to evaluate student engagement in the course.

The learning goals of the course spanned comprehension of what solution-based journalism is to instruction on how to report a news story and finally mastery of the major themes in child welfare.

THE CHALLENGE: TRANSLATING LEARNING GOALS TO THE ONLINE WORLD

MOOCs, or “Massive Open Online Courses,” became popular and available en masse circa 2012, with a strong initial focus on STEM (science, technology, engineering and math) courses. This was not coincidental; any course with thousands of students enrolled cannot be heavy in instructor-student interaction.[4] In fact, much of the initial attention among the inner circle of those teaching or providing MOOCs was on automating teaching and learning [5] in STEM courses. As a greater array of subjects began to appear as MOOC offerings, questions surrounding how to teach non-STEM courses began to seed discussions about effective teaching methods for humanities courses at scale. [6]

Enter J4SC, a course seeking to inspire civic engagement in its students and based on an interdisciplinary humanities foundation. While some best practices for teaching things such as writing methods had emerged, there had not yet been a MOOC asking students to apply journalism skills to highlight and impel policy solutions to endemic problems facing American children. Additionally, no other MOOC had offered the opportunity and incentive to students of having their final project, which in the case of J4SC was a solution-based article, published.

These factors make the J4SC MOOC a very interesting case because they both address student motivation and the notion of service learning, grounding the often-lofty study of humanities in an applied, pragmatic and progressive experience. For the burgeoning field of MOOC research, the question of motivation and J4SC’s design was most important. Student retention and engagement are the two most frequently cited problems in any course. So the question of whether, or to what extent, the course structure acted to intrinsically or extrinsically motivate students to persist through the final project demanded further study.

Converting the J4SC face-to-face course into an online course open to the masses was guided by several factors: funding constraints, purpose to serve potentially thousands of students and technical limitations of required platforms.  

The course conversion process was underwritten by a grant, which pre-assigned funds for course development. As video production is costly, and the instructional content of MOOCs is weighted heavily in favor of audio-visual media, there was little funding remaining for other non-text instructional media (e.g., interactive infographics or simulations).

Second, the course was offered via the edX consortium,[7] who have developed their own proprietary Learning Management System of the same name. The structure of the edX LMS in part determined the structure of the online J4SC course. The LMS is built so that concepts are explored laterally, with different types of content (e.g., videos, quizzes, discussions, assignments) all added in click-through frames to each specific topic. The LMS also impacted the way students could, or could not, be assessed or interact as peers. The instructor thus went through a development process of adjusting his materials to the communication medium. 

Finally, research on student persistence in MOOCs has shown that the shorter the course duration, the more likely that students will complete it. [8] Like advice from course developers resulted in a compression of the total amount of content in and duration of the course. It also reduced the amount of instructor interaction, both automated and manual. The course was organized into seven units, corresponding to seven weeks, with each week beginning with lecturettes, readings, assignments, quizzes, or guest speaker videos. Because MOOCs can have thousands of students enrolled, the instructor also had to choose teaching methods heavily leveraging peer-to-peer interaction and methods that reduced didactic feedback (e.g., grading written assignments or offering personalized critiques).

Given a different platform and a narrower intended audience, other types of rich instructional media and online teaching pedagogies could bolster the content.

EVALUATION METHODS

J4SC was first offered as a small, private online course (SPOC) on the edX Edge platform. This more-intimate setting allowed the instructor to test student reactions to the course design and work out kinks with the technology. The MOOC began just after the SPOC had closed. Multiple methods were employed to measure the impact of the course on students in both iterations of the course.

Pre- and post-course surveys: For the SPOC, 77 students completed the pre-course survey and 12 completed the post-course survey. Both pre- and post-course surveys contained several attitudinal items that measured respondents’ self-efficacy, personal motivation and comfort with online learning (see Figure 1). With such a small sample size, it was not possible to check for significant demographic differences between those who completed and did not complete the post-course survey. 

In the MOOC, however, numbers were larger; 1,244 students completed the pre-course survey and 72 completed the post-course survey. Those who completed the post-course survey had significantly higher ratings of self-motivation and personal responsibility at the start of the course (see Figure 2).

Qualitative coding: The course assistant assessed all assigned discussion posts for quality on a three-point scale. This scale matched the rubric students were given for self-evaluations (see Figure 3).

Student reflections: The final assignment in the course was to post to the discussion board with reflections on J4SC. The course assistant read each of these, looking for common themes in the most substantive posts.

Limitations: Additional usage data for respondents was not easily available from edX. In the future, it would be useful to have these measures (e.g., course completion or time spent on online materials) to develop measures of engagement.

EVALUATION        

SPOC survey results

Students in the small private online course (SPOC) were recruited through Listservs, Twitter and Facebook, focusing largely on audiences connected to social work. It is thus not surprising that two-thirds began the course with a background in child welfare. Nearly equal shares of respondents rated themselves beginners (35%), intermediate (35%) and expert (30%) in the field of child welfare at the start of the course. Smaller shares reported backgrounds in public policy (40%) and journalism (35%); more than half assessed themselves beginners in these fields. These self-perceptions did not change in the post-course survey.

Three-quarters had at least a bachelor’s degree, and the majority was relatively early in their careers (ages 23 to 35). Of those beginning with a bachelor’s degree, one-quarter also had a master’s or doctorate. Eighty-six percent were women. “Personal enrichment” was the main motivation for 74% of the respondents, followed by a desire to “improve their occupational skill” (39%).

MOOC survey results

Students for the MOOC were recruited much more broadly, including the SPOC channels but also via public listing on edX.org. It is not surprising, then, that they had less previous experience with the topics. Prior to the course, more than 50% of students rated their expertise as novice or beginner in child welfare, public policy and journalism. 

MOOC participants had more formal education. A majority had an undergraduate degree, and of those, 36% had a bachelor’s degree and 33% a Master’s. In terms of age, 16% were college-aged (18 to 22), 65% were under age 35, a larger share of young adults than the SPOC. A smaller share of MOOC students (64%) than SPOC students were women. A share similar to SPOC participants, 69%, took the course for “personal enrichment.” About half were taking the course to “improve their occupational skills” and 8% for college credit (an option not available in the SPOC).

Post-course, more MOOC participants rated their expertise in child welfare, public policy and journalism as intermediate and expert than they had pre-course. As a result of JSC4, students were more comfortable contacting leaders and conducting interviews (see Figure 1). Most importantly, they were more interested in changing public policy and more likely to believe that they could change public policy.

Student perspectives

The course was designed with three modules in each unit: journalism, social welfare and public policy. In final reflections, students recognized changes in their attitudes in each of these areas.

Journalism. Many cited activities practicing interviewing skills and evaluating data for stories as useful for their own work as journalists and as readers of news. This is reinforced by survey data (see Figures 5 and 6). At least three students started blogs as a result of participation in the course, and more than a dozen stories that began in the course were published online (see Figure 4).

Some students said the course gave them confidence in their potential in journalism: “This course has encouraged me to go ahead in my career with the purpose of addressing journalism toward social change,”[9] one wrote. Another said that “interviewing people for the main story ... made me realize that I can be a reporter.”[10]

Others made specific mention of the skills they gained, connecting them to a career goal: “I plan to take [these] new skills and practice by writing more on children of prisoners.”[11] “I intend to write for newspapers and magazines about the ill treatment of children .... Interviewing children was a wonderful experience for me.”[12]

Several reflected on their changed views of the role of journalists in society:

It has been an eye-opening course ... [I am now] thinking about the power of journalism to make a difference. (MOOC student discussion post)

Moving forward I will read the news in a more analytic way. Even though the subject matter was very focused, I believe we learned skills to apply to anything. (MOOC student discussion post)

This course has renewed my faith in journalism as a profession. (Yay!) [SPOC student discussion post]

I have learn[ed] what it takes to become a journalist. ... I have more respect for the work they do and what they do to put a story out there on paper for the public. (MOOC student discussion post)

Child welfare. In general, MOOC students ended up learning more about child welfare than they expected. Many reported that the topic is not one that originally interested them particularly, but their views changed as they engaged with debates in the field. Several reported their intention to work with specific child-welfare organizations in their communities as a result of the course.

My own views of maltreatment of kids [have] been increased to say the least. I am more sensitive to this subject and whenever I get a chance to talk about the matter, I will without reservation. (MOOC student discussion post)

At first I was kind of reluctant to deal only with the single topic. As a French [student] I felt I could be lost. ... But then I realized that in order to produce ... solution-based journalism, you need to be expert in your field. (MOOC student discussion post)

Public policy. Finally, students gained an understanding of public policy mechanics that they planned to apply in advocacy. They valued demystifying the policy process and were likely to also see these skills as transferrable to other policy areas. This echoes the finding that MOOC students gained confidence in their ability to conduct interviews, contact leaders and change public policy.

Earlier, I had an impression that journalism was meant to mirror exactly what is happening in the society, giving people a reflection of what they’re doing. This course has made me realise that there are always solutions ... and writing about such stories can make journalism a tool for social change. (MOOC student discussion post)

I have gained a wealth of knowledge from both the legislative and journalistic perspectives and I intend to use the knowledge to voice various challenges that should be addressed by the government. (MOOC student discussion post)

I intend to continue to speak out against child abuse .... I believe that this new knowledge can help me in addressing policy makers. (MOOC student discussion post)

I started the course with very little interest in doing a deep dive on the subject of California’s child welfare system. Yet I was reminded ... any policy topic has multiple layers and complexities. The layers offer the opportunity for many different kinds of stories, but also demand writers who can also move from specific to the larger story. (MOOC student discussion post)

I think I have become a more critical thinker and ... observer. (MOOC student discussion post)

Students’ active engagement with the material and one another helped them practice their critical thinking, writing and reading skills while they updated their beliefs about social policy.

LESSONS LEARNED

While both the SPOC and MOOC were well-received by students, and their performance showed promise, the course instructor and assistant learned several valuable lessons through this experience that are likely to be useful to humanities or advocacy-focused online courses.

1. Learn from your students, and engage them personally.

The instructor and course assistant made several changes to structure and pacing of the MOOC in response to SPOC student experiences. These changes anticipated the challenge of running a course with hundreds more students, many from countries outside the United States.

Students in the SPOC had to submit three pitches; MOOC students submitted one. Fewer pitch assignments made it possible for the instructor and assistant to give feedback to each student, something SPOC students had said they valued. Even short or boilerplate replies from staff in the MOOC appeared to be valuable to students.

Course material in the MOOC was released one week at a time rather than all at once as it had been in the SPOC. Putting students on similar timelines encouraged them to interact with one another in real time. Both SPOC and MOOC students reported forming relationships and collaboration with other students via the discussion board:

Networking and interacting with other students is also a positive experience that I will keep with me. The mood was really good among students and I think there is a great potential to strengthen the bonds created so far and make new ones in the future (MOOC student discussion post).

The course assistant and instructor also engaged students by replying to discussion posts with further questions. This initially took a significant amount of time. For the MOOC, the instructor and assistant developed boilerplate responses. Students did not seem to notice (or at least to complain) about these somewhat generic responses, but instead reported that they felt more engaged by the staff than in other MOOCs they had experienced.

2. MOOCs can inspire significant changes in attitudes and beliefs.

Although the response rate for the MOOC was low, respondents showed significant changes from pre- to post-course, indicating that J4SC had a positive impact on their attitudes and beliefs. A follow-up survey could be sent to respondents to get a better sense of what they are applying from the course a few months out.

As shown in the student perspective section above, even the process of acquiring skills can shift attitudes about a topic or an entire field. While some students were frustrated by the United States focus of the child welfare topics, others saw it as a training ground for coming to understand how much a solution-based journalist needs to know about the issue they would like to impact. The variety of final stories’ topics and settings highlighted just how broad the subject could be in practice.

3. Some students are motivated by publicity.

The courses each culminated with an 800-word story assignment on a child welfare topic. Students were promised a shot at publication in The Chronicle of Social Change, a leading outlet in that field, if they submitted their stories by the deadline. While a small share of total enrollees completed the story and had it published, the ones who did were significantly more engaged in the course throughout. In turn, their visible participation online was a resource to other students in the course.

At the end of the SPOC, several students who had published stories participated in a live, 30-minute video chat online. MOOC students were invited to watch as five SPOC writers discussed their subjects, from the juvenile justice system to a profile of a foster youth mother. This recording has been viewed more than 1,265 times in three months.

Three students from the MOOC participated in a live video chat at the end of that course. They included a 19-year-old home-schooled student living in Malaysia who wrote about child marriage in Malawi; an Italian freelance journalist working for a human rights organization who examined “rehoming” in Arkansas; and a college student in New Delhi, interested in the child welfare focus, who wrote about child abuse in India. The recording had been viewed 439 times just over two months later.

Two students who appeared in the video chat have continued to write for The Chronicle of Social Change, an indication of their continued interest in the subject matter and the practice of solution-based journalism

4. Set expectations early and often.

Some students felt overloaded with or intimidated by the course work. Many assignments involved reading that took longer than estimated, especially for those who were not as fluent in English. Those with limited background knowledge about child welfare or the American context generally reported that they were reluctant to join discussions with their peers. However, among those who mentioned this as a barrier, all also said they gained confidence as the course progressed.

In each of these cases, setting clear expectations up front can encourage students to participate at a degree they are comfortable with. Clarity about self-directed learning as a benefit and not a restriction seemed to improve attitudes among individual students who were frustrated by the pace.

5. Ensure your platform can provide the data you need.

The absence of data from edX limited evaluation. Data from edX specific to students’ actual engagement in the course (e.g., time spent or completion) was unavailable. Such data could be linked with students’ attitudinal data and would better address the change going on as a result of the course. The data currently gathered speak to respondents’ reactions before and immediately after the course. A follow-up survey, which addresses application of learning and change in behavior as a result of the course several months later, might be an informative next step. 

CONCLUSION

Taking an in-person class and converting it into a successful MOOC is a difficult task. While certain aspects of the Journalism for Social Change class were degraded in the conversion, others were augmented.

For example, the semester was condensed into seven weeks. While this required a lessened workload, it also required the instructor to synthesize lessons into a much shorter format, which as evidenced by the survey, seems to have proven successful. This unexpected result will help inform future iterations of the Journalism for Social Change on-the-ground class.

But the through-line is that there is a global community of people interested in solution-based journalism who are willing to apply the skills J4SC trained to help solve endemic world problems. If this program could be expanded to other subject areas, and the class itself could be streamlined, there is the potential to train an army of solution-based journalists empowered to produce stories that can change the course of public policy.

Such potential warrants further exploration and study.

__________

Figures

FIGURE 1.  Changes from pre- to post-course survey of MOOC students.

Means are presented from the 5-point scale from 1 (Strongly Disagree) to 5 (Strongly Agree). Higher means indicate greater agreement. (n = 72)

FIGURE 2. Significant differences on the pre-course survey between those who did and did not complete the MOOC post-course survey

Means are presented from the 5-point scale from 1 (Strongly Disagree) to 5 (Strongly Agree). Higher means indicate greater agreement.

FIGURE 3. Journalism for Social Change Discussion Rubric

FIGURE 4. J4SC MOOC and SPOC Student Story Headlines (Partial List)

 

  • Arkansas Becomes Fifth State to Regulate Re-Homing In Wake Of High-Profile Case [13]
  • Prosecuting Youth as Adults Fails to Address Trauma [14]
  • Legalization Without Citizenship for Stateless Children in Sabah, Malaysia [15]
  • Child Safety Versus Family Preservation In Sierra Leone [16]
  • Bill to Expand Foster Care Benefits for Probation-Involved Youth Clears Key California Senate Committee [17]
  • The Crime of Punishment [18]
  • Seven Years Later: States Working Toward Promise of a Medical Home for Every Foster Youth [19]
  • Discontinuing Solitary Confinement in Juvenile Facilities [20]
  • The Unanswered Question [21]
  • UK Report Says British Government Fails Homeless Youth [22]
  • Journalism Can Incite Social Change: Here’s How Media Coverage of One Awareness Campaign Made A Difference! [23]
  • The Vaccine for Pollyanna Attitudes Toward Public Health and Religious Beliefs [24]

FIGURE 5

FIGURE 6

__________

Sources

[1] CBS SF Bay Area (Jan. 2012), UC Berkeley Class Promotes Journalism for Social Change, Link: http://sanfrancisco.cbslocal.com/video/6661720-uc-berkeley-class-promotes-journalism-for-social-change/

[2] edX is a consortium / a group of elite universities partnering to provide MOOCs.  Harvard and MIT developed the edX learning management system(s) as the ‘open source’ vehicle through which to offer the massive courses.  Any person offering an “edX” course is required to have that course built and hosted inside of the edx LMS.  Berkeley is the third partner in the edX consortium and, instead of directly funding the consortium via membership fees, it contributes in-kind services such as engineers to work on code development projects.

[3] CalSWEC provides curricula and training to 21 graduate and undergraduate schools of social work throughout the state. They are a key partner, not only in regard to the research, but also in helping with the overall expansion of the J4SC program.

[4] Ho, Andrew Dean and Chuang, Isaac and Reich, Justin and Coleman, Cody Austun and Whitehill, Jacob and Northcutt, Curtis G and Williams, Joseph Jay and Hansen, John D and Lopez, Glenn and Petersen, Rebecca, HarvardX and MITx: Two Years of Open Online Courses Fall 2012-Summer 2014 (March 30, 2015). Available at SSRN: http://ssrn.com/abstract=2586847

[5] Yudelson, M., Hosseini,R., Vihavainen, A. & Brusilovsky, P. (2013). “Investigating Automated Student Modeling in a Java MOOC.” In the 7th International Conference on Educational Data Mining. Veletsianos, G., & Miller, C. (2008). Conversing with Pedagogical Agents: A Phenomenological Exploration of Interacting with Digital Entities. British Journal of Educational Technology, 39 (6), 969-986.

[6] Reichard, Cara. (2013). “MOOCs face challenges in teaching humanities.” The Stanford Daily.

[7] Harvard, MIT and UC Berkeley are founding members.

[8] Ho, et al.

[9] MOOC student discussion post

[10] MOOC student discussion post

[11] MOOC student discussion post

[12] MOOC student discussion post

[13] https://chronicleofsocialchange.org/news/arkansas-becomes-fifth-state-to-regulate-re-homing-in-wake-of-high-profile-case/9901

[14] https://chronicleofsocialchange.org/analysis/prosecuting-youth-as-adults-fails-to-address-trauma/9525

[15] https://chronicleofsocialchange.org/news/legalization-without-citizenship-for-stateless-children-in-sabah-malaysia/9881

[16] https://benjaminstephensmedia.wordpress.com/2015/04/11/child-safety-verses-family-preservation-in-sierra-leone-the-failings-of-a-single-solution/

[17] https://chronicleofsocialchange.org/news/bill-to-extend-foster-care-benefits-for-probation-involved-youth-clears-key-california-senate-committee/9924

[18] https://chronicleofsocialchange.org/opinion/the-crime-of-punishment/9559

[19] https://chronicleofsocialchange.org/analysis/seven-years-later-states-working-toward-promise-of-a-medical-home-for-every-foster-youth/9551

[20] https://chronicleofsocialchange.org/news/discontinuing-solitary-confinement-in-juvenile-facilities/9544

[21] https://chronicleofsocialchange.org/opinion/the-unanswered-question/9548

[22] https://chronicleofsocialchange.org/research/uk-report-says-british-government-fails-homeless-youth/9982

[23] https://fishershannon.wordpress.com/2015/04/04/journalism-can-incite-social-change-national-sexual-assault-awareness-and-prevention-month-unite-against-rape-shannon-fisher

[24] https://verdict.justia.com/2015/02/12/vaccine-pollyanna-attitudes-toward-public-health-religious-beliefs

Lower Premiums for States With Stronger Health Insurance Rate Review Authority

MINNEAPOLIS/ST. PAUL – A new study published today found state-level prior approval authority over individual market health insurance rates were in effect from 2010 to 2013 was associated a 10 percentage point lower rate of increase in premiums.
 
The research was published in the August issue of Health Affairs, and was conducted at the University of Minnesota School of Public Health in collaboration with the University of California, Berkeley School of Public Health.
 
“This study provides the first evaluation of state rate review authority in the individual market in the years immediately following the implementation of the Affordable Care Act,” said Pinar Karaca-Mandic, Ph.D., lead author and associate professor in the University of Minnesota School of Public Health. “We found significant variations in rate review authority across states and these were highly associated with differences in adjusted premiums.”
 
Premiums were adjusted for insurance carrier, insurance market, provider market, political, and population differences across states.
 
The Affordable Care Act (ACA) requires carriers in certain categories of health insurance to provide public justification for rate increases of 10 percent or more. During the 2010-13 time period, 44 states upgraded their health insurance rate review programs by hiring or contracting actuary services, upgraded information systems and enhanced insurance rate transparency.
 
Study authors collected rate review authority and anticipated loss ratio requirements from each state and the District of Columbia by examining statutes, regulations, and bulletins and distributed a questionnaire to each state and the District of Columbia.
 
“Our findings suggest that rate review by states with prior approval authority may be a viable option for moderating the growth in health insurance premiums," said Richard Scheffler, Ph.D., principal investigator and distinguished professor of health economics and public policy at UC Berkeley. “Given the massive rate increases in premiums being proposed this year, our results could not be timelier.”
 
During the 2010-13 time period, the study also found:
 

·  States with prior approval authority and a loss ratio requirement in the individual market experienced lower health insurance premiums.

·  States with prior approval authority in the individual market experienced lower growth in health insurance premiums.

 
"Many states bolstered their rate review programs after the passage of the Affordable Care Act using federal grants, but will now need state funding to maintain their programs," said Brent Fulton, Ph.D., M.B.A., assistant adjunct professor of health economics and policy at UC Berkeley.
 
The authors say given the expansions in individual-market coverage, it will be important to further evaluate state rate review authority and activity to determine whether study findings remain in effect over a longer time period and are generalizable throughout the expanded individual insurance market.

The authors are thankful for funding provided by Robert Wood Johnson Foundation through its Changes in Health Care Financing and Organization Program (Grant No. 69906). Pinar Karaca-Mandic also acknowledges funding from the National Institute on Aging (Grant No. K01AG036740).

For more information please contact katiphillipsucb@berkeley.edu or visit Petris.org