Re-writing the writing seminars

Ask most upperclassmen and they will tell you without hesitation what they think of Penn's writing seminars.

The responses range from "it could've been worse" to "it was the death of me." And student evaluations reflect the sentiment. An analysis of Penn Course Review data since 2009 shows that writing seminars are consistently rated well below the average course when it comes to "course quality" and well above average in terms of "work required."

Last year, Director of the Critical Writing Program Valerie Ross led writing faculty through the biggest transformation the department has ever seen. Penn's current freshmen are the first flock to take a stab at the new curriculum.

The current transformation highlights an ongoing struggle to improve a decade-old required course that often leaves students more frustrated with writing than when they arrived.

Survey data from last semester is not yet available to see if the student sentiment has changed, but anecdotally, department faculty is hopeful they may have finally found a solution.

Reflection and reworking

The Critical Writing Program formed in 2003 to solve a problem Penn knows well: a lack of centralization. It was the place Penn would finally integrate all of its writing resources, and to top it off, the new program would provide newly required one-semester writing seminars for all undergraduates.

Students have made their writing seminar grievances clear on evaluations since the beginning. Beyond Penn Course Review surveys provided at the end of each semester, the writing department distributes its own evaluations which probe with more detailed questions about the way the seminars are taught and various class assignments.

Sometimes, the comments cut deep. Ross described the "nonchalance of students" who occasionally compose personal attacks against professors in their evaluations, which can be especially hurtful for new professors who are less confident in their abilities.

"They should have the mindfulness that they're anonymous and we're real people," she said.

Recognizing student dissatisfaction from the outset of the program, writing faculty have worked over the years to boost the seminar experience. "We're always tinkering," Ross said. "We are constantly thinking of how to improve."

The tinkering process involves a lot of collaboration: faculty members will look at student evaluations, the execution of final portfolios and other factors before coming together to make suggestions and ultimately vote on curriculum changes.

"I've taught at seven or eight universities and I've never had the collaborative experience like this one," said Rodger LeGrand, the program's associate director of academic administration. "Normally at other universities where I've taught, you just kind of go in and go to work, but here, we're always working as a community."

Starting in 2011, the writing program began a three-year transition from teaching seminars with part-time instructors, adjuncts and graduate students to running the classes with almost all full-time instructors - the first major change in the department that appears to have affected Penn Course Review data. "Instructor quality" has been on a general rising trend since 2009, and since 2012, writing seminar instructors have actually received better ratings than the average Penn professor.

Semester by semester, though, the evaluations are still volatile, some seeing much higher ratings than others. The data shows that students' ratings of "work required" for writing seminars has also risen since 2009, and "course quality" has remained flat and well below average.



Rejecting the writing program

Ross and her colleagues are faced with obstacles that are difficult to overcome. For one thing, writing seminars constitute the only course required by each and every undergraduate across all four of Penn's schools. For another, students probably spent all of high school doing what they believed to be was perfecting their writing ability.

"Every place I've worked, the writing seminar classes were rated low," said Rob Nelson, Penn's executive director of education and academic planning. "Students often come to college feeling as though they've mastered writing. To be given a curriculum of writing instruction feels somehow wrong."

Penn students aren't the only college students putting up with writing seminars - and they aren't the only ones unhappy about them either.

By the 1980s, required writing courses at the university level were growing around the country at both small liberal arts schools and prestigious research institutions. Some schools, like Yale University, still make participation in writing seminars optional. Others have more rigorous standards than Penn, demanding students take two or more semesters of writing rather than just one.

Princeton University students have even called on their institution to make their writing seminars more like Penn's.

In an October 2014 column in The Daily Princetonian, one student argued to adopt Penn's model of offering different "levels" of writing seminars for students of varying writing proficiencies: Penn offers "The Craft of Prose" (WRIT-002) to provide a more basic course for students who feel anxious or lack confidence in writing, as well as "Global English seminars" (WRIT-011) for international students who are not yet fully comfortable writing at the college level in English. Other students can take a variation of the writing seminar at a somewhat more advanced level (WRIT-012).

So, when students say they can't stand their writing seminar, what do they really mean?

Penn Course Review data doesn't do a great job of clearing this up. Ross believes the data may not accurately reflect what's going on in the classroom. Even if it does, she said, some things may not be fixable.

For example: there is only so much that an instructor can do in a small seminar to make the classroom experience enjoyable.

"If you have enthusiastic students who want to learn, that class is going to sing even with not that great of a teacher. If your students are grumpy, indisposed and tired - you bring the best teacher in the program and they'll say, "eh,'" Ross said. "Learning communities can wreak havoc. They can hijack the class."

Academic research has pointed to all sorts of inherent biases in student evaluations of their courses. Male professors and attractive professors receive higher evaluations than their counterparts, and science and math courses consistently receive lower ratings than humanities courses.

As for Penn Course Review, when the evaluations are administered, there is no mention of what factors students should consider when thinking about "course quality," "instructor quality" or any other variables.

When trying to determine how to improve a department with a long-standing history of producing frustrated students, these ambiguities pose a challenge.

"We're not even sure what they're voting on with course quality," Ross said. "Are they saying they wish they didn't have to take a writing seminar? Are they saying it's too much work? It doesn't signify in any way that helps us."

To help narrow down the ambiguity, the writing program provides students with more specific evaluations to fill out that address particular assignments and how much students have learned from them.

When considering a professor's performance, the department expands beyond student evaluations altogether. New instructors are required to perform one evaluation of another instructor and to be evaluated themselves at least once a year. The final portfolios of their students are also assessed by other faculty.

Professors will face closer scrutiny only if they consistently score low on evaluations rather than if they receive a negative review once in a while, Ross said. At that point, other seminar instructors can step in to help the struggling professor improve his or her skills.

Writing seminar by the numbers

From Spring 2009 to Spring 2015, students have consistently rated the "course quality" of their writing seminars lower than the average Penn course. "Instructor quality" has been slowly rising over time, with the biggest jump occurring when Penn started transitioning from teaching primarily with part-time to full-time faculty. There is no Penn Course Review data yet to examine whether the writing seminar curriculum overhaul in Fall 2015 had an affect on student ratings.

Beyond one-time evaluations

The writing department is now taking evaluations one step farther.

The program is currently in the midst of a longitudinal study to learn more about what students can and do take away from the seminars. In 2014, writing faculty members videotaped three writing seminar courses every week of the class. With permission, the faculty gathered all possible data on the classroom's students, including their evaluations and final portfolios.

Then came the daunting part: a full semester of coding the videotapes, marking down the ways professors communicated and students participated in class discussion. Every second of every class was analyzed.

Each year since, the students who took the three seminars in the experiment have filled out follow-up questionnaires asking them if and how writing seminar has remained applicable to their lives. In the end, the writing program will have an overview of how these students perceived writing seminar at the time they took it and long into the future, and whether the classroom experience they had played a role in that perception.

Though data collection is ongoing, Ross says it appears so far that the students do take away something valuable long after their seminars conclude.

"Students are using what they learned," Ross said, "but they don't necessarily know they're using it until they're prodded."

Transferring knowledge

Walking the halls of Hill College House this year, Fayyaz Vellani heard something new: writing seminar students discussing their peer reviews excitedly.

"I have seen a complete reversal of the culture of people hating the writing seminar requirement," said Vellani, who teaches three writing seminars and lives in Hill as a college house fellow. "It's no longer, "Oh you poor thing. You have to take writing seminar.' People see the utility of it."

The writing program instituted an overhaul in the fall semester, reworking the curriculum to improve students' ability to apply in-class learning outside the classroom. The official phrase for this is "knowledge transfer."

The standard expository and justificatory papers endured by the now-upperclassmen have been replaced with a literature review and a public editorial submitted to a real-world publication. Students now also produce smaller career-oriented assignments like cover letters and professional emails. Add to that regular peer evaluations on every assignment, submitted electronically through myreviewers.org.

Students complete a midterm and a final portfolio as they had in the past, but the contents of the portfolios span a broader range of writing material than ever before.

"The difference was made partly in response to students' needs, who are feeling like they want to learn something relevant to them in everyday life," Vellani said. "This is also part of the evolution of writing education, which is moving toward the study of genre-based writing. The genre of email, for example, or writing a blog post or a tweet."

College freshman Elizabeth Goran, who is now taking the seminar "The Digital Audience," emphasized the applicability of her writing assignments to future jobs and courses.

"Last semester, I had to write a literature review for my [communication] class. Now, I'm getting to actually learn more about it after I kind of came into it blind last semester," she said. "I really understand what it entails now, and it will be really helpful for me to know when I continue writing these kinds of papers in future."

On mid-term internal writing program evaluations this semester, students rated their seminars more highly than any previous spring semesters, according to LeGrand, who said the ratings have been on the rise over the past few years despite little movement in the public Penn Course Review data.

Other professors are also hopeful that the program revamp will create an impactful change on the student learning experience.

"I think it has been wonderful. Terrific," said Sara Byala, who teaches seminars called "The Lion King" and "The Craft of Prose." Based on anecdotes, she said, students appear to both enjoy their seminars more and feel they are more applicable to real-life situations. "We're all feeling pretty good right now. Of course, nothing is perfect, but so far it seems to be working."

Still, plenty of students continue to express dissatisfaction with their seminars.

"My expectation coming in was that the seminars were really bad. Actually, taking it was not that horrible, just really boring," said College freshman Eric Calvo, who took "Comics and Graphic Novels," last semester. "I felt like it gave me necessary information, although I feel like most of it could've been accomplished in half the time."

Calvo described his disappointment when he realized he would be reading research texts about comics rather than comics themselves. To him, it signified that the class had so little to do with the supposed writing "topic" and everything to do with pursuing daunting writing assignments at home and little productive in-class work.

"My advice for other people taking writing seminar: Look for the best rated professor who teaches writing seminar on Penn Course Review," he said. "Odds are they'll be better at teaching it."

Perhaps future Penn Course Review data will support what faculty members suggest - that the sentiment around writing seminar really is changing. Perhaps it won't. Either way, faculty will continue trying to respond to student distress - as much as they think it makes sense.

"Though I have noticed a shift with the positive talk among students, it still is true that working on writing is hard," Vellani said. "Students can still struggle and it will still be a lot of work. It's demanding and that's not going to change."