Editor:
Jocelyn Robinson

robinson@antioch-college.edu

Contributing Writers:
Amy Harper
Fred Kraus
Jocelyn Robinson

Photography:
Callie Cary Devine ’84
Dennie Eagleson ’71
John Fleming
Amy Harper
Jocelyn Robinson

Antioch and the NSSE: Setting National Benchmarks

by Bob Devine ’67

Several years ago at a New York alumni event I had a very engaging conversation with alumnus Dick Wolf ’57 regarding College rankings and the national reputation of Antioch College.

I was reflecting on the challenges that the various national ranking systems presented for the College, and public preoccupation with such rankings as a measure of the quality of an academic institution. The U.S. News and World Report rankings had been particularly problematic for Antioch, with the criteria and weighting of the ranking having to do with:¹

1. Academic reputation – what others think about the institution. This item alone accounts for 25% of the composite score, and is based on a survey administered to Presidents, Deans and Provosts across the country. Unfortunately, what many remember about Antioch College has to do with the turbulent times of the ’70s and what alumnus Lawrence Lasser ’65 has characterized to me as “the squandering of Antioch’s reputation” during that difficult era.

2. Student selectivity accounts for 15% of the composite rating. Surveys look at acceptance rate, yield, high school class standing, SAT/ACT scores and the like. Antioch has always been a self-selecting school, and has not required SAT scores (although 85% of our applicants provide them) for a very long time. It’s a difficult measure for the College to improve.

3. Faculty resources account for another 20% of the rating. Faculty compensation, percentage of faculty Ph.D.s, student/faculty ratio and class size contribute to this measure. While our student/faculty ratio is rich and we have a strong faculty, Antioch has never been able to bring faculty compensation within a range that would contribute to improvements in this element of the overall rating.

4. Retention rate, including both first-year retention and seniors contribute another 20% of the ranking measure. Attrition rates have seldom been lower than the mid 20% range, and have soared into the low 40s on a number of occasions. There is great room for improvement in this category, but the mobility and agency of Antioch students has always differentiated the College from other private liberal arts institutions.

5. Financial resources of an institution account for an additional 10% of the composite ranking, and include educational expenditures per student and endowment per student. This is an area in which the College simply cannot compete. Antioch’s endowment is a minuscule 6% of the median of the Great Lakes Colleges Association Schools (Oberlin, Denison, Kenyon, Earlham, Wabash, Wooster, Kalamazoo, Hope, et. al.), and a full $100 million below the next lowest endowment in the GLCA.

6. Alumni giving rates contribute another 5% to the ranking, and though Antioch does quite well in this regard, the weight given to this factor is not sufficient to counterbalance some of the other factors.

7. Graduation rate performance accounts for the final 5% of the rankings. Throughout its history, Antioch’s graduation rates have only rarely risen above the low 50% range, and so this element of the ranking contributes still further to the College’s poor showing. The weight of these rankings rests upon an institution’s reputation and the resource base it has to offer. As John Brooks Slaughter, Professor of Leadership in Education at USC points out:

Nobody but higher education measures excellence by the quality of the incoming material. Everybody else measures the value that you add. I think that’s the biggest problem I have with the national ranking systems. They look at supposedly objective input measures, they look at how much money you’re spending, how many faculty you have who are Nobel laureates, the SAT scores of your incoming students… but nobody stops to ask, ‘What have your students done?’ Nobody knows how to measure that. It’s a farce that we chase our tails trying to move up in the U.S. News rankings when none of those measures necessarily equates to a high quality education.

Further, Dick pointed out to me that institutions are not always forthright in providing the data upon which such ratings are based. He noted that a recent investigative report in the Wall Street Journal had compared what institutions told Moodys for purposes of bonding with what they had told U.S. News and World Report, Fiske, Petersons Barrons, etc. for purposes of ratings. The differences, even among the most prestigious institutions in the nation, were stunning, with the ratings reports skewed to the advantage of achieving a higher overall ranking. We agreed, however, that in lieu of some other instrument or measure, these input-oriented ratings were the only game in town.

In 1997 when we brought forward a comprehensive Strategic Plan for the College, we had some conviction that the key to the long-term health and vitality of Antioch College involved the strength and depth of the academic program. With that conviction as the driving force, we’ve hired 30 tenure track positions in the last few years in a self-conscious effort to build the faculty of the future. We’ve converted temporary positions to tenure track, strengthened internal and external program delivery at every level and poured an additional $1 million a year into the academic program. In an effort to bring Antioch salaries into the range of GLCA schools, and to allow the College to attract and retain the best faculty members possible, we’ve also increased faculty salaries substantially. We implemented an 8.76% increase last year and are putting forward an additional 8.45% increase for this coming year.

Those of us involved in these efforts have harbored an intuitive and anecdotal sense that our efforts were paying off. Students seemed to appreciate the increasing rigor and vitality of the academic program, with the result that we’ve managed to cut our attrition to the lowest levels in Antioch history – 13% for first year (unheard of!), and 13% overall. Visiting academics have consistently been impressed with the quality of the students they encountered, and the stellar quality of work produced by graduating seniors has seemed to attest to the power of Antioch’s tripartite educational model and the extraordinary learning outcomes it produces in our students. Until recently, however, the College lacked external confirmation of the quality of an Antioch education.

In early 1998 the Pew Charitable Trusts convened a meeting to focus on the ratings and rankings of colleges and universities and to think about alternative measures that might refocus the conversation about accountability and quality toward the teaching and learning activities that are known to enhance student learning. The National Survey of Student Engagement (NSSE) came out of that meeting. One of the designers of the NSSE instrument, Peter Ewell, recognized the opportunity “to develop a national data collection system that might head off some of the more ill-conceived kinds of accountability measures being considered in some quarters³ ,” and to shift attention toward what we know from the research regarding student learning.

Antioch College was invited to participate in the inaugural round of NSSE benchmarking, and with 275 other institutions administered the NSSE instrument to first- and fourth-year students during the spring of 2000. The Survey collected information directly from 60,000 undergraduate students across the country regarding the quality of their education. The survey examined five major themes of student engagement: level of academic challenge; active and collaborative learning; student interactions with faculty members; enriching educational experiences; and supportive campus environment. Taken together, these categories get at those elements of educational practice that Nannerl O. Keohane, President of Duke University, characterizes as a “high quality education” – “…the importance of context, of close collaboration, of responsiveness and feedback, and personal relationships, of place…”⁴

1. Level of Academic Challenge involves the intellectual and creative challenges posed by the academic program, such the time spent preparing for class, the amount of reading and writing assigned, the institutional expectations for academic performance, and coursework emphasizing analysis, synthesis, making judgments about the value of information, experience and arguments, and applying theories and concepts to practical situations. In general the survey sought to determine the extent to which the institutional environment emphasized studying and academic work. Somewhat surprisingly, Antioch College had the highest score among the 276 participating institutions and set the benchmark score in this category. One of the consistent critiques of the College that I have heard from alums (of every generation) is that the Antioch of today lacks rigor and challenge. I would like to believe that our persistent attention to strengthening the academic program and building the faculty during the last four years has overtaken this popular mythology. Clearly Antioch challenges students to work harder than they thought they could to meet Antioch standards and expectations, and our model of situated learning – in the classroom, on co-op and in the community – enhances the learning processes involved in making judgments about the value of information/experience and applying theory to practical situations. This benchmark score signals a significant shift in the culture of the College.

2. Active and Collaborative Learning involves participation in classes, making class presentations, working collaboratively with other students both inside and outside of class, participating in community-based project work as part of a regular course, and discussing ideas from readings and classes with others outside of class. Antioch first-year and fourth-year students both scored in the 90^th percentile in this category, against both liberal arts institutions and all of the schools represented in the NSSE sample. Our small classes and the engaging pedagogical practices of our faculty obviously contribute to this high score. However, I think it’s also important to note that the culture of the College community engages faculty, students and staff alike in close daily relationships of governance and problem solving, and that this ethos of active community learning transcends the boundaries of the classroom. This is clearly one of Antioch’s strengths.

3. Student Interactions with Faculty Members includes talking with faculty members and advisors about ideas, assignments and career plans, discussing ideas with faculty members outside of class, working with faculty members on activities other than coursework, getting prompt feedback from faculty on academic performance, and working with a faculty member on a research project. This is the labor-intensive, time consuming and demanding part of faculty life at a small liberal arts college. The close interactions and mentoring, however, are critical to the quality of student learning. Antioch first-year students set the benchmark standard in this category, while fourth-year students ranked in the 90^th percentile. The community and governance dimensions of an Antioch education, however time-consuming they may be, contribute positively to student learning by putting faculty and students together in collaborative roles of intentionality and community responsibility. This is clearly another of Antioch’s strengths.

4. Enriching Educational Experiences involves: participation in co-curricular activities, organizations, publications, student government and the like; community service or volunteer work; practicum, internship, field work or co-op experience; foreign language coursework and study abroad; independent study or self-designed major; and, culminating student experience such as a capstone or senior project or thesis. This category also probes the extent to which students have the opportunity to interact with students with different religious beliefs, political opinions or values, with students of a different race or ethnicity, and the extent to which the climate encourages contact among students from different economic, social and racial or ethnic backgrounds. It should come as no surprise that Antioch scored so high in this category that it set a benchmark which is unattainable by most institutions.

The College’s Bonner Scholars and Community Responsibility Scholarship programs create a strong ethic of community service, with over half of our students engaged in some form of community service or volunteer work. The revitalized Co-op program puts students in settings across the country and around the world, and requires of them the development of the habit of critical self-reflection. Our curriculum has supported independent study and self-designed majors for four decades. A strong and path-breaking Antioch Education Abroad program is accessible to all of our students, and in combination with language proficiency and cross-cultural requirements, pushes and supports students in broadening, extending and enriching their engagement with the liberal arts. The cauldron of governance and campus life requires students to engage with difficult issues, to confront difference, and to reach agreement through dialogue and deliberation on a daily basis. The research has shown that “there is a consistent pattern of positive relationships between diversity in higher education and both learning and democracy outcomes,”⁵ and Antioch’s rich history and current practice of engaging difference – at all levels – undoubtedly contributes to our unusually high benchmarks in this category. It seems to me that the enrichment provided by Antioch’s distinctive approach and mix of learning opportunities – including co-op, shared governance, AEA, and community life and service – provide the core of what differentiates Antioch from peer institutions. Clearly this characteristic of engagement will distinguish Antioch College nationally as the NSSE benchmarking goes forward.

5. Supportive Campus Environment involves the extent to which students perceive that the campus helps them succeed academically and socially, assists them in coping with non-academic responsibilities, and promotes supportive relations between students and their peers, faculty members and administrative personnel and offices. This was the benchmark on which Antioch had the weakest performance. While Antioch scored relatively high against the full field of NSSE schools (90^th percentile for first-year and 70^th percentile for fourth-year), against other liberal arts institutions our first-year students scored in the 90^th percentile, and fourth-year students scored in the 40^th percentile. As Antiochians know, the College is volatile and constantly simmering. It conducts its problem solving and “crisis management” in a public arena, and is described by many as intense and demanding in a way that many schools are not. While we have made tremendous strides in creating a civil, supportive and trusting campus climate, as reflected by our dramatic improvement in retention numbers, the campus environment remains one of our major challenges. It is worth noting that during the sampling period last spring the campus experienced three student deaths and was inundated with protests and hate mail regarding the selection of our commencement speaker; our second year of administering the NSSE instrument should give us some indication of the consistency of our “Supportive Campus Environment” ranking.

The Long Term

¹ See Ronald G. Ehrenberg, Tuition Rising: Why College Costs So Much, Cambridge: Harvard University Press, 2000, p. 54 for a description and analysis of the U.S. News and World Report criteria and weights.

² Donna Maeda, “Building the Engaged Campus: An Interview with John Brooks Slaughter,” AAC&U Peer Review Volume 2, No. 1, Fall, 1999, p. 15.

³ Barbara Cambridge, “Assessing the Real College Experience: The Architects of the National Survey of Student Engagement talk about the Meaning Behind the Numbers,” AAHE Bulletin, Volume 53, No. 5, January 2001, p. 8.

⁴ Nannerl O. Keohane, “Going the Distance,” Educause, July/August, 2000, pp. 10-11.

⁵ Patricia Gurin, “New Research on the Benefits of Diversity in College and Beyond: An Empirical Analysis,” Diversity Digest, Volume 3, No. 3, Spring, 1999, p. 5.

⁶ Cambridge, p. 11.