Evaluating the entire university is not as easy as seeing where UT ranks in the NCAA top 25 polls in athletics. It has not been perfected down to a science.
Perhaps the biggest point of contention for people is the validity of the U.S. News and World Report rankings, arguably the ones that get the most buzz. These are the ones that, once a year, each university will interpret in order to make itself look like it has improved, even if overall it has not.
The U.S. News and World Report rankings seemingly annually name the top universities as the same. Among the top overall, there’s Harvard, Yale and Princeton at the top. Among the public schools, there’s a litany of schools from the University of California system, topped by the University of California at Berkeley and the University of California at Los Angeles. So one has to wonder how valid rankings can be when so much depends on amount from endowment and state budgets, especially as UT begins to depend more and more on private funding.
UT has fallen from a top 40 public school in 2007 to No. 52 in the U.S. News and World Report’s public school rankings, all in the midst of a financial crisis. How much does that really mean to UT, or how much should we worry about improving?
Enter the National Survey of Student Engagement. Alexander McCormick, director of the survey, said it was created to improve discussion quality about universities. In other words, it was time to get more specific about how a university does and does not excel while also refocusing the discussion on things other than simply amount of funding.
And to say that the survey is specific is an understatement. A PDF sample of the 2010 survey on its website has 26 pages full of questions, each with options on how to answer.
Data from 2008 UT seniors showed that they are less satisfied than their peers in some key categories, most notably how they evaluate their overall educational experience and whether they would attend the same university again.
This got people to thinking at the Academic Affairs and Student Success Committee meeting on June 24 about why these scores were lower than peers and just exactly how much the scores meant.
It’s hard to exactly say. The sample rate for the last data was 39 percent, which seems like a healthy percentage of the university. If you want to truly consider the results for possible changes, a greater percentage would help give an even clearer picture, but one also has to wonder how the university could possibly incentivize the survey enough for more students to complete it.
“Incentives don’t make a big difference,” McCormick said. “What matters most is students having some level of willingness to help their school improve. If schools effectively communicate that we want to know your opinion ... that’s usually the most effective.”
McCormick said it’s up to the institution to decide how to incentivize the survey, but the website lists examples of how other schools incentivize. Usually it’s raffles for prizes like an iPod. Last year, UT gave survey responders a chance to win one of four $100 credits for their All-Star account.
But if incentives will not make much of a difference to most, what can be done to improve the response rate? Losing questions from the comprehensive survey would make it less specific when specificity and comprehensiveness are its major appeals.
“There’s always a fine line to walk in between asking everything we’d like to ask and giving students a survey that a sufficient number of students will take the time to fill out and give back to us,” McCormick said.
Though perhaps a few of the questions — at least the ones from the 2010 sample survey — could be eliminated. One question asks a student about how often during the year did he or she “participate in activities to enhance your spirituality (worship, meditation, prayer, etc.)” How relevant is that to the overall educational experience of the institution?
And are we getting too specific when we start asking how often students talk about subject matter and ideas with teachers outside of the classroom? Just how often is that reasonable within the typical college student’s weekly schedule?
Two of the categories that the 2008 UT seniors ranked lower in than peers were volunteer work and study abroad. Now obviously these are two valuable aspects of university life, students being able to give back to their community and travel to different parts of the globe. Kudos to those who wish to participate, and best of luck to you. But in the overall framework of things, volunteer work and study abroad may not fit into students’ schedules either or not appeal to them. Should that really lower a university’s overall evaluation?
On top of all that, consider the students filling out the survey, which now goes out to all freshmen and seniors. Freshmen are just in their second semester of work at UT, so their experience is limited. And that limited experience will lead to amplified results. If their first semester at UT was an enjoyable one, they’ll don rose-colored glasses and say they would attend again. If it was not, they may be ready to lambast the university before their first year is even over.
As for seniors, the likewise can be true. Nostalgia may put on the rose-colored glasses, or struggles to get through to graduation (sixth-year seniors, I’m looking at you) may lead to bitterness.
Furthermore, just how honest are the students filling out the survey? Autobiographers always make themselves look better than they were in their memoirs. Maybe students do the same.
So while a move from evaluating universities by U.S. News and World Report rankings to a more comprehensive college survey is beneficial, even that survey’s results are problematic. And at the end of the day, even with all this data, we have to sit back and wonder: Is it meaningful?

— Robby O’Daniel is a graduate student in communication and information. He can be reached at rodaniel@utk.edu.