Every year around this time of September, a number of college rankings and the "Best Of" listings are released. The most widely regarded is U.S. News & World Report, and others include Forbes, Wall Street Journal (WSJ), Business Week, Princeton Review. While many of the same schools occupy similar ratings or tiers, what causes some lists to rank a school higher or lower? Each list prioritizes different features. When you inevitably look at how schools compare in particular ranking lists, it's important to remember the origin of these rankings, and how these numbers were calculated. The lists are deeply tied to the methodologies and philosophies represented by each institution that puts them out.
Thanks, Education Corner for a thorough breakdown, as listed below.
US News & World Report
For almost three decades, the U.S. News & World Report has published one of the most popular, reputable and comprehensive college rankings indices. U.S. News ranks colleges in a number of categories including National Universities, Liberal Arts Colleges, Regional Universities, Undergraduate Programs and Graduate Programs, among many others.
U.S. News uses various data points and qualitative measures to determine college rankings. They purport their rankings are based on the most reliable indicators of academic quality. Categorization of colleges ranked by U.S. News is based on the Carnegie classification–a widely accepted framework developed by the Carnegie Commission on Higher Education.
U.S. News says they use 16 "reliable" indicators of academic excellence to determine the relative value of each school the rank. Each indicator is assigned a value (determined by their staff) and then each college is ranked based on their composite weighted score.
Schools that do not use SAT or ACT scores in their admission decisions for first-time students are not included the U.S. News main college rankings. Schools with too few respondents to the peer assessment survey are also not included in college rankings. Schools with fewer than 200 students, too many nontraditional students, or those who don't accept first year students are also exclused from rankings.
The data used for ranking computations comes from surveys provided by the individual colleges. U.S. News collects data for about 1,800 colleges each year. Other data sources include the National Collegiate Athletic Association (graduation rates), the Council for Aid to Education (alumni giving rates) and the National Center for Education Statistics (financial resources, faculty, SAT and ACT admissions test scores, acceptance rates, retention rates, and graduation and rates).
U.S. News also uses a variety of methodologies for ranking colleges in diverse categories such as law, graduate studies, engineering, etc.
U.S. News relies on the following value indicators to generate its general undergraduate college rankings, which represent a majority of its ranking reports.Undergraduate academic reputation (22.5%) - represents the opinions of college presidents, provosts, deans of admission, high school counselors, college counselors and other peer groups with respect to a school's undergraduate academic excellence. Each peer group is assigned a weight.
Retention (20%) - this indicator is based on the assumption that the higher the percentage of first-year students who return to school and eventually graduate, the better the school and its programs. This indicator is based on a six-year graduate rate (80% of the retention score) and the first-year retention rate (20% of the retention score).
Faculty resources (20%) - this indicator is designed to measure student satisfaction with school faculty and professors. Data points included within this indicator include class size, faculty salary, professor's level of education, student-to-faculty ratio and percentage of faculty who are full time.
Student selectivity (12.5%) - represents the quality of students admitted to the college or university. This indicator has three components: admissions test scores, percentage of admitted student who graduated in the top 10% o their high school class, and the ratio of students admitted to applicants (acceptance rate).
Financial resources (10%) - represents the average spending per-student on instruction, student services, and other educational expenditures.
Graduation rate performance (7.5%) - assumes that the graduation rate of students is representative of the quality of a college's programs and policies. This metric compares actual graduate rate to predicated graduation rate.
Alumni donation rate (5%) - reflects the percentage of alumni with a bachelor's degree or higher who have donated to the school. The assumption is that alumni donations are a good indicator of students' satisfaction with the education they received.
With respect to other college ranking methodologies, we feel that U.S. News college rankings are above average. However, we believe they are weighted heavily on measurements that are not aligned with several important value indicators of a college education. Specifically, U.S. News ranking negate several highly relevant, longer-term indicators of value including (1) career success, (2) contribution to society and (3) increased quality of life. U.S. News rankings fall short in all three areas.
Princeton Review
The Princeton Review publishes two annual colleges lists, (1) the Princeton Review College Rankings report and (2) the Princeton Review College Ratings report.
Each year Princeton Review publishes its 62 college rankings list. Ranking categories include Best Colleges, Happiest Students, Party Schools and Jock Schools, among many others. College rankings are based almost entirely on survey feedback received from 136,000 students attending the 380 ranked schools. The survey consists of 80 questions and is designed to assess each student's opinion of:Academics/administration
College life
Their fellow students
Themselves
The Princeton Review College Rankings report is based almost entirely on student experience and opinion, and it only ranks the top 20 schools in each of the 62 categories. While useful, and interesting, we feel the college rankings report produced by Princeton Review ignores several value indicators of a good college, such as career development, contribution to society, retention/graduation rates, reputation and admissions selectivity. It does however provide an accurate representation of student sentiment regarding each college–and again, most people find Princeton Review college rankings very interesting.
Princeton Review also produces a "Ratings" report. Unlike the Princeton Review College Rankings report, the ratings report is a bit more quantitative in nature. The ratings report ranks colleges based on:Academics - measures study time outside of class, quality of student institution attracts, students' assessments of their professors, student-to-faculty ratio, average class size, use of teaching assistants vs full-time professors, registration and student resources.
Admissions Selectivity - based on class rank, admission scores, and average high school GPA of entering freshman, percentage of students from out-of-state, and the school's overall acceptance rate.
Financial Aid - measure of how much aid a school awards on a per student basis and how satisfied students are with the financial aid they receive.
Fire Safety - measures how well a school is prepared to prevent or respond to fires on campus.
Quality of Life - measure of how happy students are with their over college experience outside of the classroom.
Green - measure of how environmentally aware, friendly and prepared a school is.
So how valuable are Princeton Review College Ratings? It's open for debate. The ratings offer a more quantitative based view of a college's relative value, but the ratings include several measurements that we would argue are not a very good indicators of academic quality (i.e., Fire Safety, Green, etc.) And, notwithstanding any accuracy gained by including more quantative value measures, the vast majority of people still prefer reading the Princeton Review College Rankings over the Princeton Review College Ratings–even if it's simply to satiate their curiosity. Heck, who doesn't want to know which school was nominated as the top Party School in the nation?
Forbes
Every two years Forbes magazine produces its "America's Top Colleges" lists which covers a number of categories including Best Business Schools in America, Top Midwest Schools, Top Public Colleges and Top Liberal Arts Colleges, among others. Forbes magazine, through a partnership with the Center for College Affordability and Productivity (CCAP) based in Washington D.C. ranks 650 colleges nationwide.
What do we like about how Forbes ranks colleges? They focus on "output" over "input". They use metrics that measure actual ROI–the return students get from their investment in education. We like it! While their rankings aren't perfect, in our opinion, they certainly incorporate a few more accurate indicators of value than other college ranking reports.
Data used to produce college rankings are gathered from various sources. There are 12 general factors used to calculate rankings, with each falling into one of five categories. These categories include:
Student Satisfaction (25%) - satisfaction metrics are based on evaluations from RateMyProfessors.com (7.5%), freshman-to-sophmore retention rates (15%) fromIPEDSDepartment of Education database (IPEDS) and various other social media websites.
Post-Graduate Success (32.5%) - is based on alumni salary figures as reported by Payscale.com (10%), America's Leaders List (22.5%) and a few other publications that rank successful college graduates.
Student Debt (25%) - this an affordability metric that focus on cost of education and ability of graduates to pay back their student loans. It is composed of student loan debt load (10%), student loan default rates (12.5%) and predicted vs. actual percent of students taking out federal student loans (2.5%) to finance their educations.
Graduation Rates (7.5%) - this value indicator is based on a four-year graduation rate and considers both the actual graduation rate (5%) and the actual vs. the predicted rate (2.5%).
Academic Success (10%) - this metric is based on student success as measured by student scholarships and fellowships (7.5%) and percentage of students who go on to earn a doctorate degree (2.5%).
However, Forbes computes a college's overall annual ranking based on a moving average of the school's ranking scores over the last three years.
What do we like most about Forbe's College Rankings? In short, Forbes recognizes that the value of the education a college provides its students should be based in large part on the success of its students. This is where Forbes College Rankings stand out from the pack. Forbe's believes if a colleges students are successful, then the college is successful. If its students aren't successful, then the college isn't successful. Makes sense to us.
Wall Street Journal
WSJ is another publication the produces its own list of college rankings each year. WSJ produces rankings for a number of categories including Rankings by Major, Cream of the Crop, Top 25 Recruiter Picks, and Top Business Schools.
WSJ is unique among college ranking publications in that its methodology focuses almost exclusively on ranking colleges based on how well each school is able to help students land a job in key careers and professions.
WSJ works in partnership with Cambria Consulting to survey over 800 recruiters each year from the largest public and private companies, nonprofits and federal agencies in the nation across a myriad of industries. What does the survey produce? (1) A list of the colleges and universities that recruiters believe, based on their experience, produce the best-trained and educated bachelor-level graduates. (2) How many new graduates the recruiters hired the prior year from any colleges on the list. And (3) which school's graduates were best trained/educated in each major listed on the survey–and from which they actively recruited.
While narrow in scope, and relatively simplistic, we really like the WSJ methodology for producing college rankings. At the end of the day, we go to college to (1) prepare for a career and (2) land a job when we graduate. If this isn't accomplished, all else is for naught. The only major fault we find with WSJ rankings is that the rankings are limited to the top 100 colleges and universities. This really limits the usefulness of their rankings to support the research initiatives and decision making process of aspiring students, higher education institutions and employers. If the WSJ rankings included more schools in their rankings, it would be nice.Business Week
Every two years Business Week produces its list of the best business schools in the nation. It ranks approximately 112 full-time MBA programs using the following three measures:Student satisfaction (45%) - is based on the survey results of over 10,00 full-time MBA students attending 138 MBA programs across the nation. The survey addresses students' opinion and satisfaction with career development, cultural aspects, and skill development.
Employer satisfaction (45%) - is based on the survey results of over 1,300 recruiters working at 614 companies. Recruiters are asked to assess the performance of schools' graduates based on specific qualities including how effective they were on the job once they were hired.
Intellectual Capital (10%) - is based on level of research expertise of schools' faculty.
So how do Business Week college rankings stack up to the competition? Really well, in our opinion. Obviously, Business Week rankings only focus on colleges offering graduate level business programs–but that's okay. We think the value indicators used, and the respective weights they're assigned to calculate rankings, provide a fairly accurate measure of value with respect to how well an MBA program is going to help students prepare for a career, land a good job and become successful.
Kiplinger
Kiplinger produces what we consider a sound college ranking report based on pragmatic criterion such as affordability and academic quality.
Kiplinger starts with a list of 500 public four-year colleges and universities provided by Peterson's/Nelnet. Using various academic quality measures–including admission and retention rates, graduate rates, SAT and ACT scores, and student-faculty ratios–Kiplinger narrows its list to about 120 schools. It then ranks each school based on cost and financial aid.
With respect to ranking calculations, the academic quality measure carries almost two-thirds of the weight, with cost and financial aid accounting for a third of the weight.
We don't think the college rankings produced by Kiplinger are necessary the best, but they're certaining not the worst when it comes to assessing relative value of colleges based on common quality metrics.