A Better Way
to Rank Colleges

A physics professor from Wake Forest University is taking on US News and World Report and building a better ranking system in the process.

Jed Macoskoa's fascination with college rankings led him to start AcademicInfluence.com to determine which schools are best for the individual, not the other way around.

Currently, US News and other lists reflect institutional priorities rather than individual student priorities.

AcademicInfluence.com ranks schools based on influence, desirability and other indicators outside of institution control.

This method of measuring metrics prevents colleges from "gaming the system" to improve their rankings.

The goal is to have a ranking system that benefits all schools, rather than just the top schools, while also providing families with better decision-making data for choosing colleges.

Listen to the complete podcast or read the show notes below:

Welcome everyone. I'm Amy Sealy, president of Sealy Test Pros, helping students to succeed on all kinds of tasks. From eighth grade to grad school in Cleveland, Ohio. And I'm Mike Bergen, president of Chariot Learning, helping students with test, school and life based out of Rochester, New York. Between the two of us today, we have over 50 years of experience at the highest levels of the test, preparation, and supplemental education industries.

We both love to talk and learn about the latest issues in education, testing, and admissions. So let's get down to tests and the rest. The fascinating topic we want to explore today is which college rankings can you trust? But first, let's meet our special guest, Jed Macoskoa. Jed is a professor of physics at Wake Forest University.

He graduated with the BS from the Massachusetts Institute of Technology and worked with Noble Laureate Carrie Mulis at a biotech startup in Southern California. In the past 20 years, he has focused his research on the biophysics cancer, drug discovery, the mechanical properties of cells and cellular transport.

Jed's work has been cited thousands of times, and his patents have been developed in the biotech sector. One of his most fulfilling intellectual projects so far has been to improve academic rankings with a team at AcademicInfluence.com. Welcome. Thanks. It's great to be here. Jed, It's great to have you on the show.

The topic of college rankings, is one that comes up again and again, and we love to discuss it. But before we get into that, you have a fascinating academic and scientific background that doesn't really have anything to do with college rankings. Can you help us understand how you moved into this particular area?

Yes. Well, I'm at a great university that really values teaching. So there was a grant opportunity to use technology to teach better. And it was right when the iPad first was developed. So I jumped on that and said, Hey, we can make textbooks that are completely electronic born for being on an iPad or a device like that.

That launched me into developing educational technology and once I was in that field, it was just a matter of time be found before I found my way into a project that used big data to rank universities. And that's the one I really latched onto because the use of big data is so fascinating.

It kind of hits all the same exciting buttons and whistles that my research in biophysics also hits. It's discovery, it's finding trends, it's all kinds of interesting stuff. So, I've been doing that for six years now, using big data to rank universities and I just find it so intellectually fulfilling and fun and I've learned a lot along the way.

So what is academic influence? Well, that's the website where you can go and find how we rank universities and what the rankings are for what we call the influence of those universities. So it's just a website, but the engine behind the website does a lot of really cool things.

First, it ranks people according to their influence using big data, and then it finds out where those people went to university or where they work at a university and it distributes their influence points. Between those universities that they attended or are working at. And then those universities sum up all their points from all their people, and then they get a ranking based on how many points they have.

And that's the essence of it. But of course, you can slice and dice it lots of ways that suit your particular taste. So for example, one obvious way is you could divide the total number of influence points that a university has by the total number of people there because you're sort of competing for those influence points.

And if it's a very small undergraduate population, you could think of yourself as like having more access to that influence. And so that's what we call concentrated influence. And, you know, so different schools do better. You can imagine Caltech does pretty well on concentrated influence, whereas, a school like Harvard is gonna just rack up the influence points from all the people that have been affiliated with that school.

We've come up with other kinds of rankings too, to sort of supplement because the influence of the university is not the whole story. I personally was really proud of developing a tool that ranks universities by what I call desirability or what our company calls desirability, that looks at actually where students have decided to go when they have the choice between two schools.

So that one is a kind of an all-encompassing quality factor that has real students giving us the data that we need, in order to do that. Again, it's a big data project. It's a lot of, well, it's really interesting and it makes sense now to see how you might have fallen into exploring all the data associated with rankings, but also, you say it's really interesting and I'd like to, before we dive back into different ways to parse college rankings.

Let's just talk about the big picture. Why are college rankings so interesting to those that study them and influential to everyone else that's trying to use them? Well, they're interesting to me because they're trying to solve a problem and they use different strategies, so, that's always interesting if you're trying to solve a problem and you like to see how other people are trying to solve that same problem.

So that's absolutely fascinating. Now, why are they so important to everybody else? Well, that's an interesting story. It has to do with how universities had a golden age back in post World War II until the 1970s era where university funding from the government in the form of research grants was growing exponentially.

And then that stopped and universities had to pick up the slack if they wanted to continue to support all the programs and keep growing the way that they had been used to. And so they raised tuition as you know. Once tuition started to go, and that was by the 1980s, people started really feeling they needed a way to decide where their money would be best spent.

So that's why rankings became so important to everybody is because they wanted to know, am I spending my money wisely? It's a lot of money I'm spending and so I wanna know is school A better than school B? Because I really wanna make sure my money is well spent.

And that's how rankings got going and of course the, the big one is US News and World Report, which started in 1983. Just in time to sort of catch that wave of people saying, wait, I don't just wanna send my my kid to any old school. I wanna make sure it really counts. Well, that's a funny the way you put it, right?

I don't wanna send my kid to any old school. We find ourselves in an age now more than ever with attention on specific schools where specific names seems to have crystallized. Yes and there's this sense that there are a handful of great schools and everyone else has to suffer along.

Like if they can't get in, that's it. They're out of luck. I guess there's a difference between what people look to college rankings for and what they should be looking them for. In a sense, I get the impression that you and your colleagues are solving a problem with interpretation.

So what do people when they look at college rankings, they think they're getting a list of schools that would be better towards the top of the list and worse towards the bottom list of a deal for their money. So if, if they're gonna be like, well, these ones are higher up on the list, so these ones are more worth the money where I'm spending to send my kid.

And if you're a kid, you're like, well, it's more, more about the time. Like, is this worth the four years I'm gonna be spending there, you know, plus my parents money, or, you know, if I'm getting scholarships, fine, but still I wanna make sure that this counts. So that's what they think they're getting.

Now, what they're actually getting obviously depends on which list they're looking at. So again, most people are looking at US News and World Report so what they're actually getting is largely a measure of prestige, as seen by the eyes of three individuals at each university, the president, the provost, and the dean of admissions, because those are the three people that are surveyed.

Which schools do you think are in the top and right. Once they get all those surveys, then they create a portion of the score that they give, but, it is a portion that really swings the needle a lot. So that ends up being what they get. And it's an interesting thing, but is prestige equal to, am I getting my money's worth here?

And is this the value place to go? Yeah, the value. That's the big question here. I will say it's nice you mentioned the word crystallize, Mike, because I've done some reading on what the best schools were back a hundred years ago. And what was funny is that each city had a certain set of schools that were nearby.

Okay. Which makes sense. And so students would be sent to those schools or they would pick a far away prestigious old school that people had heard about, but it was different for each city. So St. Louis would send theirs to Princeton maybe and, I don't know, Kansas City might send their kids to Yale. So it was just interesting how and what I imagine is that people started talking at cocktail parties for the wealthy people.

And they're like, well, where are you sending your kid? Oh, okay, well, and then that group of wealthy people in that city would all sort of get the idea that that's the place to send your kid. But that's changed now because, each city doesn't really have its own idea if a far away school might be good.

Which far away school should I send my kid to? and the nearby schools also, are no longer as great of an option because they're so far down the list? Right. Not because their programs are worse. No, just because they're not the consensus top 20, top 50, they don't have division one football or basketball.

There's all these factors that shoppers use, speaking to the quality of educational programs, especially when you talk about specific majors. Right. And what would we have wanted back in 1983? Would we have wanted US News to give us a list for each city? This is the list. Then we would've been scratching our head, well, why is the list different for Kansas City and St. Louis.

Why are they sending to two different Ivy League schools? You know, we wouldn't have really understood that. So there was no way to keep that going with the tuition rising as it did. People wanted something to latch onto and they latched onto that. That was sort of the one that kind of caught people's eye.

And it has never been dethroned. There's no new one that took its place. I don't know if there ever will be, we at AcademicInfluence.com offer one of many alternatives. There are many alternatives out there, and I just like ours because it's so cool, So I'm happy to offer an alternative and spend my time researching and making it as best, the best kinda list we can make because it's so cool.

And I think the coolness factor will appeal to people and of course, because there already is US News, they're gonna look at our list and compare it to that list. And, you know, they won't only look at our list. And by the way, I think that's the healthiest thing to do. Don't look just at one list, look at a bunch of lists.

So when it comes to trusting lists, right, first of all, I guess I'll ask the question. Families, students, educators, should they look at any given list and just swallow its premise, wholesome? No, of course not. I could see where you're going with that, but of course not. Now, the premise of any list, and we probably make it sound like this too on our list, is that if you look at this list, you'll find a great [college].

Okay. And you should not swallow that idea because a great school for one person, of course, is gonna be different for a great school for another person of course. But you can trust some aspects of it, which I think is sort of. Where you're going with this question. Wonderful. That's exactly, you're reading me right Jed, so what?

What can you trust? Of course you can't trust it. Thank you. There we go. Exactly. So you, I'm just gonna stop talking , right? I mean, you can trust each list that's out there, starting with the one that everybody knows about US News and working your way through all the different ones you can find on the internet.

For what it's good at. Okay, so I can start with the two that are nearest and dearest to me. The US news, which everybody thinks about, it's good at telling you which schools are seen as prestigious top rate. It's seen as that, so you know if what you most care about is that other people when they hear where you went to school will see.

As better than what other options you have. If you have five options and you pick the one that's highest up on that list, then you'll be maximizing that sort of feeling that you get when you tell somebody. Okay, now, the bottom line is like, after a certain point, maybe the 50th school on that major list, it doesn't really matter because at that point they're sort of all blending together in people's minds.

So if you're not going to one of the top 50 schools, I don't even think that piece of the puzzle could even give you much value. Right. You're saying it doesn't matter whether you go to the 75th versus the 100th or 5,075th, because I don't think we're talking about thousands of schools.

Yeah, that's a thousand school gap. But I don't know if it matters. I don't know. Maybe it does. But there are other aspects that the US News does sort of put into its equation that maybe start mattering. Down there I was saying that the lions share of their ranking is those surveys that they put out and ask the present provost and dean of admission, but maybe down the road down when you get down to the 75th or 175th or a 1000th school, maybe some of the other things matter, like how much they're spending per student and stuff like that.

So, maybe that is good to look at. But anyway, you get what you pay for with that list is you get. First and foremost. Well, I think what you're suggesting though is you have to kind of know like upon what basis is this listing or ranking. Yes. Created meaning if it matters, if there's a feature, a factor that matters to you.

Right. Like whether it be, are my professors doing research? Or will I have the opportunity at research if, if that matters to me. Then, I want a rankings list based upon that factor. That's right. So it's what's very interesting is that people are very different.

And so everybody comes to colleges with different personal prior priorities, right. Not the institutional priorities. That is what the sort of lists, I think reflect what do institutions wanna signal as opposed to what do students want in that institution? Absolutely. So you've gotta be kinda savvy, right, about what do I want?

Yeah, I agree, and, I will say that I've done maybe a poor job of explaining what is the basis of US News and World Report. It's a place to start. You can look at it for yourself. No, I think you actually really hit on something that many consumers don't understand that besides all of the numbers and the research that goes into those rankings, that survey they do, where they ask other presidents and provost, which schools do you think are, are great.

Right. They ask each other highly. That's distinctive. Yeah. Right. That doesn't get baked into other rankings. Academic influence has a completely different metric, but a lot of people don't know the point that you're highlighting. So yeah. I'm glad I said that. You actually shared a lot of great information and, and I'll just say that the reason our list is a list.

I mean, again, you shouldn't trust it as being the best list for you, but the basis of our list is people. We think that universities are primarily good because they have people there and been a part of university since the day I was born. My dad is a professor and I've been on university campuses and I've been a professor my whole career.

You know what? I think that's right. I think people are what drive universities to make them great. So we rank universities on the people, and that's what I like about our ranking. So if you're interested in going where there are good people right there, right then, and that they have graduated good people in the past, then you can look at our list.

Okay, and you can slice and dice it the way you want, if you like. On a per student basis, you can look at our concentrated influence. If you wanna see what other students have done in the past, you can look at our desirability rankings and kind of get a feel for that. That's why I like our ranking.

So you used a word before that I think is interesting. You said trust. So how can or should applicants trust the rankings that they might be using to either do their college research or do their college books? Yeah, I mean, they can trust that hopefully there's no funny business going on and it's done the way that the website says it's done.

And so you trust that that's what they're doing. There's nothing else you can do. You can't go and inspect them or audit them. So you trust that they're doing it the way they say to you and that therefore that's information now for you about the prestige of the university or the quality of the people at the university.

Niche.com looks at people's recommendations for the university. So, you know whether it's gotten lots of good recommendations or not. Other lists use other things, how much money you can expect to make after graduation. Again, you can trust that they've at least tried to give the best possible number of return on investment or popularity among graduates or whatever they say they're using to rank it.

You can trust that's what it is and then you can use that to say, well, which one do I care about? and then I'll sort of more look at that list than at the other list. But I can look at all of them and sort of get an average idea in my mind. So Jed, when you look at colleges and you think about the value proposition in terms of interest and kind of concentrated interest that students would get.

Do you differentiate schools based on majors or any other factors? Yes. That's one really nice thing about our website is because we base it on big data on the names of these people and on subject areas that the people's names are associated with. We can rank it by just about anything you want.

Anthropology, some real subset of anthropology, chemistry, biology, whatever you want and you can rank schools by that now. How much can you trust it? Well, you kind of have to think about, okay, they say they're using these big data database. How big are they? Is it enough information to really get so finely grained that I can really trust that this school is better than that school in some tiny little subset of anthropology?

I don't think you should trust it that much, but you know, it's a place to start and it's certainly good for broad areas like computer science. If you wanna go into computer science, well look at our computer science rankings and see which schools are the top in that. I mean, certainly that will work.

And so it's a lot of fun. You can play around on our website and sort by whatever you're interested in. So when it comes to fun, I think about gaming a system that's, I'm a test prep person, so I always think about how and we know. Famously about schools that have tried to game the US News and World Report rankings and there are ways to do it.

The playbook is public by now. So I'm wondering if this, I'm thinking about as you're describing Academic Influence, so you were using the example of anthropology? If I wanted to improve my institution's ranking in your database, would the best way for me to do that, say for anthropology, be to go and hire an acclaimed professor?

Yes and that would be the way. And hopefully understand you graduate some really acclaimed, anthropologists, but you know, that takes time. Right? It's hard to game our system because everything that you do takes a lot of money or time and what you end up doing is improving your university.

So, that's good and you know what a lot of people have criticized US News and World Report for, saying, look, your ranking has not really helped people get better universities, they've just gotten universities that game the system and that gaming doesn't really help students. So I mean, I don't know.

Well, I think from what you're saying though when we talk about US News and World Report, we're really saying, because it's the institutions themselves that are providing the data and they're doing all of like the, you know, kind of looking at, the massaging, they're massaging the system. When you get a ranking system that's sort of external, even let's say if students were ranking their own institutions, you're going to get a totally different perspective, so I think that's the interesting thing, it sort of feels like with if if institutions get to pick the data they provide, they're going to tell their story in the most favorable light possible.

Right? So that gives 'em the ability to game the system. You know, again, the issue would be how much time does it take? Yeah. You know, from one year to the next, can you do a lot or over five years? Does that make a significant impact? Well, it took about 10 years for Northeastern University to start.

That's what I was gonna say. Northeastern is an example. Right, right. We talk about that all the time. It's an interesting example. Here's the thing though, I think actually Northeastern is a lot better of a place now that it has a higher ranking than it was before, but which came first?

Is it a cause or an effect? And what I would argue is that one way to argue this is that they gamed the system or did whatever needed to be done to get lower and lower numbers, so higher and higher in the ranking. And then by being higher in the ranking, they got better students and professors were more willing to work there.

And so then they got a better school. So the problem with that is that it's a zero sum. For every school that goes up and gets better students, you get schools that go down and what I would like is a ranking that makes all boats float higher. A rising tide that rises all boats, and that would be one that says, Hey, you want to get better?

You don't make your students more influential. Okay, now everybody's scrambling. It's still kind of a zero sum game because only one school can be number one on only one school, number two or some ties or whatever, but everybody's striving for that and all schools get better to attend now that would be great.

Right. So that, because every student benefits exactly from that, if you're going to an institution that students actually benefit. For sure. Exactly. So you got it. So I am really excited about our ranking because I feel like it does that, and I don't know whether or not US News has done that, but people argue that it hasn't.

So I think this is worth trying and worth trying something new and I encourage all of your listeners to go and check out AcademicInfluence.com and see for themselves if they think it's better, you know, so the proof will be in the pudding. That makes perfect sense. Well listen, we could obviously talk about college rankings all day.

Unfortunately. We are out of time. Thank you so much, Jed, for joining us today. You are welcome and anybody wants to reach out to me. Can email me at macoskjc@wfu.edu. So thank you so much for the time. It's been wonderful to talk to you. You, Amy and you Mike. Awesome.

We hope you enjoy this discussion as much as we did. Be sure to join us for another fascinating topic and guest on the next Test and the Rest. Did you enjoy this episode as much as we did find expanded links and more for this and all of our episodes at our show page of https://gettestbright.com/tests-and-the-rest/. While you're there, be sure to explore everything the larger GetTestBright.com has to offer, including real expert, unbiased answers and the toughest technicians testing questions.

Of course, we'd love for you to rate, review, and subscribe to Test and the Rest on your favorite podcast platform. If there's a topic or guest you'd like us to feature, just let us know and spread the word.