I love TripAdvisor. I use it all the time and am rarely steered wrong. One thing I’ve learned in using it over the years is that the hotel or restaurant or activity that best meets my needs isn’t necessarily going to be on the first page of results. This is especially true in big cities, where my final choice may not even be in the top 50 listings.
Take looking for a restaurant in Washington DC, for example. Sure, my wife and I love Rasika at #14 and Jaleo at #40. But we also think Busboys and Poets at #61 has a great vibe and lots of yummy options for us vegetarians; DAS at #69 is a phenomenal choice for Ethiopian food after seeing a movie in Georgetown; and Oyamel at #90 has amazing Mexican food. Even way down at #177, Maggiano’s is always reliable for tasty Italian food when we’re in Friendship Heights; Fancy Radish at #178 has a great vegetarian menu; and Cuba Libre at #182 has a very fun atmosphere for drinks and snacks.
So how is it that places so far down in the rankings can meet our needs so well? At least part of the answer is that there are 2224 ranked restaurants in DC. Thus, all of the restaurants I just mentioned are comfortably within the top 10%.
The size of the list is very important to consider when using ranked lists. Sticking to the top 100 restaurants in a city that only has 200 makes a lot of sense, but not in a city with thousands of options.
Ranked lists with thousands of listings often have very little difference in quality between options that are hundreds apart. In this case, some restaurants that appear to be far down the list are actually ones that meet our primary criterion of providing a great place to eat just as well as those at the top of the list. When this is true, it makes sense to look to secondary criteria to determine which place to choose. Location may be important for the meal you’re planning. Perhaps it needs to be open for lunch on Sundays or be able to accommodate a party of 12. Or maybe you want a place with good vegetarian options, or that isn’t too formal because you’ll be out walking around in the heat for hours before eating. Since you know you can dive so deep into the list, you can and should use these secondary criteria to make your decision.
Okay, you’re smart and can see where I’m going with this. With almost 3000 4-year colleges in the US, why do folks assume that there are only 20 or 50 or even 100 that can meet the needs of smart, hard-working, talented students?
But, you say, choosing a college is a lot bigger decision than choosing a place to eat, and TripAdvisor isn’t really all that scientific in its approach to ranking. Good points. Let’s look at another example that has longer term consequences and for which an enormous amount of research has been done.
Consider this: how many of your family members, friends and neighbors own a minivan or pickup truck? Okay, now add to that the number who own any of the following brands of any type of vehicle: Honda, Hyundai, Acura, Nissan, Ford, Infiniti, Mini Cooper or Buick. Oh, and also include those who own any Toyota other than an Avalon. I’ll bet the total comes to over 50% of what’s probably a well educated crowd that researches their big decisions before making them, right?
Purchasing a new car is one of the biggest outlays of financial resources we make, and the depth and independence of the research at Consumer Reports makes it the most dependable of the numerous ratings sites available, in my opinion. They are incredibly thorough in their testing of multiple facets of a vehicle’s performance, and also include reliability and satisfaction ratings from consumers who own them. What they do for each vehicle would be roughly equivalent to US News and World Report sending researchers to every college they rank for a semester to attend classes, live in the dorms, eat the food, ask all of their fellow students about their experience, etc.
The overall scores at Consumer Reports are not provided in ranked fashion, but it’s easy enough to put them in order. There are currently 239 vehicles with an overall score, so the top 24 scores comprise the top 10%.
None of the types or makes of vehicles mentioned above is in that top 24 vehicles, and yet many very educated, smart consumers delve deeply into Consumer Reports’ ratings and purchase one of these vehicles because they have specific criteria they’re looking for and know they can still meet all of their needs by going deeper than just the top 10%.
You should be thinking this way if you use rankings to help choose a college, too.