Lawyer Ranking and Rating Sites: Why They Are Problematic

With 228 million adult internet users in the United States, it is no surprise that people are using the internet to find lawyers when they need them.  And entrepreneurs did not miss a beat; lawyer ranking/rating sites (to be referred to as “lawyer evaluation sites”) have become not only common, but trusted by consumers and legal professionals.  However, these sites misrepresent themselves as being driven by the desire to educate consumers when in fact they are driven by profits.

Upon investigating lawyer evaluation websites, their flawed and self-interested processes for calculating their rankings/ratings becomes abundantly clear; however, where they explain the methodology behind their evaluations is not on pages consumers would generally look.  Three lawyer evaluation websites held in high esteem in the law community that are particularly guilty of this are Avvo, Super Lawyers, and U.S. News & World Report’s Best Law Firms.

Avvo
Avvo creates and hosts profiles for lawyers.  Lawyer profiles contain background information about lawyers gathered by Avvo, and possibly given to Avvo by the lawyers themselves.  Lawyer information is entered into Avvo’s secret algorithm to produce the Avvo rating.  Avvo refuses to release how the information is weighted in their algorithm and freely admits that the way in which information is weighted is based solely on Avvo’s opinion.

Additionally, the way for lawyers to fully utilize the marketing capabilities of Avvo is for them to pay a monthly fee, and buy “Display Ads” and “Sponsored Listings.”  Profiles of whose who pay for these options are seen substantially more often by consumers than those who do not.

The problem with Avvo:

  • Weak criteria is used to create the Avvo Rating.
    • Lawyers may freely enter information about themselves onto their own profile and it is not evident that Avvo verifies that information.
      • This data is then weighted and included in the Avvo rating.
  • Information is collected by Avvo through the internet, and other “blind” avenues.
    • Avvo is known to post incorrect data to lawyers’ profiles.
  • It is weighted based purely on Avvo’s opinion of what is important to determine a “good” lawyer.

Super Lawyers
Super Lawyers is a lawyer rating publisher that determines the best lawyers in a given state.  According to their website, lawyers deemed “Super Lawyers” and “Rising Star” are highly regarded by the law community for their perfessional achievement.  Super Lawyers publishes Super Lawyers and Rising Stars lists, as well as ads lawyers pay for, in their magazines and special sections in other magazines throughout the country.

Super Lawyers lists are the top 5 percent of lawyers in each state and Rising Stars lists are the top 2.5 percent of lawyers in each state; both are based on the criteria established by Super Lawyers.  Super Lawyers’ lists are determined by 4 steps.  Their first step is to determine who will be in their pool of candidates; this is based on any one of the following: formal nominations by peers, research conducted by attorneys employed by Super Lawyers, and informal nominations. Though Super Lawyers claims to implement “safe guards” to prevent “back scratching,” what lengths they go to in order to protect the integrity of votes is not publicized.  The second step is to assess candidates in the pool based on 12 Super Lawyers’ determined criteria; their criteria is simply stated and not expanded beyond that.  The third step, which is skipped in the Rising Star determination, is for those candidates rated highest in their area of practice to evaluate candidates that are rated lower than them.  In the fourth step, those candidates that rated the highest when compared to others who belong to a firm of the same size are selected to be Super Lawyers or Rising Stars.  Information to be published and lawyer backgrounds are supposedly further researched to ensure accuracy.

The problem with Super Lawyers:

  • The candidate pool is determined by lawyers through nominations and hired lawyers through judgments and research.  It is not clear that conflicts of interest are addressed; nor is it clear how vigorously Super Lawyers’ professed “safeguards” are pursued to ensure that back scratching does not take place.
  • They mention that they conduct research, but are not remotely specific about how or what venues they use.
  • How much weight different criterions are given, especially those that can be misleading (e.g. “transactions” and “representative clients”), is not publicized.
  • A step in their evaluation process involves asking the highest rated candidates to evaluate other candidates in their same category.
  • Supposedly, Super Lawyers represent 5 percent and Rising Stars represent 2.5 percent of all lawyers in their relevant state; whether or not these percentages are based on lawyers who are actively practicing law is not publicized.  If these percentages include non-practicing lawyers, the “prestige” of Super Lawyers is lessened.
  • Though it is denied by Super Lawyers, it is implicit that the more a lawyer pays to advertise with them, the more exposure they will have with people who nominate Super Lawyers; and more exposure translates to a higher likelihood being nominated to Super Lawyers.

U.S. News and World Report’s “Best Law Firms”
Joined with Best Lawyers, U.S. News and World Report (USNWR) annually releases a ranked list of the top law firms in the United States.  Rankings are determined by surveying various law-related groups in the U.S.; the methodology behind their surveys is listed on their website. Their surveys were developed by sending a primary set of surveys to various groups in the law community to determine what factors are relevant in evaluating a law firm’s quality.  The surveys they developed are sent to selected groups in the law community annually to determine law firms’ rank.  The following are the details of the surveys according to their website, as well as questions relevant to evaluating the “Best Law Firms” lists:

  1. Client surveys are sent to 52,480 clients.
  2. Lawyer surveys are sent to 43,900 lawyers, including every U.S. lawyer listed in Best Lawyers.
  • How were those lawyers that were not listed in Best Lawyers chosen?
  • Best Lawyers released the report with USNWR.  Sending a survey to determine the best law firms to all of those listed as Best Lawyers is a conflict of interest.
  1. Surveys are sent to 2,314 marketing officers and 2,322 recruiting officers.
  • These officers have a conflict of interest because they are hired by the very law firms that they are assessing.  The better their law firm looks, the more they will benefit financially.
  • Their assessment seems irrelevant; the only way that these officers are related to lawyers it that they are being paid by law firms to generate business and recruit new lawyers.
  1. Surveys are sent to 8,597 firms without marketing- or recruiting-office contacts.
  2. Associate surveys are sent to 2,322 firm associates and 1,775 summer associates.
  • These lawyers have conflicts of interest.  They are employed by the law firms they are assessing.  Those surveyed have an incentive to rate their firm higher than the firm deserves because it ultimately benefits the associates financially.

Those chosen represent only a small fraction of the law community; a question relevant to all of the surveys is how and why those surveyed were chosen.

The problem with USNWR:

  • How they determine who they send their surveys to is not publicized.
  • One can assume that those they send rank-determining surveys to are the same people they used to determine what questions to include in the surveys; effectively, they are asking a only a small, select group their opinion on something that affects the law community as a whole and the public.
  • The majority of those surveyed have obvious conflicts of interest.  For instance, every lawyer on Best Lawyers’ list is queried despite Best Lawyers being one party in the Best Law Firms publication; asking the “Best Lawyers” to determine the “Best Law Firms” demonstrates a clear conflict of interest.

The Investigation
The American Board Association (ABA) was prompted to investigate lawyer evaluation services by the House of Delegates in 2010.  The investigation’s purpose was to determine if the questionable methods said services use to determine a lawyer’s or law firms rating/ranking are causing harm to people who use their services.  The ABA’s Commission on Ethics 20/20 (ABACE) concluded the investigation in August of 2011; it claimed that it found no evidence of an obvious problem.  The ABA admitted that evidence of lawyer evaluation sites affecting the public or law community was absent altogether.  The absence of evidence is due to lawyer evaluation services being relatively new.  As a result, no research has been done to understand how they affect the public.  The ABA also stated that it was not willing to hire the experts such research would require.

The bottom line:  Ultimately, the investigation was pointless.  Its purpose was to determine if harm is being done but no research has been conducted, so no evidence exists.  And the ABA is not willing to change that.

An experienced car accident lawyer will be able to assist you if you’ve been a victim of a car accident.

Close

CONTACT US

Vestibulum id ligula porta felis euismod semper. Nulla vitae elit libero, a pharetra augue. Aenean eu leo quam. Pellentesque ornare sem lacinia quam venenatis vestibulum. Maecenas mollis interdum!

Subscribe error, please review your email address.

Close

You are now subscribed, thank you!

Close

There was a problem with your submission. Please check the field(s) with red label below.

Close

Your message has been sent. We will get back to you soon!

Close