Tuesday, October 20, 2015

Is Salt Lake City America's Vainest City? Correcting Forbes

Background

In 2007 Forbes magazine published an article entitled "America's Vainest Cities" which dealt with how Americans have become obsessed with their looks and as a result were having more plastic surgery. Associated with the article was a list of the 10 vainest cities in America, according to their criteria which was largely based on the number of plastic surgeons per 100,000 adults. This was a fairly unremarkable list except for the fact that #1 on the list was Salt Lake City. That result caused a minor stir among some news outlets in Utah. Many expressed surprise, while others said in triumphant self-righteousness "I knew it!". But the furor quickly died down.

The article, and the fact that they ranked Salt Lake City as #1, would have been forgotten and only mentioned in the occasional blog post and local news article had it not been resurrected by a recent article published in Time. The article had the provocative lead "Believe it or not, the rise in Mormon breast implants and $100,000 Jewish dowries can explain why you're alone on Friday night."

While the article in Time dealt mostly with male to female ratios in Utah, it did cite the 2007 Forbes article to support its assertion that Mormon women in Utah were going to great lengths to attract a potential mate. As stated in the article, "A culture of plastic surgery has taken root among Mormon women." So according to the Time article, this "culture of plastic surgery" has resulted in Salt Lake City having the highest number of plastic surgeons per 100,000 adults, based on the Forbes data. This argument was not central to the Time article but it did support a major point.

The Time article has been getting a lot of traction lately with a few news articles and blog posts in response and numerous shares on social media. The first time I ran across the article I glanced over the assertion that Salt Lake City has more plastic surgeons per capita than any other city and did not think anything of it. The second time someone I know shared something that used the original Forbes article I stopped just long enough to do a simple calculation and determined that something was wrong with the original numbers from Forbes.

The Original Results

Below is the original top 10 "Vainest Cities" with the number of plastic surgeons and surgeons per 100,000 adults. The data comes from 2007.

RankStateCitySurgeonsper 100,000 adults
1UtahSalt Lake City456.0
2CaliforniaSan Francisco1755.4
3CaliforniaSan Diego1155.2
4CaliforniaSan Jose705.2
5FloridaMiami2185.2
6KentuckyLouisville414.4
7TennesseeNashville484.4
8VirginiaVirginia Beach514.1
9New YorkNew York5914.1
10CaliforniaLos Angeles3844.1

Even though the Forbes article also gave data about the amount of money spent on cosmetics, personal care, hair dye, and other similar items, the ranking was based entirely on the number of plastic surgeons per 100,000 adults. They acknowledged that Salt Lake City coming in first was surprising, but that did not deter them or raise questions about their data or methodology.

They explain their data gathering and methodology in the following way.
"To rank the cities, we collected the number of plastic surgeons in the country’s 50 most populated cities. We excluded residents under the age of 18, leaving out a small number of children and adolescents who undergo reconstructive or cosmetic plastic surgery.... We obtained the number of plastic surgeons in each city from the American Society of Plastic Surgeons, a membership organization that represents about 90% of all plastic surgeons certified by the American Board of Plastic Surgery."
If we look at the data from Salt Lake City, 45 surgeons and 6 surgeons per 100,000 adults means for their calculation they assumed that in 2007 Salt Lake City had a total adult population of 750,000 adults. According to the 2010 census 23.6% of the population of Salt Lake City is under 18, which means Forbes was assuming a total population of ~980,000 people in 2007. The current (2014) estimate of Salt Lake City's population is 190,884. Already we see there is a problem since Forbes was assuming a population almost 530% larger than the actual population of Salt Lake City. Using 2010 census data I checked how the other cities on the list fared.

RankStateCityper 100,000 adultsUnder 18"Apparent Population"2010 Population
1UtahSalt Lake City6.023.6%981,675186,440
2CaliforniaSan Francisco5.413.4%3,742,195805,235
3CaliforniaSan Diego5.224.0%2,909,9191,307,402
4CaliforniaSan Jose5.224.8%1,790,098945,942
5FloridaMiami5.221.7%5,354,161399,457
6KentuckyLouisville4.424.3%1,230,936597,337
7TennesseeNashville4.422.0%1,398,601626,681
8VirginiaVirginia Beach4.127.5%1,715,728437,994
9New YorkNew York4.124.0%18,966,6248,175,133
10CaliforniaLos Angeles4.123.1%12,179,2643,792,621

As can be seen in the table above the population used for the calculation in the original Forbes article is much higher than the actual population of the cities. Based on the numbers it would seem that Forbes used the 2007 metropolitan population which includes more than just the listed city. That would not be a problem, but based on their description of their methodology they were not considering metropolitan areas, but individual cities.

The next logical question is to check if the number of plastic surgeons in each city was accurate. I went to the web site of the American Society of Plastic Surgeons to check Forbes's numbers. For Salt Lake City I found only 15 surgeons with an address in Salt Lake City and a total of 47 for the entire state of Utah. There are a total of 26 plastic surgeons with an address somewhere in Salt Lake County but nowhere near the 45 reported by Forbes. I did notice that if I searched for a specific city then the American Society of Plastic Surgeons site returned results from nearby that city, though "nearby" is a relative term. A search for surgeons in "Salt Lake City" returned results from Ogden and Provo, which are both outside the Salt Lake City metropolitan area, but still "nearby". But further down the list are results from Idaho, Wyoming and Colorado. So assuming that the search mechanism on the American Society of Plastic Surgeons website has not been modified since 2007 it would be quite easy to find more plastic surgeons in a particular city than are actually in that city.

I have no insight into how the reporters did their search, but if they were using 2007 metro area populations then they may have rationalized using all or most of the results returned by entering a specific city into the search field. The problem with this is that they may have included plastic surgeons from as far away as Logan and St. George, and then only used the population of the Salt Lake City metro area. This would skew the numbers for all the cities in unexpected ways.

Corrected Results

With this in mind I decided to check all the cities on the Forbes list and also to expand the search to all cities, towns, villages, hamlets or Census Designated Places that had a plastic surgeon listed on the American Society of Plastic Surgeons website. For my analysis I used the total population from the most recent available estimates.

For cities over 100,000 people I used 2014 population estimates, for locations smaller than that I used the most recent estimates, which generally were 2013, but with a few exception for 2010 data and an even smaller number (~5) for which the most recent data was from the 2000 census. This covered a little more than 1400 locations in the US with populations ranging from 132 (Crestone, Colorado) to 8,491,079 (New York, New York). This included five military bases for which there is no population data.

Below I give the results for the original top 10 "Vainest Cities" as reported by Forbes, along with my count of surgeons in that city and a recalculation of the number of surgeons per 100,000 people.

Forbes RankStateCitySurgeonsper 100,000 adultsActual SurgeonsActual/100,000
1UtahSalt Lake City456.0157.9
2CaliforniaSan Francisco1755.4505.9
3CaliforniaSan Diego1155.2382.8
4CaliforniaSan Jose705.2111.1
5FloridaMiami2185.24710.9
6KentuckyLouisville414.4233.8
7TennesseeNashville484.4314.8
8VirginiaVirginia Beach514.1153.3
9New YorkNew York5914.11922.7
10CaliforniaLos Angeles3844.1451.1

As you can see when we only consider the cities and not the metro areas the rankings change significantly. Just in these cities Salt Lake City drops to #2 with 7.9 surgeons/100,000, while Miami moves to #1 with 10.9 surgeons/100,000. Also both New York and San Jose drop to last place with 1.1 surgeons/100,000. Just this small sample indicates that my new methodology will significantly change the results. If we just consider the 50 largest cities, not metropolitan areas, the top 10 become:

CitySurgeonsPopulationSurgeons/100,000
Atlanta51456,00211.2
Miami47430,33210.9
San Francisco50852,4695.9
Boston36655,8845.5
Dallas671,281,0475.2
Baltimore31622,7935.0
Nashville31644,0144.8
Seattle31668,3424.6
Austin38912,7914.2
Houston912,239,5584.1

In this case Salt Lake City does not make it into the list of the 50 largest cities since it is currently the 124th largest city in the US. All of the other cities in the original Forbes top 10 are in the list of 50 largest cities in the US. If we include Salt Lake City despite its lower rank it would come in at #3. But if we expand our criteria to include all cities (and Census Designated Places) larger than 100,000 people then the top 10 changes with Scottsdale, Arizona leading the pack.

StateCitySurgeonsPopulationSurgeons/100,000
ArizonaScottsdale40230,51217.4
MichiganAnn Arbor16117,77013.6
South CarolinaCharleston17130,11313.1
MichiganGrand Rapids25193,79212.9
CaliforniaPasadena17140,88112.1
GeorgiaAtlanta51456,00211.2
GeorgiaSavannah16144,35211.1
FloridaMiami47430,33210.9
PennsylvaniaPittsburgh33305,41210.8
FloridaFort Lauderdale19176,01310.8

In this case Salt Lake City comes in at #22 right behind Cincinnati, Ohio and Tampa, Florida. It even has a lower rate of surgeons/100,000 people than Metairie, Louisiana which is not even a city but is a Census Designated Place near New Orleans. As can be seen, only Atlanta and Miami are on this new to 10 from the previous top 10 where the selection criteria was limited to the 50 largest cities. This shows that as we go to progressively smaller populations the rate of surgeons/100,000 goes up. This means that a population limited sample, such as the one used by Forbes, even if they used correct numbers, would skew the results since the highest rates of surgeons/100,000 occur in the smallest population centers.

We can see this effect if we now include any population center with a plastic surgeon. Below are the top 10 population centers with the highest rates of surgeons/100,000 people.

StateCitySurgeonsPopulationSurgeons/100,000
ColoradoCrestone1137729.9
MarylandChevy Chase202,824708.2
MichiganSt. Joseph1276362.3
CaliforniaBeverly Hills8934,658256.8
PennsylvaniaNew Castle1407245.7
New YorkGreat Neck2110,088208.2
New YorkLake Success63,030198.0
KentuckyCrestview Hills63,159189.9
New YorkCooperstown31,834163.6
PennsylvaniaLebanon1670149.3

Here the highest population is Beverly Hills California with 34,658, but has 89 plastic surgeons for a rate of 256.8 surgeons/100,000 people. As an interesting #1 is Crestone, Colorado, a tiny community of 132 people. While the 89 plastic surgeons in Beverly Hills might tell you something about that city, the one plastic surgeon in Crestone only tells you that a plastic surgeon happened to open her practice there. As can be seen this list is dominated by tiny communities with very few plastic surgeons that presumably service a larger area. Then there are others like Beverly Hills, Great Neck, Lake Success, Chevy Chase and Crestview Hills that are suburbs of larger cities.

These extremely high rates of plastic surgeons/100,000 for extremely small populations indicates that a better selection criteria would be a limit on the number of plastic surgeons rather than a limit on population. To show this in graphical form I plot the number of plastic surgeons/100,000 people vs. population for the 50 largest cities, cities with more than 100,000 people and for all population centers.
If we consider the above graph there is no real trend, but when we extend it to all cities a distinct trend emerges.
This demonstrates the problem of using a population limited sample.

The Beverly Hills

Upon further inspection there are a few places that stand out from the rest. These are the ones I like to call The Beverly Hills. Beverly Hills, California is in a class of its own with 89 plastic surgeons, but there are several cities just like it with a small population, close proximity to a large metropolitan area and a large number of plastic surgeons. Below are 24 cities that fit the Beverly Hills classification.
StateCitySurgeonsPopulationSurgeons/100,000
MarylandChevy Chase202,824708.2
CaliforniaBeverly Hills8934,658256.8
New YorkGreat Neck2110,088208.2
FloridaNaples1520,53773.0
CaliforniaNewport Beach4787,27353.9
New JerseyParamus1426,34253.1
New YorkGarden City1022,55244.3
KansasLeawood1332,99139.4
CaliforniaLa Jolla1846,78138.5
VirginiaMcLean1848,11537.4
MinnesotaEdina1749,59634.3
DelawareNewark1132,54933.8
MarylandAnnapolis1038,88025.7
FloridaBoca Raton2289,40724.6
FloridaSarasota1353,32624.4
CaliforniaPalo Alto1566,64222.5
MichiganTroy1580,98018.5
MarylandRockville1162,33417.6
ArizonaScottsdale40230,51217.4
GeorgiaMarietta1059,08916.9
CaliforniaWalnut Creek1166,90016.4
CaliforniaSanta Monica1392,47214.1
MichiganAnn Arbor16117,77013.6
TexasSugar Land1183,86013.1

Making this list are small cities that no one making up a list such as "Vainest Cities in America" would think about including but only show up when you look at all of the data. Cities like Edina, Minnesota; Leawood, Kansas; Troy, Michigan; and Chevy Chase, Maryland. Most are not surprising for anyone who lives near one of these places (I was personally not surprised by Scottsdale). I have included La Jolla even though it technically is part of San Diego, since it is, at least by this measure, the Beverly Hills of San Diego. Almost all of these cities have less than 100,000 people but a large number of plastic surgeons. If Forbes had wanted to do a real list of the vainest cities in America this would be a good place to start.

The Centers

Then there are a group of cities that I call The Centers. These cities are usually the center of metropolitan areas, have more than 100,000 people and are surrounded by smaller cities with few or no plastic surgeons. These cities obviously service more people than those who live in that city. These include cities such as Charleston, South Carolina; Albany, New York, Salt Lake City, Utah; Birmingham, Alabama; Chattanooga, Tennessee; and Houston, Texas. Some of these centers have a Beverly Hills next door (for example, Sugar Land, Texas for Houston), but others such as Salt Lake City have no other cities close by that contain a significant number of plastic surgeons.

There are a few notable outliers among the Centers that deviate from the rest to the point that they are almost Beverly Hills. Below is a list of cities that deviate significantly from other cities of comparable population that they can be considered Beverly Hills-Centers. Some of these cities have a Beverly Hills right next door, such as Houston with Sugar Land.
StateCitySurgeonsPopulationSurgeons/100,000
South CarolinaCharleston17130,11313.1
MichiganGrand Rapids25193,79212.9
CaliforniaPasadena17140,88112.1
GeorgiaAtlanta51456,00211.2
GeorgiaSavannah16144,35211.1
FloridaMiami47430,33210.9
PennsylvaniaPittsburgh33305,41210.8
FloridaFort Lauderdale19176,01310.8
AlabamaBirmingham22212,24710.4
VirginiaRichmond21217,8539.6
MissouriSt. Louis29317,4199.1
FloridaTampa31358,6998.6
OhioCincinnati24298,1658.0
OhioCleveland28389,5217.2
CaliforniaSan Francisco50852,4695.9
MassachusettsBoston36655,8845.5
TexasDallas671,281,0475.2
TexasHouston912,239,5584.1

Properly Placing Salt Lake City In Context


When you properly place Salt Lake City in context it drops from 1st place according to Forbes to #499 out of 1407 cities, towns, villages, military bases and CDPs that have plastic surgeons. With its rate 7.9 surgeons/100,000 it may seem high compared to other cities of similar size. But it is a Center so it has more plastic surgeons than most cities of similar size. But even as a Center it does not rise to the level of being a Beverly Hills-Center since for Centers of a similar size the average is 7.7 surgeons/100,000.

If we consider Salt Lake County the rate drops to 2.4 surgeons/100,000 and for the entire Wasatch Front there are 1.7 surgeons/100,000. This compares with an average of 1.6 for the entire nation and about 1.9 surgeons/100,000 if we just consider urban areas. For the entire state of Utah there are 1.6 surgeons/100,000 perfectly inline with the national average.

Below I plot the results for all cities, The Beverly Hills, The Centers, the original top 10 from Forbes with corrected results, and Salt Lake City.

So rather than being a surprising result according to Forbes, Salt Lake City is rather unsurprising. Forbes's strange result that found that Salt Lake City had more plastic surgeons/100,000 than any other city is due to using a population limited sample, combined with some unclear counting that cannot be cleared up without input from Rebecca Ruiz who wrote the original article for Forbes.

If you would like to take a look at the raw data I used, here is a link to a Google Spreadsheet of the data.

The Geeky Extra Stuff

Congratulations on making it this far into my post. You are one in ten, as in probably only one person out of my total ten readers will make it this far. Because I am a scientist and not a journalist I feel compelled to explain the complications associated with my methodology.

The data from the American Society of Plastic Surgeons is not the cleanest data, nor is the search function entirely reliable. As noted above, if you search for a particular city it will return results from places other than that city, sometimes from a significant distance away. For example a search for Salt Lake City will return results from Sandy and West Jordan, but also as far away as Twin Falls, Idaho. Also I found that a search for a particular city did not always return all of the plastic surgeons associated with that city. But if you search by state and then sort them by city it showed all listed surgeons. This did not significantly affect the numbers and was rare but noticeable.

Some surgeons were listed more than once because they had more than one office. For one surgeon I noticed that she had five offices spread over a state. I counted each office as a surgeon. These cases were rare but noticeable.

Some surgeons had nonsensical addresses. For example, I found a "New York, South Carolina" with a New York City zip code. There were a handful of cases like this where the city and zip code did not correspond to state listed. By my count this affected nine out of 5129 surgeons. Some surgeons were listed twice with slight variations of the same address, so I only counted one surgeon.

There were others that had variations in their address that made it hard to determine what town, village or hamlet they were associated with. This was prevalent in Maryland, Pennsylvania, New York, Massachusetts and Connecticut.

Some listed the city they lived in incorrectly. This is different from listing the wrong state as noted above. For example, there were several surgeons who listed their city as "Crestview, Kentucky", but their zip codes and address indicated they were in Crestview Hills, Kentucky. Both Crestview and Crestview Hills are directly across the Ohio river from Cincinnati and are very small communities just a few miles from each other, but are distinct incorporated places. There were some other variations on names that were particular to the location. Other than Crestview this did not affect rankings.

Some listed their address to indicate a particular borough or neighborhood. For example, there were several listings for Queens, Brooklyn, the Bronx, Manhattan and Staten Island, in addition to addresses for just New York City. There was no overlap as far as I could tell (i.e. surgeons double listed), so when counting the total number of surgeons for New York I included all those that listed their city as one of the boroughs. I did not try to determine how many that listed their city as New York were in each borough, so the number of surgeons and surgeons/100,000 for the boroughs are lower bounds only. The same was done for other neighborhoods of major cities, such as La Jolla in San Diego, but only as far as was aware of it (sorry, I am not intimately knowledgeable about all neighborhoods in all major cities to the point that I know them by name, who do you think I am? Ken Jennings?).

Not all plastic surgeons had up-to-date addresses with American Society of Plastic Surgeons. A spot check indicated that this may affect ~2% of the surgeons listed, but that would not change the rankings nor classification of cities as Centers or Beverly Hills. I also only used data from the American Society of Plastic Surgeons (ASPS) but I did gather statewide data from the American Society for Aesthetic Plastic Surgery (ASAPS) which is another major society of plastic surgeons. Some surgeons are members of both but with a significant number being members of only one. In all states there were fewer members of ASAPS than ASPS, with the notable exception of Texas. Also interesting to note is Florida which had, by a large margin, a higher rate of ASAPS members/100,000.

While I tried to use either 2014 or 2013 data, for a few places that data was not readily available. For all cities with a population over 100,000 and for all cities that fall into the category of the Beverly Hills 2014 data was used. I have assumed that the number of plastic surgeons did not significantly change from 2014 to 2015.

State Stuff

I calculated the total number of surgeons in each state and calculated the number of surgeons/100,000. For both states and cities with more than about 300,000 there is a power law relationship where,
Surgeons = α Populationβ
where Î² = 1.08 and Î± = 4.4e-06. [Edit: When I put this equation in I accidentally grabbed the wrong numbers from my Matlab output. The values for Î± and Î² are now correct.]
Surgeons/100,000 for all cities and states.
Total surgeons for all cities and states.
Answers to Random Questions

Q: Why do you care whether an article in Forbes from 2007 is accurate?

A: I don't. But lately some people (i.e. the Time article mentioned at the beginning) have been using the data to support dubious claims about Utah and by extension Mormons in general. I couldn't pass up the opportunity to undercut the support for that claim when the original data could be shown to be questionable.

Q: Don't you have better things to do?

A: As someone I know once said, "Some people eat, sleep and chew gum, I do genealogy and write...". I don't own a TV, nor do I have a Hulu Plus/Amazon Prime/Netflix account, nor do I have a gaming station. So for entertainment I look up random things on the internet and research stuff like this. It's what I do.

Q: What about all the other data in the Time article?

A: It may or may not be correct (the demographics definitely are correct), but some of the conclusions are not correct, especially those relating to plastic surgery and the statement "A culture of plastic surgery has taken root among Mormon women." My data seriously call into question that statement.

Q: Should Forbes retract their article?

A: I don't care. The original author has long since moved elsewhere, but if Forbes wants to issue a correction, by all means go ahead. If anyone in the Salt Lake City government cares enough about it they can ask Forbes to retract it. But I don't live or work in Salt Lake City.

Q: Are you really an astrophysicist?

A: Yes.

Q: Can I get your original data?

A: Yes. You can find it here in this Google spreadsheet. If you use it for anything make sure you cite your source (me, this blog). I always take points off from my student's lab reports if they don't cite their sources and I will do the same to you.

Q: Did you just threaten to take points off if I don't cite you?

A: Yes.

Q: Did I really just read all of this?

A: Yes. Yes you did.

13 comments:

Sam said...

Sam Bishop. Now you know who your reader is. :)

Ardis said...

I hope that having a second reader who made it to "Congratulations" and then to the end does not cast doubt on your skills, because this was awesome. Thanks.

Jason Glenn said...

Your loving Grandma made it to the end. You have been blessed with great intellect. You and Marcelaine make a great team.

Jacob said...

Great post! One comment from a stranger (friend of a friend?): I think the alpha and beta you recorded for the power law is in error... you have your coefficient as a negative number, your power is something times 10^-5, and it doesn't really look like that. Looks to me more like the power is somewhere close to 1/2 (and definitely with a positive coefficient). I'm guessing you just did a fit of the log of the data in some way, but any chance you could translate and fix that?

Quantumleap42 said...

Thanks Sam and Ardis for your comments.

Quantumleap42 said...

Jason,

Thanks for letting me know. I'm glad someone reads what I write.

Quantumleap42 said...

Jacob,

I noticed that last night, but I didn't have time to open my Matlab code back up and figure out what I did wrong. I thought about inserting a note that I would have to check that, but you noticed it before I got to it. I have corrected the numbers. I was working in lin-lin and log-log space and I just used Matlab's forward slash operator when I knew I just should have done it properly with the polyval function, but I got lazy and then I grabbed the wrong numbers and didn't convert from log to linear space, so I did a double no no.

Also interesting is the US Virgin Islands, the one lone point off by itself with a population of ~100,000, falls pretty close to the line. So one plastic surgeon is what you would expect for the Virgin Islands. So even they fall the national trend.

Eliza said...

I made it to the end. Good work. I think I'll send this on to Kirk if he needs some examples for his research classes.

James Tanner said...

I am going to use your blog post as an example for genealogists who think they can rely on "sources" without evaluating the conclusions.
Like you said, Some people eat, sleep and chew gum, I do genealogy and write.

Steven Montgomery said...

Okay so call me vain, or vanity of vanities, I made it all the through (even the comments). I skipped over the statistical stuff (so I'm trusting you on that one), but overall a good job. Kudo's.

Ranbato said...

The other partial explanation I have heard before is the U hosts the Intermountain burn center which has a number of plastic surgeons associated with it.

Quantumleap42 said...

Ranbato, that is something I was aware of but forgot to mention. I did mention that I looked at the number of surgeons who were members of the American Society for Aesthetic Plastic Surgery (ASAPS), which focuses on cosmetic procedures and not reconstructive. I did not attempt to break it down by how surgeons were members of ASAPS but not ASPS or were members of both, but based on a spot check, anywhere between 30-70% of the surgeons in a city are members of both. This would give an idea of how many surgeons are doing only reconstructive work and not cosmetic work.

A particularly interesting case is Nashville, Tennessee. It has 31 plastic surgeons but I estimate that half of them are based in Vanderbilt University Hospitals. This is something that is typical of the Centers. Usually there is a major hospital or university with a medical center in the city and a number of plastic surgeons are actually associated with the hospital/medical center. These surgeons would not be focused on the "vain" purely cosmetic surgeries, but on reconstructive procedures. That is one reason why I put the Centers in their own class even though they had a higher number of surgeons per capita.

Andrew said...

This is freakonomics and Chris masterjohn PhD amazing. Thanks for this.