fbpx

Putting Licensing to the Test

How Licenses for Tour Guides Fail Consumers—and Guides

Angela C. Erickson

October 2016

Executive Summary

In most of America, all a person needs to start working as a tour guide is something interesting to say. However, in several U.S. cities, aspiring tour guides must pass a test and get a government license before being allowed to work. And such licenses are just one piece of a much larger trend—today, more Americans than ever need a license to work. But what do these licenses actually accomplish?

To find out, this report puts occupational licensing to the test, using the District of Columbia’s now-defunct tour guide licensing scheme as a case study. It finds that the scheme had no effect on the quality of tours in the nation’s capital: An examination of 15,000 TripAdvisor reviews reveals that consumers rated guided tours just as highly after D.C. stopped licensing guides in 2014 as they did before—despite the entry of many new and untested guides into the market. While the scheme was in force, consumers awarded D.C. tour companies an average of 4.27 out of 5 stars; after it was eliminated, the average rating was 4.3 stars.

Thus, instead of ensuring quality tours, D.C.’s licensing scheme just made it harder for some would-be guides to break into the business and kept others out altogether.

These findings point to a better way to encourage quality: consumers. Consumers—not licensing officials—kept D.C.’s tour guides on their toes while the license was in force, and they still do today. Through websites like TripAdvisor and Yelp, consumers weed out providers who fail to deliver quality.

This report adds to a growing body of research that finds many licenses do not improve services for consumers; they just shut people out. By eliminating licenses that exclude people for no public benefit, policymakers can expand economic opportunity and consumer choice—without compromising quality.

dc-flag-img_4663

Introduction

In 2004, Tonia Edwards and Bill Main hit upon an innovative business idea: a company that would offer guided tours of different cities by Segway, the two-wheeled, self-balancing personal transportation vehicle. Tonia and Bill soon started Segs in the City, a company operating Segway tours in Annapolis, Baltimore and Washington, D.C. Their idea proved popular with tourists, but operations in the District ran into a snag. To legally share stories and descriptions of the nation’s capital, Tonia, Bill and any guides they employed needed to obtain tour guide licenses from the District’s Department of Consumer and Regulatory Affairs.

Securing each license required paying three separate fees totaling $200, submitting a time-consuming application and passing a 100-question multiple-choice examination about the District. Anyone caught describing, explaining or lecturing “concerning any place or point of interest in the District to any person” during a sightseeing trip without official permission faced fines of up to $300 or imprisonment for up to 90 days.[c]http://ij.org/case/dc-tours/#backgrounder[/c]

dc-tours130130

In most American cities, tour guides are free to share stories and describe sights without passing a test. For example, all Tonia and Bill needed to start their successful Maryland-based tour businesses was their idea and some Segways. Tour guides who wish to lead walking tours face almost no startup costs at all. However, a handful of cities have licensing schemes similar to what the District had. Charleston (South Carolina),[c]In January 2016, three Charleston tour guides joined with the Institute for Justice to challenge the city’s tour guide exam on First Amendment grounds. As of press time, the case is pending before the U.S. District Court for the District of South Carolina. http://ij.org/case/charleston-tour-guides/[/c] New Orleans,[c]In 2011, four New Orleans tour guides teamed up with the Institute for Justice to sue the city of New Orleans over its tour guide exam. The 5th U.S. Circuit Court of Appeals ultimately ruled against the tour guides, and the U.S. Supreme Court declined to hear their appeal. http://ij.org/case/nola-tours/[/c] New York, St. Augustine (Florida), and Williamsburg (Virginia) all force tour guides to pass a test before they can work.[c]Charleston, S.C., Code of Ordinances § 29-58; New Orleans, La., Code of Ordinances ch. 30, art. XXI, § 30-1551; N.Y.C. Admin. Code § 20-243; St. Augustine, Fla., Code of Ordinances § 17-125; Williamsburg, Va., Code of Ordinances § 9-332. In response to Institute for Justice lawsuits on behalf of tour guides, Savannah, Georgia, repealed its similar scheme in 2015, while in 2008 Philadelphia stopped enforcing its tour guide licensing law, though it remains on the books. http://ij.org/case/savannah-tour-guides-free-speech/; http://ij.org/case/tait-v-city-of-philadelphia/[/c]

Tour guide licensing is only one example of a broader—and growing—phenomenon of city and state governments demanding permission slips before people can work. More than 800 occupations are licensed somewhere in the United States,[c]Kleiner, M. M. (2011). Occupational licensing: Protecting the public interest or protectionism? (Policy Paper No. 2011-009). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.[/c] and one in four American workers now needs a license to work.[c]Department of the Treasury Office of Economic Policy, Council of Economic Advisors, & Department of Labor. (2015). Occupational licensing: A framework for policymakers. Washington, DC: White House.[/c] In the 1950s that figure stood at about one in 20.[c]Kleiner, M. M., & Krueger, A. B. (2013). Analyzing the extent and influence of occupational licensing on the labor market. Journal of Labor Economics, 31(2, pt. 2), 173–202.[/c] This growth is capturing widespread attention and concern.[c]See, e.g., Casselman, B. (2016a, February 5). The rent-seeking is too damn high. FiveThirtyEight. Retrieved from http://fivethirtyeight.com/features/the-rent-seeking-is-too-damn-high/; Casselman, B. (2016b, April 22). Licensing laws are shutting young people out of the job market. FiveThirtyEight. Retrieved from http://fivethirtyeight.com/features/licensing-laws-are-shutting-young-people-out-of-the-job-market/; Cohen, P. (2016, June 17). Moving to Arizona soon? You might need a license. The New York Times. Retrieved from http://www.nytimes.com/2016/06/18/business/economy/job-licenses.html; Editorial Board. (2015, August 6). A license to be a florist? How occupational rules can be a burden on workers. The Washington Post. Retrieved from https://www.washingtonpost.com/opinions/a-license-to-be-a-florist-how-occupational-rules-can-be-a-burden-on-workers/2015/08/06/212ad5b6-3abb-11e5-9c2d-ed991d848c48_story.html; Gowins, H. (2015, July 31). Why the White House is opening up about occupational licensing. The Huffington Post. Retrieved from http://www.huffingtonpost.com/hilary-gowins/why-the-white-house-is-op_b_7913774.html; Kleiner, M. (2015). Reforming occupational licensing policies (Discussion Paper 2015-01). Washington, DC: The Hamilton Project, The Brookings Institution. Retrieved from https://www.brookings.edu/wp-content/uploads/2016/06/THP_KleinerDiscPaper_final.pdf; Koenig, A. (2016, February 2). It takes 890 days to become a barber in Nevada. Politico. Retrieved from http://www.politico.com/agenda/story/occupational-licensing-laws-states-000035; Licenses to Kill. (2015, August 31). The Wall Street Journal. Retrieved from http://www.wsj.com/articles/licenses-to-kill-1441063678; Neyfakh, L. (2015, January 11). How requiring too much training hurts workers and consumers alike. The Boston Globe. Retrieved from https://www.bostonglobe.com/ideas/2015/01/11/how-requiring-too-much-training-hurts-workers-and-consumers-alike/oAXFzNY37P9V9sy9W3WuJM/story.html; Porter, E. (2015, January 27). Job licenses in spotlight as Uber rises. The New York Times. Retrieved from http://www.nytimes.com/2015/01/28/business/economy/ubers-success-casts-doubt-on-many-job-licenses.html[/c] In 2015, the White House released a report examining the trend and calling for reform to rein it in.[c]Department of the Treasury Office of Economic Policy et al., 2015.[/c]

Concerns over rampant licensing stem in large part from the daunting hurdles it forces aspiring workers to clear: A 2012 Institute for Justice study of 102 low- and moderate-income occupations found that, on average, licensing regimes force prospective licensees to spend nine months in education or training, pay more than $200 in fees and pass one exam.[c]Carpenter, D. M., Knepper, L., Erickson, A. C., & Ross, J. K. (2015). Regulating work: Measuring the scope and burden of occupational licensure among low- and moderate-income occupations in the United States. Economic Affairs, 35(1), 3–20.[/c] These hurdles create significant barriers to employment, entrepreneurship and job creation. During Segs in the City’s busy season, for example, about half its tours are led by part-time employees, mostly college students on summer break.[c]http://ij.org/case/dc-tours/#backgrounder[/c] Studying for the licensing test and paying the fee required a heavy investment of time, energy and money for a job lasting only a few months.

[caption id="attachment_46621" align="aligncenter" width="648"]washington-monument-img_4961 Tour guide licensing is only one example of a broader—and growing—phenomenon of city and state governments demanding permission slips before people can work.[/caption]

Proponents justify licensure, and the burdens it imposes, by claiming that it weeds out substandard service providers, thereby ensuring quality. Quality assurance was, in fact, the District’s rationale for the tour guide test, the centerpiece of the licensing scheme. But can a test really ensure a high-quality tour experience? This report took advantage of a unique opportunity to find out—an opportunity that came when Tonia and Bill joined with the Institute for Justice to challenge D.C.’s tour guide test in federal court, and won.

In 2014, after nearly four years of litigation, the U.S. Court of Appeals for the D.C. Circuit struck down D.C.’s licensing scheme as a violation of the First Amendment.[c]The ruling left D.C. with what is, in effect, a registration requirement for tour guides. It involves filing a simple form and paying a fee. D.C. Mun. Regs. tit. 19, § 1203.[/c] That ruling also set the stage for this report, creating ideal conditions for an analysis comparing the quality of D.C. tour guides with and without the test. Looking at tour-goers’ satisfaction, as self-reported on TripAdvisor, this study detects no difference in tour guide quality following the end of the test and the entry of untested guides into the market. Not only that, it finds that quality was high both before and after the test ended. In other words, the test appears not to have made a difference to quality one way or the other.

These results add to a growing body of evidence suggesting that claims about licensing’s ability to ensure quality are, at best, exaggerated. Moreover, they point to alternatives to burdensome occupational licenses that empower consumers, not government bureaucrats, to decide which service providers deserve their business. These alternatives, consumer review websites and ordinary business incentives, were already at work in the D.C. tour market before the exam’s demise, and they remain at work now. They kept tour quality high when the test was in place and are still doing so without it, even as new guides are free to enter the market. These results give another reason to celebrate Tonia and Bill’s victory over the District of Columbia’s tour guide test—and have implications that reach far beyond the tour guide industry.

dc-tours146146

Occupational Licensing: A Growing Barrier to Work

Occupational licensing is a pervasive and growing reality for American workers. And much of this growth reflects an increase in the number of licensed occupations, not other factors such as the changing composition of the workforce. The White House’s 2015 report on licensing attributed almost two-thirds of the increase in the share of state-licensed workers to newly adopted licensing regimes.[c]Department of the Treasury Office of Economic Policy et al., 2015.[/c] Put simply, the primary reason that more workers need a license today is that states are licensing more jobs.

The expansion of occupational licensing means more burdens for aspiring workers, who may be required to earn a certain amount or type of education, complete specialized training or an apprenticeship, pass one or more exams, attain a certain age or grade level, pay fees and more.[c]Carpenter et al., 2015.[/c] This is often the case for occupations that are otherwise well suited to people on the first rungs of the economic ladder[c]Carpenter et al., 2015; Dorsey, S. (1983). Occupational licensing and minorities. Law and Human Behavior, 7(2–3), 171–181; Hazlett, T. W., & Fearing, J. L. (1998). Occupational licensing and the transition from welfare to work. Journal of Labor Research, 19(2), 277–294.[/c]—occupations like florist, packager, travel agent, shampooer, auctioneer and tour guide. Tests, like the one D.C. once required of tour guides, are a common barrier to occupations like these: The Institute for Justice’s 2012 study found that 79 out of the 102 low- and moderate-income jobs required at least one exam.[c]Carpenter et al., 2015.[/c]

Scholarly research has shown that licensing burdens add up to fewer opportunities for workers,[c]Adams, A. F., Jackson, J. D., & Ekelund, R. B. (2003). Occupational licensing of a credence good: The regulation of midwifery. Southern Economic Journal, 69(3), 659–675; Harrington, D. E., & Treber, J. (2009). Designed to exclude: How interior design insiders use government power to exclude minorities and burden consumers. Arlington, VA: Institute for Justice; Federman, M. N., Harrington, D. E., & Krynski, K. J. (2006). The impact of state licensing regulations on low-skilled immigrants: The case of Vietnamese manicurists. American Economic Review, 96(2), 237–241; Kleiner, M. M. (2006). A license for protection. Regulation, 29(3), 17–21; Kleiner, M. M., & Kudrle, R. T. (2000). Does regulation affect economic outcomes? The case of dentistry. Journal of Law and Economics, 43(2), 547–582.[/c] and there is evidence that some groups are impacted more than others. People working in the low- and moderate-income occupations IJ studied in 2012 were more likely to be racial or ethnic minorities than members of the general population. They also tended to have less education. Compared with only 9.5 percent of the general population, 15.7 percent of workers in lower-income occupations had less than a high school education.[c]Carpenter, D., Knepper, L., Erickson, A. C., & Ross, J. K. (2012). License to work: A national study of burdens from occupational licensing. Arlington, VA: Institute for Justice. Minorities and people with less education are not the only ones who disproportionately find their opportunities stifled. Occupational licensing also reduces opportunities for people looking to change careers or move across state lines. The White House report noted that licensing burdens fall particularly heavily on military spouses, who move more frequently than their civilian counterparts because of their enlisted spouses’ assignments. Other groups hit hard by licensure include immigrants and people with criminal records. Immigrants may have valuable education, experience and skills—such as African hair braiding or eyebrow threading—that they are unable to leverage for lack of a license, leaving many underemployed. Nearly one in three adults in the United States has a criminal record, yet in half of U.S. states, ex-offenders may be disqualified from receiving a license, no matter how old or irrelevant their conviction—a problem that may help drive recidivism as people find themselves unable to find work and fall back into crime. Licensing also means copious red tape and expense for entrepreneurs and small-business owners, who may find it harder to offer services or recruit licensed employees—and therefore to get their businesses off the ground and keep them up and running. Rodriguez, M. N., & Avery, B. (2016). Unlicensed and untapped: Removing barriers to state occupation licenses for people with records. National Employment Law Project. Retrieved from http://www.nelp.org/publication/unlicensed-untapped-removing-barriers-state-occupational-licenses; Department of the Treasury Office of Economic Policy et al., 2015.[/c] Therefore it is likely that barriers to entering these occupations disproportionately affect minorities and those with less education.

[caption id="attachment_46623" align="aligncenter" width="300"]ifj_8214 A Savannah, Georgia tour guide displays his permit.[/caption]

In addition to burdening aspiring workers, licensing imposes costs on consumers because it restricts the supply of practitioners.[c]Adams et al, 2002; Cox, C., & Foster, S. (1990). The costs and benefits of occupational regulation. Washington, DC: Bureau of Economics, Federal Trade Commission; Kleiner, M. M. (2000). Occupational licensing. Journal of Economic Perspectives, 14(4), 189–202; Kleiner and Kudrle, 2000; Shepard, L. (1978). Licensing restrictions and the cost of dental care. Journal of Law and Economics, 21(1), 187–201.[/c] This means fewer choices and higher prices[c]Adams et al, 2002; Anderson, G. M., Halcoussis, D., Johnston, L., & Lowenberg, A. D. (2000). Regulatory barriers to entry in the healthcare industry: The case of alternative medicine. The Quarterly Review of Economics and Finance, 40(4), 485–502; Chevalier, J. A., & Morton, F. M. S. (2008). State casket sales restrictions: A pointless undertaking? Journal of Law and Economics, 51(1), 1–23; Haas-Wilson, D. (1986). The effect of commercial practice restrictions: The case of optometry. Journal of Law and Economics, 29, 165–186; Harrington, D. E., & Krynski, K. J. (2002). The effect of state funeral regulations on cremation rates: Testing for demand inducement in funeral markets. Journal of Law and Economics, 45(1), 199–225; Kleiner and Kudrle, 2000.[/c] as service providers are able to command higher wages due to their relative scarcity.[c]Chevalier and Morton, 2008; Friedman, M., & Kuznets, S. (1945). Income from independent professional practice. New York, NY: National Bureau of Economic Research; Kleiner, 2000, 2006; Pfeffer, J. (1974). Some evidence on occupation licensing and occupational incomes. Social Forces, 53(1), 102–111.[/c] This observation points to a possible explanation for the growth in licensing: Practitioners themselves pursue licensure of their occupations seeking to capture the economic benefits licensing confers on the licensed. And, indeed, academic research finds that occupational licensure in an industry is typically sought by insiders to that industry—existing practitioners of the occupation and their professional associations and companies.[c]Carpenter, D. M. (2008). Regulation through titling laws: A case study of occupational regulation. Regulation and Governance, 2(3), 340–359; Erickson, A. C. (2013). White out: How dental industry insiders thwart competition from teeth-whitening entrepreneurs. Arlington, VA: Institute for Justice; Kleiner, 2006; Paul, C. (1984). Physician licensure legislation and the quality of medical care. Atlantic Economic Journal, 12(4), 18–30; Pawlicz, A. (2013). Economic rationale for licensing tour guides in Poland. In Tourism in Southern and Eastern Europe (Vol. 2, pp. 295–304). Rijeka, Croatia: University of Rijeka; Stigler, G. J. (1991). The theory of economic regulation. Bell Journal of Economics and Management Science, 2(1), 3–21; Wheelan, C. J. (2005). Politics or public interest? Licensing and the case of respiratory therapists. Perspectives on Work, 8(2), 42–43; Young, S. D. (1991). Interest group politics and the licensing of public accountants. Accounting Review, 66(4), 809–817.[/c]

Calls for licensure of an occupation are often accompanied by appeals for the need to ensure quality.[c]Carpenter, 2008; Carpenter, D. M. (2011). Blooming nonsense: Do claims about the consumer benefit of licensure withstand empirical scrutiny? Regulation, 34(1), 44–47; Cox and Foster, 1990; Kleiner and Krueger, 2013.[/c] However, there is little research to bolster such claims.[c]Carroll, S. L., & Gaston, R. J. (1981). Occupational restrictions and the quality of service received: Some evidence. Southern Economic Journal, 47(4), 959–976; Kleiner, M. M., & Petree, D. L. (1988). Unionism and licensing of public school teachers: Impact on wages and educational output. In R. B. Freeman & C. Ichniowski (Eds.), When public sector workers unionize (pp. 305–322). Chicago, IL: University of Chicago Press; Shilling, J. D., & Sirmans, C. F. (1988). The effects of occupational licensing on complaints against real estate agents. The Journal of Real Estate Research, 3(2), 1–9.[/c] The scholarly evidence, in fact, suggests that claims about the benefits of licensing to consumers in terms of higher quality are overstated.[c]Buddin, R. J., & Zamarro, G. (2008). Teacher qualifications and student achievement in urban elementary schools. Journal of Urban Economics, 66(2), 103–115; Carpenter, 2008, 2011; Haas-Wilson, 1986; Huang, S., & Weiler, B. (2010). A review and evaluation of China’s quality assurance system for tour guiding. Journal of Sustainable Tourism, 18(7), 845–860; Kleiner and Kudrle, 2000; Kleiner, M. M., & Todd, R. M. (2007). Mortgage broker regulations that matter: Analyzing earnings, employment, and outcomes for consumers (NBER Working Paper No. 13684). Cambridge, MA: National Bureau of Economic Research; Paul, 1984.[/c] The same goes for licensing tests: Academic research on occupational licensing in industries as diverse as floristry, construction contracting and education has found that testing is used by members of the occupation to restrict entry and raise wages with no positive effect on quality.[c]Angrist, J. D., & Guryan, J. (2008). Does teacher testing raise teacher quality? Evidence from state certification requirements. Economics of Education Review, 27, 483–503; Buddin and Zamarro, 2008; Carpenter, 2011; Kleiner, 2006; Maurizi, A. (1974). Occupational licensing and the public interest. Journal of Political Economy, 82(2, pt. 1), 399–413; Maurizi, A. R. (1980). The impact of regulation on quality: The case of California contractors. In S. Rottenberg (Ed.), Occupational licensure and regulation (pp. 26–35). Washington, DC: American Enterprise Institute.[/c] To take one example, a 2010 experiment put Louisiana’s florist licensing exam to the test, asking florists from licensed Louisiana and unlicensed Texas to judge floral arrangements from both states, without knowing which arrangements were from which state. The experiment essentially found no difference between the arrangements created by licensed and unlicensed florists.[c]Carpenter, 2011.[/c] Indeed, far from raising quality as proponents claim, some studies have shown that licensing may dampen innovation[c]Timmermans, S. (2008). Professions and their work: Do market shelters protect professional interests? Work and Occupations, 35(2), 164–188.[/c] and limit or even lower quality.[c]Kleiner and Todd, 2007.[/c]

There is a large body of research on tour guides outside the U.S., some of which looks at tour guide quality. However, none explores quality from the perspective of what consumers actually value. Instead, questions of quality are mediated by experts and insiders, whose priorities may or may not align with those of consumers. And apart from this report, I know of no research at all that looked at tour guide quality in the U.S. context. This report therefore makes a novel contribution to knowledge about tour guide licensing and quality in the United States. It also adds to the evidence casting doubt on claims that licensing, in general, ensures quality.

Testing the Test: Before and After D.C.’s Tour Guide Licensing Exam

Methods

To examine whether D.C.’s tour guide examination improved quality, this report relies on TripAdvisor consumer ratings of D.C. tour companies from before and after July 2014, when the D.C. Department of Consumer and Regulatory Affairs ended the test. TripAdvisor was chosen because it is the most popular consumer review travel website, with 375 million unique monthly visitors and over 250 million reviews of more than 5.2 million businesses as of July 2015.[c]TripAdvisor. (n.d.). About TripAdvisor. Retrieved July 2015 from http://www.tripadvisor.com/PressCenter-c6-About_Us.html[/c] There is no comparable resource compiling reviews of individual tour guides. However, many guides are employed by tour companies, making reviews of such companies a good proxy for determining tour guide quality. If D.C.’s test successfully weeded out lower-quality guides, one would expect to see a fall in the average tour company rating following the test’s removal.

I collected data from the TripAdvisor website for all D.C. attractions labeled as “Tours & Activities” and for all reviews of these companies made between August 1, 2010, and July 31, 2015. Historically, D.C.’s tourist season has two peaks, the busiest season from mid-March through early June and then the lighter season from September through October,[c]Pond, K. L. (1993). The professional guide: Dynamics of tour guiding. New York, NY: John Wiley & Sons.[/c] so I collected data for a full year following the change to capture both peak seasons. In all, I examined 14,762 consumer reviews across five years. The review information included the business name and website, date of the review, reviewer’s rating, number of reviews contributed by the reviewer at the time of collection, number of helpful votes received by the reviewer at the time of collection, and text of the review.

I calculated the average consumer rating per business before and after the test—disaggregated by type of tour (bike, boat, bus, mixed, Segway and walking). These raw numbers are useful but not sufficient for comparing consumer satisfaction during the time guides were required to take the test and after the test requirement was eliminated. For example, the moment the testing requirement was removed, all of the tour guides still fell under the old regime and had taken the exam; however, over time, many new guides entered the market who had not taken the exam. Additionally, some reviewers may be more reputable, and research suggests such reviewers have a larger impact on potential consumers’ choices.[c]Lee, H. A., Law, R., & Murphy, J. (2011). Helpful reviewers in TripAdvisor, an online travel community. Journal of Travel and Tourism Marketing, 28(7), 675–688; Zhu, L., Yin, G., & He, W. (2014). Is this opinion leader’s review useful? Peripheral cues for online review helpfulness. Journal of Electronic Commerce Research, 15(4), 267–280.[/c]

To control for these factors, which can cloud the comparison, and to determine whether there were differences in consumer satisfaction with and without the exam, this report relies on an interrupted time-series analysis. This analysis specifically controls for the effects of an intervention—in this case, the end of the testing requirement for tour guides—at a specific time. It can also detect differences that may be delayed for a period of time after the intervention—such as new, untested, guides entering the market—while controlling for general changes over time that would have occurred without the intervention. Further details on the sample and analysis are provided in Appendix A.

[caption id="attachment_46624" align="aligncenter" width="624"]ifj_8097 Scholarly research has shown that licensing burdens add up to fewer opportunities for workers.[/caption]

Results

Tour guide businesses in D.C. are generally well regarded by reviewers. Seventy-two percent of the nearly 15,000 reviews collected for the period August 1, 2010, through July 31, 2015, awarded businesses 5 out of 5 stars. Another 13% gave 4 stars. The average overall rating for all the businesses was 4.3 stars.

[caption id="attachment_46603" align="alignright" width="238"]figure1 Figure 1: Average Tour Guide Business Rating During and After the Exam Requirement[/caption]

And these ratings did not change after the exam requirement was removed, although 272 new tour guides—making up 16% of registered guides—had, as of August 1, 2015, entered the D.C. market following the exam’s removal.[c]Figures come from the full list of registered (and previously licensed) tour guides in D.C. and the basis of their license or registration (test or no test), which the Institute for Justice requested from the D.C. Department of Consumer and Regulatory Affairs.[/c] The findings show that, overall, tour businesses provided the same quality of tours after the exam was eliminated as they provided under the licensing regime—an average of 4.27 stars before and 4.3 after (see Figure 1). As Figure 2 shows, average ratings, disaggregated by tour type, shifted only slightly after the exam’s removal. For some tour types the average rating increased (boat, bus, mixed and Segway) and for bike tours it decreased. Notably, the difference in quality based on the District’s exam is small. These differences are not statistically significant for the whole sample or for any tour type, meaning they are not related to the change in licensing regime (see Appendixes).

[caption id="attachment_46608" align="alignnone" width="693"]figure2 Figure 2: Average Tour Guide Business Rating During and After the Exam Requirement by Tour Type[/caption]

 

infographic

 

Discussion & Conclusion

The only major change for D.C. tour guides in July 2014 was that the testing requirement went away. Absent that requirement, new guides entered the market, many of whom had never taken the test. If the test had really worked to screen out lower-quality tour guides, as theorized by proponents, one would expect to see a drop in quality among tour businesses. Yet customers rated tour quality just as highly following the test’s demise. It could be that the District’s test was simply a bad test and that another test might have done a better job of weeding out substandard tour guides. However, this study shows that quality was high while the test was in force and remained high after the test ended.

A more likely explanation for why D.C.’s test did not affect tour quality is simply that licensing tests are a poor way of ensuring tour guide quality. And, indeed, previous research suggests there is a mismatch between one-size-fits-all approaches to quality assurance, such as testing, and the tour guide occupation. In 2014, Weiler and Black, the leading academic authorities on the subject, published a global review of the scholarly research on tour guides and determined that the “blanket approach” of licensing may not be appropriate for all guides.[c]Weiler, B., & Black, R. (2014). Tour guiding research: Insights, issues and implications. Bristol, UK: Channel View Publications, p. 149.[/c] They also noted that licensing “does not necessarily … provide an incentive for excellence” or “support advanced levels of role performance”—i.e., expressive or interactive storytelling—among tour guides.[c]Weiler and Black, 2014, p. 149.[/c]

But why might a one-size-fits-all approach such as testing not be the right one for tour guides? To answer this question requires an understanding of the function of testing and the nature of the tour guide occupation.

All a test can do is restrict entry to the tour guide occupation only to those who know and can recall a certain set of facts and stories under testing conditions. D.C.’s exam, for example, covered 14 categories of information from nine sources, most of them relating to traditional topics such as history and major points of interest.[c]The D.C. test forced guides to memorize facts about architecture; dates; government; historical events; landmark buildings; locations; monuments and memorials; museums and art galleries; parks, gardens, zoos and aquariums; presidents; sculptures and statues; universities; pictures; and regulations. See Edwards v. Dist. of Columbia, 755 F.3d 996, 999–1000 (D.C. Cir. 2014).[/c] But while historical and general interest tours are popular and common, not every guide wants to give such tours—and not every tour-goer wishes to go on them. Yet tests like D.C.’s force the guide who wants to focus on Civil War history, the guide who wants to specialize in tours of TV and film locations and the guide who wants to lead pub crawls to know the same information, much of it useless for what they want to do.

A related issue is that there are thousands of stories that guides might wish to tell about their cities, and not all of them deal with facts in the conventional sense. For example, since the 1970s, there has been an increase in the number of so-called alternative guides whose aim is to disseminate narratives counter or parallel to the “official” or “mainstream” stories of their cities.[c]Bryon, J. (2012). Tour guides as storytellers–From selling to sharing. Scandinavian Journal of Hospitality and Tourism, 12(1), 27–43.[/c] Other guides may wish to focus on legends or ghost stories rather than historical facts. Tour guide tests force all guides, regardless of their unique perspectives and goals, to master information that may be irrelevant or even anathema to them—and exclude those who refuse or otherwise fail to conform.

[caption id="attachment_46625" align="aligncenter" width="638"]dc-tours155155 Overall, tour guides provided the same quality of tours after the exam was eliminated as they provided under the licensing regime.[/caption]

But perhaps the biggest problem with using a test to decide who can work as a tour guide is the reality that there is more to good guiding and enjoyable tours than facts. Today’s tourists do not want to be lectured at[c]Bryon, 2012; Pond, 1993.[/c]—they want to discover a place, to be immersed in the local culture and to have a unique experience,[c]Pond, 1993.[/c] whether that means watching costumed tour guides interact, shivering along with ghost stories, sampling local cuisine, or cruising around on Segways with Tonia and Bill. Consistent with these demands, research has found that tour-goers value guides’ ability to tell stories—to interpret, entertain, engage and inspire—more highly than they do guides’ knowledge about the destination or actual tour content.[c]Bryon, 2012; Pond, 1993; Weiler and Black, 2014.[/c]

This is not to say that factual knowledge is not important, only that there are many other qualities that people also look for in a tour guide, such as storytelling ability and charisma—qualities that a written test does not, and arguably cannot, measure. Tests like D.C.’s therefore risk shutting out gifted tour guides simply because they do not know or care to learn a particular set of facts that may not even be relevant to their tours or business models. Even for those who are willing and able to attempt them, such exams impose costs: Every hour they spend studying potentially useless or contested information is one they are not spending honing their storytelling skills, researching the topics they actually are interested in speaking about, or doing other productive things like earning money giving tours. And despite all that hard work, they may still fail because they focused on the wrong things or are poor test-takers.[c]Guides may also be tripped up by tests with ambiguously phrased questions or incorrect answer keys. The D.C. exam is known to have had several such issues. See OldTownTrolley. (2010). DC tour guide test [Flashcards]. Retrieved from http://www.cram.com/flashcards/dc-tour-guide-test-1318009[/c]

The stories of three would-be guides who in January 2016 joined with the Institute for Justice to sue Charleston, South Carolina, over its tour guide test illustrate the human costs of licensing tests.[c]http://ij.org/case/charleston-tour-guides/[/c] Kim Billups had everything she needed to launch her historical character tour company, right down to a replica antebellum gown—everything, that is, except a license. Mike Warfield, an insurance broker and a popular volunteer at a local museum, earned an offer to lead evening ghost and pub tours of the city. But Mike did not have a license and so could not legally accept the job. Retired book editor Michael Nolan, who moved to Charleston in July 2015 after a lifetime spent helping others tell their stories, planned to supplement his income by telling some stories of his own as a tour guide, but he could not without a license. Despite countless hours spent studying Charleston’s 490-page “training manual” for prospective guides, all three failed the city’s written test—even Mike who had passed the notoriously difficult Series 7 financial securities certification exam on his first try.

 

[caption id="attachment_46628" align="alignleft" width="300"]charleston-tours-mike-warfield-ifj_9947 IJ client Mike Warfield[/caption]

[caption id="attachment_46626" align="alignright" width="248"]charleston-tours-kim-billups-ifj_9834 IJ client Kim Billups[/caption]

[caption id="attachment_46627" align="alignleft" width="300"]charleston-tours-michael-nolan-ifj_9979 IJ client Michael Nolan[/caption]

 

 

 

 

 

 

 

 

 

 

 

In response to the lawsuit, the city retained its 200-question written test but lowered the minimum passing score retroactively. It also eliminated the oral exam that aspiring guides had been required to take subsequent to passing the written test,[c]The city also repealed its temporary tour guide license, receipt of which had been contingent on guides’ being sponsored by an existing company, passing a separate written exam and obtaining the city’s approval for the script they would use during tours. Charleston, S.C., Ordinance 2016-054 (Apr. 26, 2016). Notwithstanding these modifications, the licensing scheme continues to infringe the free speech rights of aspiring guides. As of press time, Kim, Mike and Michael’s federal civil rights lawsuit against Charleston is therefore still active.[/c] allowing Kim and Mike to collect their licenses in May 2016. Showcasing the arbitrary nature of tour guide exams and their passing requirements, Kim and Mike went from being potential threats to the public in the city’s eyes one day to fully sanctioned guides the next. They are free to give tours in Charleston, though only after about a year of preparation and waiting—and a constitutional lawsuit. Michael, for his part, is still waiting, unable to work as a tour guide. And they are only a few of the people who have been kept from offering tour experiences to willing customers by Charleston’s tour guide license. They—and the tour-going public—are poorer for their exclusion.

Licensing exams thus rob energetic and entrepreneurial people of the right to earn an honest living in the occupation of their choosing. They likewise rob customers of the chance to listen to, learn from and be entertained by guides shut out of the market. At the same time, they do little to ensure quality because knowing the facts they test often has little or no bearing on whether guides are good at their jobs, even if it might be useful in some cases. This certainly seems to have been the case in D.C., where the tour guide test appears to have had no impact on tour quality.

Furthermore, since quality was good both while the testing requirement was in force and after it ended and new, untested guides entered the market, something else must have been working to weed out poor guides. The findings here suggest that something, or rather someone, was consumers. Consumers will not recommend a tour they did not enjoy. In fact, they are likely to do the opposite, telling their friends, family and, nowadays, anyone with an internet connection about their negative experience through consumer review websites and social media. Meanwhile, positive word of mouth will drive consumers to tour providers with a reputation for quality. Poor tour guides and companies will be forced to improve or continue to lose market share and, before long, go out of business. With their dollars, their feet and their opinions, consumers provide feedback and make plain their expectations, which they can now share far beyond their personal social circles thanks to online platforms like TripAdvisor. And businesses, most of which are in the business of staying in business, respond.

Consumer reviews thus work in harmony with ordinary business incentives to keep quality up to par. Written for consumers by consumers, they are more attuned than any test or license to what consumers actually care about. Consumer reviews tell readers, often in quite granular detail, what a tour was like, whether it was a good value and other information more relevant to their experience than whether their guide passed a test or had a license. This information is helpful both to other consumers deciding which businesses deserve their patronage and to businesses trying to learn what consumers want and how to better serve them.

In short, any review—good, bad or indifferent—is information that can help businesses learn how to become more competitive. In this way, reviews serve both to ensure quality and to drive creativity and innovation. The following 4-star review of a D.C. tour company submitted to TripAdvisor while the test was still in force is a good example of how this can work:

I thought this tour was a fairly inclusive tour of the Washington monuments, although I would have liked to visit the Vietnam Veteran’s Memorial. However, I understand that there isn’t enough time in 3.5 hours to visit every memorial in the city (especially since some of them are actually outside of the city and in Arlington, VA!).

Having said that, I still felt very rushed at each monument and just didn’t have enough time to really see everything I wanted to see at each one. Our tour guide, Becca, was very knowledgeable and nice, however, I do think she should wait in front of the bus for everyone to disembark, so that those sitting at the back of the bus are not having to run to catch up. There were many times that I missed some of what she was saying because I had been sitting toward the back and it took a while to get off of the bus and catch up with her.

Overall, though, I thought it was a good overview of most of the main monuments. I just needed more time in my visit to go back to the ones I felt I didn’t have enough time at![c]Patty F. (2013, April 11). Great tour of the monuments! [Consumer review]. TripAdvisor. Retrieved from https://www.tripadvisor.com/ShowUserReviews-g28970-d676041-r157453175-DC_by_Foot-Washington_DC_District_of_Columbia.html#[/c]

From this one review, consumers and the tour operator being reviewed could derive a wealth of useful information. Consumers interested in seeing most of the major Washington monuments learn that this tour could be a fine option, while consumers adamant about visiting the Vietnam Veterans Memorial Wall during their stay in D.C. learn that doing so will require a separate outing. The tour operator learns that it is, at least in one consumer’s view, doing a good job overall but should, among other things, consider giving people more time at the various attractions.

The D.C. Circuit Court, in its opinion in Tonia and Bill’s case, came to a similar conclusion about the value of consumer reviews for ensuring tour quality:

One need only peruse such websites [as Yelp and TripAdvisor] to sample the expressed outrage and contempt that would likely befall a less than scrupulous tour guide. Put simply, bad reviews are bad for business. Plainly, then, a tour operator’s self-interest diminishes—in a much more direct way than does the exam requirement—the harms the District merely hypothesizes.[c]Edwards, 755 F.3d at 1006–07.[/c]

And there is good reason to believe that what works to keep tour guide quality high in D.C. is already working in other locations and for other occupations as well. The hundreds of U.S. cities that do not license tour guides are not apparently beset with poor-quality guides. Moreover, while people know that restaurants, salons and autoshops must carry various licenses and permits and obey other laws, they also know that this means only that a service provider jumped through certain government-mandated hoops. Licenses and permits say nothing about the quality of meals, haircuts or car repairs consumers can expect. So when people are deciding which businesses to patronize, they ask their friends or colleagues for recommendations, or they visit websites like Yelp, Angie’s List or Thumbtack to read reviews. This dynamic is also visible in the workings of popular peer-to-peer services such as Uber, Lyft, Airbnb and eBay that rely heavily on ratings in order to function. More now than ever before, consumers are aware of their power to demand quality—and enjoying the improvements that result.

Empowering consumers can be a more effective way than occupational licensing to ensure quality in products and services—especially given the growing body of research, including this study, that casts doubt on the ability of tests and other licensing schemes to promote quality. Evidence suggests that many licenses can be eliminated without compromising quality, particularly for those occupations such as tour guide where minor risks, if any, are easily mitigated by consumers opting to take their business elsewhere. City and state governments interested in expanding opportunity and consumer choice need only look to the nation’s capital, where tour guide freedom rolls on under the Segs in the City banner, for a glimpse of what is possible when people are cut loose from unnecessary red tape.

capital-building-sky-xlarge

Appendix A: Methods

Sample

The final dataset contained 82 different tour guide businesses, including subsets of larger businesses, and 14,762 unique user reviews—5,026 of which were submitted after July 2014, the month the exam requirement was removed. I also looked at each of the companies’ websites to determine the type of tours they provided (bike, boat, bus, mixed, Segway or walking) and the cost of their lowest priced tour.

Table A1 disaggregates the sample of reviews by regime and tour type. The number of businesses differs between the periods with and without the exam as businesses enter and leave the marketplace at different times. The analysis therefore includes some businesses that only existed in one period, making for an unbalanced panel. Average ratings are based on the average consumer rating for each company.

Tour companies were removed if their tours were never led by a guide or they were free, because these types of tours were exempt from licensing in D.C. I kept separate any business that turned out to be part of another business in the sample because a consumer going through the TripAdvisor website would see them as two separate sightseeing tours and might not know that they were the same company. Additionally, sub-businesses may use different modes of transportation or one may offer tours led by a guide, while the other may offer mixed tours, as in some with a guide and some without.

Table A1: Average Ratings by Tour Type

[table] Regime,Tour Type,Bike,Boat,Bus,Mixed,Segway,Walking,Total
Exam,Companies,7,5,13,19,5,16,65
,Reviews,257,191,"3,038","3,787","1,887",576,"9,736"
,Avg. Rating,4.96,3.34,3.79,4.54,4.84,4.14,4.27
No Exam,Companies,8,5,16,20,5,24,78
,Reviews,143,78,"2,065","1,968",461,311,"5,026"
,Avg. Rating,4.61,3.52,4.09,4.60,4.85,4.14,"4.30"
All,Companies,9,5,16,23,5,24,82
,Reviews,400,269,"5,103","5,755","2,348",887,"14,762"
,Avg. Rating,4.66,3.32,4.05,4.63,4.84,4.15,4.31[/table]

Analysis

To test the District’s hypothesis that tour quality would decline without the existence of its test, t-tests were run on the average ratings before and after the regime change for the whole sample as well as for each tour type grouping. These results show that there is no statistical difference between the average consumer ratings of tour guide companies before and after the change.

To further isolate the influence of the exam regime on the average monthly consumer rating (Y) for each company, this analysis measured differences with ordinary least squares (OLS) regression using an interrupted time-series analysis with clustered standard errors. In equation (1), ratings were regressed on the study period measured in months (β1), a dichotomous variable for the presence of the testing regime (β2), a growth score measuring the number of months since the test’s removal (β3), the company’s lowest tour price (β4) and tour type dummy variables (X). Additionally, equation (2) was run with interaction terms between tour type and period, regime and growth score to isolate whether individual tour types were affected by the change.

The dependent variable (Y) is the average rating of each business during each period (months). Each rating was given an importance weight based on the reviewer’s reputation signaling—the reviewer’s number of helpful votes. The weights are based on the quantile the review was put in based on the reviewer’s number of helpful votes. Helpful votes are given to a reviewer when another user finds one of their reviews to be “helpful.” These votes are used as a measure of reputation and may reduce uncertainty for readers of reviews.[c]Lee et al., 2011; Zhu et al., 2014.[/c]

The following are the OLS interrupted time-series models used:

(1) Y = β0+ β1 (Study period) + β2 (Regime) + β3 (Growth score) + β4 (Lowest price) + X

(2) Y = β0+ β1 (Study period) + β2 (Regime) + β3 (Growth score) + β4 (Lowest price) + X

    +β5 (Boat*Period) + β6 (Boat*Regime) + β7 (Boat*Growth score)           

    +β8 (Bus*Period) + β9 (Bus*Regime) + β10 (Bus*Growth score)

    +β11 (Mixed*Period) + β12 (Mixed*Regime) + β13 (Mixed*Growth score)

    +β14 (Segway*Period) + β15 (Segway*Regime) + β16 (Segway*Growth score)

    +β17 (Walking*Period) + β18 (Walking*Regime) + β19 (Walking*Growth score)

The study period is a count variable that starts at one for August 2010 and counts upward to 60 for July 2015. The growth score is a count variable that remains at zero until the regime change at 49 months, when it starts to count upward to 12 for July 2015.

About half of the reviews were for companies that offered both led and unled tours. There is no way to distinguish reviews of tours that had guides from those of tours without guides. The outcome is the same if these companies are removed.[c]The analysis was also run with the sample limited to businesses that had reviews in at least three periods both before and after the regime change, and the results were not substantively different.[/c] A company fixed effects model similar to equation (1) was also run and the results remained the same. The regression output for these analyses can be supplied upon request.

Appendix B: Regression Results

(1)
(2)
Coefficient Robust SE p Coefficient Robust SE p
Period 0.009 0.005 0.06 -0.003 0.001 0.04
Regime -0.042 0.081 0.61 0.013 0.140 0.93
Growth score -0.019 0.011 0.08 -0.005 0.016 0.76
Lowest price 0.004 0.002 0.01 0.003 0.001 0.02
Boat -1.356 0.256 0.00 -1.514 0.387 0.00
Bus -0.751 0.254 0.00 -1.931 0.527 0.00
Mixed -0.238 0.172 0.17 -0.542 0.321 0.09
Segway 0.044 0.149 0.77 -0.008 0.146 0.96
Walking -0.489 0.237 0.04 -0.922 0.406 0.02
Boat*Period 0.005 0.007 0.47
Boat*Regime -0.137 0.270 0.61
Boat*Growth score -0.046 0.038 0.23
Bus*Period 0.030 0.012 0.01
Bus*Regime 0.008 0.263 0.98
Bus*Growth score -0.044 0.033 0.18
Mixed*Period 0.006 0.008 0.46
Mixed*Regime 0.123 0.196 0.53
Mixed*Growth score 0.000 0.019 0.99
Segway*Period 0.000 0.003 0.98
Segway*Regime -0.042 0.163 0.80
Segway*Growth score -0.006 0.017 0.73
Walking*Period 0.014 0.008 0.08
Walking*Regime -0.381 0.220 0.08
Walking*Growth score -0.001 0.029 0.96
Intercept 4.253 0.229 0.00 4.705 0.134 0.00
Sigma_u 0.738 0.780
Sigma_e 0.778 0.767
Rho 0.473 0.508

Endnotes


    Related Reports

    Sign up to receive IJ's biweekly digital magazine, Liberty & Law along with breaking updates about our fight to protect the rights of all Americans.

    JOIN THE FIGHT!