Most Popular
1. Banking Crisis is Stocks Bull Market Buying Opportunity - Nadeem_Walayat
2.The Crypto Signal for the Precious Metals Market - P_Radomski_CFA
3. One Possible Outcome to a New World Order - Raymond_Matison
4.Nvidia Blow Off Top - Flying High like the Phoenix too Close to the Sun - Nadeem_Walayat
5. Apple AAPL Stock Trend and Earnings Analysis - Nadeem_Walayat
6.AI, Stocks, and Gold Stocks – Connected After All - P_Radomski_CFA
7.Stock Market CHEAT SHEET - - Nadeem_Walayat
8.US Debt Ceiling Crisis Smoke and Mirrors Circus - Nadeem_Walayat
9.Silver Price May Explode - Avi_Gilburt
10.More US Banks Could Collapse -- A Lot More- EWI
Last 7 days
US Interest Rates - When WIll the Fed Pivot - 1st Mar 2024
S&P Stock Market Real Earnings Yield - 29th Feb 2024
US Unemployment is a Fake Statistic - 29th Feb 2024
U.S. financial market’s “Weimar phase” impact to your fiat and digital assets - 29th Feb 2024
What a Breakdown in Silver Mining Stocks! What an Opportunity! - 29th Feb 2024
Why AI will Soon become SA - Synthetic Intelligence - The Machine Learning Megatrend - 29th Feb 2024
Keep Calm and Carry on Buying Quantum AI Tech Stocks - 19th Feb 24
How to Profit from the Global Warming ClImate Change Mega Death Trend - Part1 - 17th Feb 24
Why Rising Shipping Costs Won't Cause Inflation - 17th Feb 24
Intensive 6 Week Stock Market Elliott Wave Training Course - 17th Feb 24
INFLATION and the Stock Market Trend - 17th Feb 24
GameStop (GME): 88% Shellacking Yet No Lesson Learned - 17th Feb 24
Nick Millican Explains Real Estate Investment in a Changing World - 17th Feb 24
US Stock Market Addicted to Deficit Spending - 7th Feb 24
Stocks Bull Market Commands It All For Now - 7th Feb 24
Financial Markets Narrative Nonsense - 7th Feb 24
Gold Price Long-Term Outlook Could Not Look Better - 7th Feb 24
Stock Market QE4EVER - 7th Feb 24
Learn How to Accumulate and Distribute (Trim) Stock Positions to Maximise Profits - Investing 101 - 5th Feb 24
US Exponential Budget Deficit - 5th Feb 24
Gold Tipping Points That Investors Shouldn’t Miss - 5th Feb 24
Banking Crisis Quietly Brewing - 5th Feb 24
Stock Market Major Market lows by Calendar Month - 4th Feb 24
Gold Price’s Rally is Normal, but Is It Really Bullish? - 4th Feb 24
More Problems in US Regional Banking System: Where There's Fire There's Smoke - 4th Feb 24
New Hints of US Election Year Market Interventions & Turmoil - 4th Feb 24
Watch Consumer Spending to Know When the Fed Will Cut Interest Rates - 4th Feb 24
Blue Skies Ahead As Stock Market Is Expected To Continue Much Higher - 31st Jan 24
What the Stock Market "Fear Index" VIX May Be Signaling - 31st Jan 24
Stock Market Trend Forecast Review - 31st Jan 24

Market Oracle FREE Newsletter

How to Protect your Wealth by Investing in AI Tech Stocks

Explaining the US Consumer Price Index

Economics / Inflation Jul 10, 2007 - 11:17 AM GMT

By: John_Mauldin

Economics This week in Outside the Box we look at a Congressional Budget Office publication that dives us the details on how the consumer price index for all urban consumers (CPI-U) is created. This is, as the CBO posits, the best-known official measure of inflation.

The brief that follows will venture to explain the methods used to construct the CPI-U, and how and why the index's estimates of inflation might differ from a consumer's perceptions of price changes. Though I think this article may be somewhat akin to discovering what components comprise the makeup of sausages (knowledge that is usually best left undiscovered), it is useful to understand just how the inflation data is constructed. Some of the points they make are controversial, especially when it comes to "hedonic" pricing, which they refer to as "shifts in the quality of goods and services over time." This does allow for some quite subjective influence in the inflation numbers. While I will visit this topic in a later letter, it is good to know what is and is not being measured.

I have included the footnotes at the end, and you can link directly to the site at .

John Mauldin, Editor

Explaining the Consumer Price Index

by Adam Weber and John Peterson

The consumer price index for all urban consumers (CPI-U) is the best-known official measure of inflation. Published monthly by the federal Bureau of Labor Statistics (BLS), the CPI-U tries to approximate changes in the cost of living—that is, changes in the cost of maintaining a constant standard of living from one month to the next. To construct the CPI-U, BLS surveys the prices of thousands of goods and services (the index has more than 200 categories of items) in 38 regions, averaging the results to form a nationwide estimate of inflation. Over the past five years, the cost of living, as measured by the CPI-U, has varied but, on average, has risen by about 2-3/4 percent per year (see Figure 1).1

Figure 1.

Growth in the Consumer Price Index for All Urban Consumers

(Percentage change)

Figure 1
Sources: Congressional Budget Office; Department of Labor, Bureau of Labor Statistics.

Notes: The percentage change is measured from the previous year, and the last data point is for May 2007.

The shaded bars in the figure indicate periods of recession. (A recession extends from the peak of a business cycle to its trough.)

The purpose of this brief is to explain some of the methods used to construct the CPI-U and why, in some cases, the index's estimates of inflation may differ from consumers' perceptions of how much prices are rising.2 The brief focuses on six aspects of the CPI-U's construction: averaging regional price indexes to create a nationwide index; estimating the expenditure weights that BLS assigns to the major categories of prices in the CPI-U to account for the categories' relative importance; allowing for shifts in relative prices, a phenomenon known as economic substitution; adjusting for changes in the quality of various goods and services; measuring prices for medical care; and measuring prices for shelter.

Two criticisms of the CPI-U are not discussed. One is the necessary limitations of the index: Despite its broad coverage of prices, the CPI-U does not track the cost of many factors that affect a person's well-being—for example, crime, traffic, pollution, the prevalence of disease, the quality of education, and civil liberties. Cost-of-living measures rarely include such factors because few market prices or consumer expenditures are associated with them, making it essentially impossible to define and measure changes in their price or value.

A second criticism of the CPI-U is that, for technical reasons, the index may overstate changes in the cost of living. Such arguments of upward bias were stronger in the 1990s, before BLS instituted a number of technical improvements. Some analysts argue the opposite, that the CPI-U actually understates inflation.3 Those criticisms go beyond the scope of this brief.

Averaging Regional Price Indexes

BLS uses prices from around the country when it constructs the CPI-U, so changes in the nationwide index do not necessarily mirror changes in a particular region. To produce the nationwide average, BLS first constructs separate price indexes for all 211 basic categories of goods and services in each of the 38 regions covered by the CPI-U—a total of 8,018 basic price indexes. It then weights each index on the basis of expenditures by consumers in that region (see the next section for additional discussion) and averages the indexes together by category and region to arrive at an overall nationwide estimate.4

BLS publishes the regional indexes that it develops, and those measures may diverge noticeably from one another and from the nationwide average. For example, the CPI-U for Los Angeles rose at different rates than the CPI-U for the Cleveland/Akron area during several periods in the past. From January 2000 to January 2007, prices in Los Angeles grew at an average annual rate that was 1.2 percentage points faster than the growth of prices in Cleveland. And as Table 1 shows, regions near the East and West Coasts during the past 10 years have generally experienced higher rates of CPI-U inflation than noncoastal regions have experienced.

Table 1.

Growth in the Consumer Price Index for All Urban Consumers, by Region
Total Growth,  
January 1997 to
January 2007
Nationwide   27.4
Atlanta   25.7
Boston    33.8
Chicago   24.3
Cleveland   24.6
Dallas   25.7
Detroit   27.4
Houston   25.1
Los Angeles   33.6
Miami   30.7
New York   31.1
Philadelphia   28.6
San Francisco   35.1
Seattle   32.3
Washington, D.C.   29.4
Source: Congressional Budget Office based on data from the Bureau of Labor Statistics.

Estimating Expenditure Weights

To develop an overall measure of the change in prices, BLS must assign a weight to each price category in the CPI-U to account for the categories' relative importance. The weights represent the average share of total consumer expenditures on a particular good or service and are calculated by using data from BLS's Consumer Expenditure Survey. If consumers, on average, spend a relatively large amount of their annual income on a particular good or service—food, for example—BLS assigns a large weight to the price of that good or service in the CPI-U.5

Because the expenditure weights are averages, people whose patterns of spending differ greatly from the average might face changes in prices that are larger (or smaller) than those the CPI-U indicates. Elderly people, for example, usually spend more of their annual income on medical care than the average consumer does. As a result, if the cost of medical care (that is, care not covered by insurance or paid for by the government) shot upward, the elderly might feel that inflation was higher than the CPI-U indicated.6

Similarly, a person who spends a higher-than-average amount of his annual income on gasoline might also argue that inflation in recent years has been higher than the CPI-U measured. The average consumer spends a relatively small amount on gasoline, so that price carries a small weight in the CPI-U. In 2005, the price of gasoline rose by 23 percent, an increase that would be felt especially by people who spent a large amount of their income on gasoline.

Measuring Economic Substitution

When relative prices change—for example, when the price of beef rises relative to the price of chicken—consumers may be able to lessen what could be an adverse effect on their standard of living by reducing their consumption of the item whose price has gone up. (In this case, they might substitute chicken for beef.) Such a change in consumption patterns, known as economic substitution, usually cannot eliminate all of the adverse effect of a rise in prices. Nevertheless, if consumers only slightly prefer one good over another, they may not feel that their standard of living is much diminished by substituting one for the other.

BLS's procedures implicitly allow for substitution within most basic categories of goods and services in the CPI-U—such as substituting round steak for sirloin—which helps keep the index from overstating a rise in the cost of living.7 But some parts of the CPI-U, such as rents, certain utilities, and medical services, do not incorporate the procedures because consumers cannot easily substitute one good for another within those categories.

When economic substitution occurs across basic categories of goods and services—when consumers can partially offset an increase in the price of beef by buying more chicken—the official CPI-U misses it and overestimates inflation. To address that issue, BLS in 2002 introduced a CPI measure, the chained consumer price index for all urban consumers (chained CPI-U), that can pick up substitution across categories. BLS cannot reliably calculate that index, however, until data on consumer expenditures for an entire year are available, so it publishes the final calculation of the chained CPI-U more than a year after it publishes the official CPI-U. From 2002, when the methods currently used for calculating the official CPI-U were put in place, to 2005, the last year for which the chained index is not subject to revision, the average growth of the chained CPI-U was about three-tenths of a percentage point slower than that of the official CPI-U.8

Adjusting for Changes in Quality

BLS uses a number of statistical procedures to adjust prices in the CPI-U for shifts in the quality of goods and services over time.9 By removing the effects of such changes, BLS can estimate pure price change—the change that occurs in a good's or service's price if its quality is held constant. Some of the prices in the CPI-U that are adjusted for improvements (or degradations) in quality include those for new and used vehicles, computers, apparel, various appliances and electronic equipment, shelter, and medical care.

Quality adjustment requires BLS to identify material changes in an item and then determine how much those changes contribute to the overall change in the item's price. In the case of medical care, for example, suppose a dentist raises the price of a standard office visit because she decides to include an additional service, such as a fluoride rinse, that before might have been optional and priced separately. BLS determines the value of the added service and subtracts it from the change in the price of the office visit that it has identified through its survey. In the case of costs for shelter, suppose that rents increase because the average unit that BLS uses for its survey is bigger than it was in the past. To calculate the pure price change, BLS identifies the value of the added square footage and subtracts it from the price change it has recorded.10

Estimating pure price change is a more accurate way to measure changes in the cost of living than simply observing changes in market prices. Take the case of a new appliance that is more expensive than an older model but uses less energy. Switching to the new model could increase the cost of living because the new model costs more to buy than the old one did. But the value of the energy that the new model saves might offset the higher purchase price—in which case the new model will not increase the consumer's cost of living. BLS thus factors in energy efficiency as well as appliances' purchase prices in estimating changes in living costs.

Because BLS adjusts prices for changes in quality, the price of a good or service in the CPI-U whose quality is improving declines more quickly or grows more slowly than it would if those adjustments were not being made. For example, BLS has estimated that prices of computers fell by about 9.8 percent (on an annual basis) between March and September 2004. Without BLS's adjustment for improvements in the quality of computers, the drop in prices is about 6.6 percent.11

Adjusting prices for changes in the quality of goods and services is not a straightforward process. BLS has refined its statistical procedures over the years, but those methods may still be subject to error, given how difficult it is to assign a dollar value to improvements in quality. Those kinds of valuations are particularly difficult for developments in medical care technology or upgrades in medical techniques.12

Measuring Prices for Medical Care

No completely satisfactory way has been found to measure health care prices in a cost-of-living index. Rapid changes in quality and the large role that insurers play hamper BLS's measurements, and its estimates of underlying prices are not without controversy. That uncertainty adds to the growing concerns of consumers and policymakers about the rising costs of medical care.

The CPI-U aims to measure the prices of all health care that consumers pay for, either out of their own pockets or through a third party (such as a private insurer or government program) if the consumer pays a premium or some kind of fee—for example, a copayment. BLS uses the results of its surveys to directly measure the prices that consumers pay for care they purchase out of pocket; however, it must measure prices for health insurance indirectly, because a direct approach—attempting to measure the cost of the premiums that insurers charge consumers—raises insuperable issues of quality adjustment. Insurance products may differ widely, and a single product may vary from year to year. Thus, the annual change in the premium that an individual pays will reflect both changes in the price of insurance and changes in the amount or quality of the insurance being purchased.

Disentangling changes in price from changes in quality requires information that insurers have not so far been able to provide, and BLS therefore surveys doctors and other health care providers to determine the prices that they charge insurers. Those prices are assigned the greatest weight in the health care price index. The CPI-U also includes an estimated price for health insurers' overhead (which includes the cost of administration and of maintaining reserves plus profits). That price is assigned a relatively small weight.13

Prices for goods and services that are not paid for by the consumer out of pocket or through premiums or fees are excluded from BLS's price indexes. Thus, prices for medical goods and services that are covered by Medicaid or Part A of Medicare (the Hospital Insurance program) are not part of the index because the government pays for all costs associated with those programs. In contrast, prices for goods and services covered by Parts B, C, and D of Medicare (the Medical Insurance, Medicare Advantage, and prescription drug programs) are included because consumers pay premiums or fees to participate in the programs.

BLS uses different methods in weighting categories of medical prices. To weight the prices of goods and services that consumers pay for directly, out of pocket, BLS looks at the share of total consumer expenditures that those items make up. (It excludes payments on a consumer's behalf by the government, employers, or charitable organizations.) To weight the prices that health care providers charge insurers, BLS considers insurers' payments to providers; to weight the price of health insurance overhead, it assigns a small portion of total consumer expenditures on health insurance—the portion corresponding to the overhead—to the index weight.

Measuring Prices for Shelter

BLS bases its price index for shelter on a survey of rents, using it to construct a measure for the rent paid by households that do not own their homes (rent of primary residence) and a measure for the rent that homeowners would pay if they rented their homes instead of owning them (owner's equivalent rent).

BLS does not use the prices of houses to measure shelter prices because house prices and rents are not directly related.14 The prices of houses may influence rents in the long run, but there is no close month-to-month or even year-to-year relationship between the two measures; moreover, rents are less volatile than house prices (see Figure 2). House prices may be useful for measurement in some cases—for example, if shelter is being treated as an asset that can be bought and sold for a profit or at a loss—because they measure the cost of that investment. In a cost-of-living context, however, shelter is not an investment but a source of housing "services" that are used, or consumed, over time, a form of consumption that economists measure through rents.15

Figure 2.

House Prices and Owner's Equivalent Rent

(Percentage change)

Figure 1
Sources: Congressional Budget Office; Department of Labor, Bureau of Labor Statistics; Office of Federal Housing Enterprise Oversight (OFHEO).

Notes: House prices refer to the House Price Index published by OFHEO, which tracks single-family house prices. Owner's equivalent rent is the rent that homeowners would pay if they rented their houses instead of owning them.

The percentage change is measured from the previous year, and the last data points are for the first quarter of 2007.

The shaded bars in the figure indicate periods of recession. (A recession extends from the peak of a business cycle to its trough.)

Similarly, BLS does not use mortgage payments to measure the cost of shelter because the payments do not necessarily represent the cost of the housing services that owners are paying for. Mortgage payments are based on the amount of the purchase price that is financed and the terms of the financing. Good credit, low interest rates, large down payments, and long-term mortgages can reduce an owner's monthly mortgage payments on a home, but they will not necessarily reduce the rental rate of the property if it is rented out.

Other measures of inflation have risen at roughly similar rates in recent years. Those measures include the consumer price index for urban wage earners and clerical workers, which is used to adjust Social Security benefits for inflation, and the personal consumption expenditure chained price index, which is computed by the federal Bureau of Economic Analysis.

Recent articles in the popular media have discussed the validity of the CPI-U and whether it reflects the inflation that consumers actually experience. See, for example, Floyd Norris, "What Happens If Inflation Is Overstated?" New York Times, June 9, 2006, p. C1; David Wessel, "Why Inflation Seems to Have Sharper Teeth Than the CPI Suggests," Wall Street Journal, March 16, 2006, p. A2; Daniel Gross, "How Home Prices Can Be Hot but Inflation Cool," New York Times, June 26, 2005, section 3, p. 5; and "New and Improved: An Inflation Debate Brews Over Intangibles at the Mall; Critics Say U.S. Plays Down CPI Through Adjustments for Quality, Not Just Price; Value of a TV's Flat Screen," Wall Street Journal, May 9, 2005, p. A1.

See Robert J. Gordon, The Boskin Commission Report: A Retrospective One Decade Later, NBER Working Paper No. 12311 (Cambridge, Mass.: National Bureau of Economic Research, June 2006), available at; and David S. Johnson, Stephen B. Reed, and Kenneth J. Stewart, "Price Measurement in the United States: A Decade After the Boskin Report," Monthly Labor Review (Bureau of Labor Statistics, May 2006), p. 11, available at

For more information on the structure of the CPI-U, see Bureau of Labor Statistics, Handbook of Methods (October 2006), Chapter 17, available at

Using expenditure shares to weight the CPI-U implies that the index is more representative of people who spend large amounts than of people who spend small amounts. See Mary F. Kokoski, Alternative Consumer Price Index Aggregations: Plutocratic and Democratic Approaches, BLS Working Paper No. 370 (Bureau of Labor Statistics, December 2003).

BLS publishes an experimental CPI, the CPI-E, which may more closely approximate the rate of inflation that the elderly experience. Creating inflation measures for subgroups is difficult, and comparing the CPI-E with the CPI-U is complicated by various limitations in methodology. However, according to tentative findings from a recent BLS study, elderly people from December 1997 through December 2005 may generally have experienced a slightly higher rate of inflation than other consumers did. (During that period, the CPI-E grew by 24 percent, and the CPI-U grew by about 22 percent.) See Bureau of Labor Statistics, The Consumer Price Index—Why Published Averages Don't Always Match an Individual's Inflation Experience, Factsheet 93-1 (revised March 2002); and Bureau of Labor Statistics, Consumer Price Index Program, Experimental Consumer Price Index for Americans 62 Years of Age and Older, 1998–2005 (April 2006), available at

In 1999, when the BLS first introduced the procedures that allow for substitution within most basic categories of goods and services in the CPI-U, the agency expected those procedures to lower the annual growth rate of the CPI-U by about two-tenths of a percentage point per year. See Kenneth V. Dalton, John S. Greenlees, and Kenneth J. Stewart, "Incorporating a Geometric Mean Formula into the CPI," Monthly Labor Review (Bureau of Labor Statistics, October 1998), p. 3. For a recent look at the effect of adjusting the CPI-U for substitution within basic categories of goods and services, see Johnson, Reed, and Stewart, "Price Measurement in the United States," pp. 12–13.

For a technical description of the chained CPI-U, see Robert Cage, John Greenlees, and Patrick Jackman, "Introducing the Chained Consumer Price Index" (paper presented at the Seventh Meeting of the International Working Group on Price Indices, Paris, May 2003), available at

For more information on the procedures used to adjust for changes in the quality of goods and services, see Mary F. Kokoski, "Quality Adjustment of Price Indexes," Monthly Labor Review (Bureau of Labor Statistics, December 1993); and Bureau of Labor Statistics, Handbook of Methods, Chapter 17.

Bureau of Labor Statistics, Handbook of Methods, Chapter 17, pp. 25 and 28.

See Johnson, Reed, and Stewart, "Price Measurement in the United States," Table 6.

See Bureau of Labor Statistics, Handbook of Methods, Chapter 17, p. 28. For an example of the intricacies involved in adjusting for quality change in medical care, see David M. Cutler and others, "Pricing Heart Attack Treatments," in David M. Cutler and Ernst R. Berndt, eds., Medical Care Output and Productivity (Chicago: University of Chicago Press, 2001), pp. 305–362.

For more information about the treatment of health insurance in the CPI, see Dennis Fixler, "Direct Pricing of Health Insurance in the Consumer Price Index" (paper presented at the Sixth Meeting of the International Working Group on Price Indices, Canberra, Australia, April 2001), pp. 2–6; Ina K. Ford and Daniel H. Ginsburg, "Medical Care in the Consumer Price Index" in Cutler and Berndt, eds., Medical Care Output and Productivity, pp. 214–217; and Bureau of Labor Statistics, Handbook of Methods, Chapter 17, p. 26.

See Richard Rosen, "Explaining Recent Changes in Home Prices," Chicago Fed Letter, no. 216 (Federal Reserve Bank of Chicago, July 2005), available at; and Robert Poole, Frank Ptacek, and Randal Verbrugge, "Treatment of Owner-Occupied Housing in the CPI" (paper presented to the Federal Economic Statistics Advisory Committee, Washington, D.C., December 2005), available at


Some cost-of-living indexes, particularly those that attempt to account for consumer substitution over longer periods, specify a larger role for changes in asset prices. See Ricardo Reis, A Cost-of-Living Dynamic Price Index, with an Application to Indexing Retirement Accounts, NBER Working Paper No. 11746 (Cambridge, Mass.: National Bureau of Economic Research, November 2005), available at


I hope this brief has shed light upon the methods through which we quantify inflation.

Your still concerned about inflation analyst,

By John Mauldin

To subscribe to John Mauldin's E-Letter please click here:

Copyright 2007 John Mauldin. All Rights Reserved
John Mauldin is president of Millennium Wave Advisors, LLC, a registered investment advisor. All material presented herein is believed to be reliable but we cannot attest to its accuracy. Investment recommendations may change and readers are urged to check with their investment counselors before making any investment decisions. Opinions expressed in these reports may change without prior notice. John Mauldin and/or the staff at Millennium Wave Advisors, LLC may or may not have investments in any funds cited above. Mauldin can be reached at 800-829-7273.


John Mauldin Archive

© 2005-2022 - The Market Oracle is a FREE Daily Financial Markets Analysis & Forecasting online publication.

Post Comment

Only logged in users are allowed to post comments. Register/ Log in