Consumer Reports’ Car Reliability FAQ

What is unique about Consumer Reports' survey?

Consumer Reports' auto reliability information is unique in several respects:

• We bring this data to the information marketplace with no fear or favor. Because we have no clients beyond our readers, we can report all the data, not just the autos with top results. We needn't worry about losing advertising in our magazine, so we can interpret the data with total independence.

• Our subscribers tend to be well-educated and appreciative of objective, independent research, which makes them unusually qualified to provide valuable data about their experiences.

• The sheer size of the reliability survey conducted by the Consumer Reports National Research Center enables us to look at owners' experiences with hundreds of thousands of cars. Unlike other reliability surveys, we are able to look at car models in several variants, in some cases reporting them separately by type of engine, drive types, trim lines or other classifications.

How does CR's survey compare with other reliability surveys?

The timeframes used in collecting data also differ among surveys. J.D. Power's IQS is based on the first 90 days of ownership of new vehicles only, and its VDS (Vehicle Dependability Study) is exclusively based on 3-year-old vehicles. CR's survey asks about subscribers' experiences with their vehicles over the course of the previous 12 months and, starting with 2006, Consumer Reports' survey now covers 10 model years—from brand-new models to models that are 10 years old, providing a more complete profile of the life of a model. Our Predicted Reliability, which forecasts how well a new model is likely to hold up, is based on the cumulative experiences of owners of the three latest model years.

Where is the data from?

Consumer Reports obtains its reliability data from an Annual Questionnaire that is sent to subscribers to ConsumerReports.org and Consumer Reports magazine. In all, the survey was sent to over 8 million subscribers in 2013, and we received responses on 1.1 million vehicles.

How is the survey conducted?

The Consumer Reports National Research Center conducts the survey each spring. In the questionnaire, we ask subscribers to note any problems with their cars that occurred in the past 12 months. They are asked to identify problems that they considered serious (because of cost, failure, safety, or downtime). We ask them to include problems covered by warranty, but not the ones resulting from accident damage or due solely to recall. Respondents check off problems from a list of trouble areas, ranging from the engine and transmission to climate system, brakes, electrical system, and power accessories. See the full list of trouble spots.

How current is the data?

All our reliability information is completely updated annually. We begin sending out each year's survey in the spring. By late summer, we have collected and organized responses, and we complete our analysis and update the information online by late October. The new information first appears in print in the Consumer Reports Best & Worst New Cars, on newsstands in mid-November. Subsequent auto publications, such as the New Car Buying Guide, also use this new information. In the pages of Consumer Reports, we update Predicted Reliability and Recommendations in the vehicle Ratings beginning in the road tests in the November issue. Changes to new car recommendations and predicted reliability scores by vehicle type are published in the December issue and used car results are published in the following April issue. All reliability information we publish is based on subscribers' experiences with cars in the 12-month period immediately preceding the survey.

What expertise does CR's survey staff have?

CR's Annual Questionnaire is constructed and implemented by the staff of our National Research Center. The staff of this department includes professional social scientists, some with more than 30 years of experience in constructing, conducting, and interpreting large-scale surveys. The automobile section of the survey is developed in consultation with CR's automotive engineers and statisticians to ensure that we are capturing the most important aspects of auto reliability. The data is analyzed by professional statisticians and survey analysts.

Our decisions about the construction of the survey and the analysis and interpretation of the data are continually reviewed with an eye on giving consumers the most valuable and useful information possible.

How many cars do you have information on overall?

CR's Annual Questionnaire is one of the largest scientific surveys conducted in the United States. Our 2013 survey, which was sent to subscribers of Consumer Reports magazine and to ConsumerReports.org subscribers, gave us feedback on our subscribers' experiences with 1.1 million vehicles. This high number of responses allows CR to provide the most comprehensive reliability information available to consumers.

How many samples do you have of each model?

While we do not publish information on individual sample sizes for specific models, we require a minimum of around 100 cars to publish reliability information for a model in a given model year. Our sample sizes tend to track quite closely with market sales. Individual sample sizes vary from year to year and range from a hundred to several thousand for the more popular models. A typical model has about 200 to 400 samples for each model year and engine variant.

Since we've opened the survey to subscribers of ConsumerReports.org, we have seen a substantial increase in survey responses in the past few years, so individual sample sizes have generally been on the rise, as well. This has given us sufficient sample sizes on a number of low-volume models.

What effect does having a larger sample size for some vehicles compared with others have on the validity of the reliability data?

Given an appropriate sample, the more data you have, the more statistical confidence you have in your information. A larger sample will always give more accurate information than a smaller sample (assuming, of course, that the data are valid and collected from an appropriate source).

While we require a minimum of about 100 cars to publish reliability information, most models have larger samples than that, some being as large as several thousand. We present our data primarily to allow subscribers to compare the detailed reliability histories and overall reliability for different models. While models whose scores are based on more cars are reported with greater accuracy than those based on smaller sample sizes, the way we calculate our scores has been devised to allow valid comparisons for all samples we publish, regardless of the particular sample sizes of individual models.

Where does the minimum sample size of 100 come from?

Our statisticians have determined that a minimum sample of 100 is sufficient to allow us to report statistically-meaningful differences among models. With larger sample sizes, we could detect even finer differences among models. However, using a higher threshold for minimum sample size, we would have insufficient data for most lower-volume models, as well as new models introduced late in the model year. With smaller sample sizes, we would be more limited in our ability to detect differences among models, although we would then have sufficient data for more models.

The minimum sample size of 100 cars allows a good balance for us to provide accurate information on model differences, while still covering a majority of models on the market.

Why is there no reliability information for some models?

Consumer Reports sets a minimum sample size of about 100 cars; this size sample allows us to ensure that we have sound feedback from our subscribers to properly gauge the reliability of a model. We won't offer reliability information on those models for which we do not have a sufficient amount of data to draw a solid conclusion.

Our sample size generally tracks well with consumer market sales but may not correlate well with models that have high fleet sales. However, if a new model is introduced to the market at a time of year that does not coincide with our survey period, we might not get sufficient samples on that new model.

Are all automotive problems included?

Respondents to our survey are asked to identify problems they have experienced in a 12-month period in any of 17 trouble spots. We do not publish scores for advanced safety systems and air bags since the problem rates in the area are almost universally very low.

What do the trouble areas cover?

We have been revising the trouble areas since our 2007 survey to be more comprehensive and more closely reflect problems in newer vehicles and better define areas with high problem rates. The following are some changes:

Starting with the 2010 survey, Engine electrical now includes hybrid battery and related system in addition to other charging and ignition systems.

The trouble spot Transmission is separated into Transmission major (more serious problems such as transmission rebuild or replacement and torque converter) and Transmission minor.

Body Hardware includes power components which were formerly under Power equipment.

Our Reliability History charts cover problems in any of 17 trouble areas. Here's a look at what's covered in each of those areas:

ENGINE MAJOR: Engine rebuild or replacement, cylinder head, head gasket, turbocharger or supercharger, timing chain or belt.

ENGINE MINOR: Oil leaks, accessory belts and pulleys, engine mounts, engine knock or ping.

ENGINE COOLING: Radiator, cooling fan, water pump, thermostat, antifreeze leaks, overheating.

TRANSMISSION (AND CLUTCH)-MAJOR: Transmission rebuild or replacement, torque converter, clutch replacement.

TRANSMISSION (AND CLUTCH)-MINOR: Gear selector and linkage, transmission computer, transmission sensor or solenoid, clutch adjustment, rough shifting, slipping transmission.

DRIVE SYSTEM: Driveshaft or axle, CV joint, differential, transfer case, four-wheel-drive/all-wheel-drive components, driveline vibration, electrical failure, traction control, electronic stability control (ESC).

FUEL SYSTEM/EMISSIONS: Check-engine light, sensors (O2 or oxygen sensor), emission-control devices (includes EGR), engine computer, fuel-injection system, fuel cap, fuel gauge/sender, fuel pump, fuel leaks, stalling or hesitation.

ENGINE ELECTRICAL: Starter, alternator, hybrid battery and related system, regular battery, battery cables, engine harness, coil, ignition switch, electronic ignition, distributor or rotor failure, spark plugs and wires failure.

CLIMATE SYSTEM: A/C compressor, blower (fan) motor, condenser, evaporator, heater system, automatic climate system, electrical failure, refrigerant leakage.

SUSPENSION/STEERING: Shocks or struts, ball joints, tie rods, wheel bearings, alignment, steering linkage (includes rack and pinion), power steering (pumps and hoses, leaks), wheel balance, springs or torsion bars, bushings, electronic or air suspension.

BRAKES: Antilock system (ABS), parking brake, master cylinder, calipers, rotors, pulsation or vibration, squeaking, brake failure or wear.

EXHAUST: Exhaust manifold, muffler, catalytic converter, pipes, leaks.

PAINT/TRIM/RUST: Paint (fading, chalking, peeling or cracking), loose trim or moldings, rust.

BODY INTEGRITY: Squeaks, rattles, wind noises, loose or cracked seals, and/or weather stripping, air and water leaks.

BODY HARDWARE (Power or manual): Windows, locks and latches, doors or sliding doors, tailgate, trunk or hatch, mirrors, seat controls (movement and temperature), seat belts, sunroof, convertible top.

POWER EQUIPMENT AND ACCESSORIES: Cruise control, clock, warning lights, body control module, keyless entry, wiper motor or washer, tire pressure monitor, interior or exterior lights, horn, gauges, 12V power plug, alarm or security system, remote engine start.

AUDIO SYSTEM (excluding aftermarket systems): radio, speakers, antenna; CD or DVD player; GPS, iPod & MP3 interface; communication system (e.g. ONSTAR, Bluetooth), backup camera/sensors.

Are all problems considered equally serious?

Problems with the engine-major, cooling system, transmission-major, and driveline are more likely to take a car out of service and to be more expensive to repair than the other problem areas. Consequently, we weigh these areas more heavily in our calculations of Used Car Verdicts and Predicted Reliability. Problems in any area can be an expense and a bother, though, so we report them all in the Reliability History charts.

What different reliability scores does CR publish?

Consumer Reports uses the data from its Annual Questionnaire to compile detailed Reliability Histories on several hundred makes and models of cars, minivans, pickups, and sport-utility vehicles, spanning 10 model years. For each model that we have sufficient data, the Reliability History Chart shows you whether the model has had more or fewer problems than the average model of that year in each of 17 trouble spots. That information can be a big help when inspecting and purchasing a used car. The Used Car Verdict summarizes the 17 trouble spots for each model year and compares that to the average of all vehicles in the same model year. We use these Verdicts to identify lists of Reliable Used Cars and Used Cars to Avoid.

For new models that are currently available, our Predicted Reliability rating is based on the model's recent history, provided the model hasn't been significantly redesigned for the current model year. Online, Predicted Reliability is presented in the new car model overview pages in the Ratings Report Card, and Ratings & Specs and in the Vehicle Overall Ratings comparison. It is also incorporated into the Reliability History charts as the New Car Prediction.

We also present Predicted Reliability in more detail in our graphs. In this presentation, bar graphs show the percentage difference between each model's overall reliability and the average reliability of all models. We group models by vehicle type (for example, family cars or minivans), for ease of comparing models that are direct-market competitors.

Our statisticians also do in-depth analyses of the reliability data to provide information to consumers about trends in automotive reliability, reliability of newly introduced models, and other important issues. These analyses are presented in the April issue of Consumer Reports, at ConsumerReports.org, and in newsstand auto publications throughout the year.

What are the Reliability History charts?

The chart for an individual model will tell you where a model's strengths and weaknesses have been. Scores are based on the percentage of survey respondents who reported problems for that trouble spot, compared with the average model of that year. Models with problem rates close to the mean receive a

for that trouble spot. Models with scores of

or

are not necessarily unreliable, but they have a higher rate of problems than the average model. Similarly, models that score

are not necessarily problem-free, but they had relatively few problems compared with the average model.

In particular, within each trouble spot and within each model year, we create equal-width intervals for the

,

,

, and

, constructed so that the average model is the middle of the

, and the interval for the

begins at 0 percent.

An exception to this rule occurs when the mean problem rate is quite small (less than 2.5 percent), as is often the case in newer model years. In order to avoid making distinctions that are too fine to be meaningful, we do not assign a

unless the problem rate is at least 3 percent, or a