BMW Reliability Ranks #3 in CarMD 2012 Survey

[bradleyland]

It's like anecdote hell in here. Your individual experience doesn't dictate overall reliability. You are a single point of data. Reliability is measured as a probability, in which case a dataset of 1 is useless.

The good news is that automobile reliability is measured in terms of defects per thousand, and cars have never been more reliable than they are today [1]. We've become exceedingly good at precision manufacturing.

On to the actual survey. Keep in mind what CarMD does/is. Their product is a code reader that can be used to diagnose automobile issues. Their claim is that the report is "unbiased", but I'd say that's a misnomer. It may be free from the bias associated with customer surveys, but the data itself may be biased by product itself. That is to say, the nature of the product is going to dictate their data set.

I can't see why a new car buyer would purchase a CarMD unit or subscription when service at the dealership is free, so I expect that their data set is from older cars. That still makes it useful, but I wish they were more clear about what their data set is composed of. What is the age distribution (min/max/avg) of the cars in the database? A histogram of the age by year would be nice. Also, what is the distribution by make? Another histogram of the records by make would be nice.

The link says the data is available somewhere. If I can get my hands on it, I'll do some looking in to these questions and post back here. One thing is certain: this is actual data collected from the field, so we shouldn't be so dismissive. It may not answer the question we're asking, but it certainly contains answers to some questions.

1 - http://www.cnbc.com/id/46397659/Car_...ts_Record_High

[bradleyland]

Looks like the data on the website is only available in summary form. This makes sense, as they'd be giving away the secret sauce if they disclosed the full dataset.

[Kyle B]

Other than maybe an oil change, I'd love to know what people are having repaired on their 2012 BMWs for only $502.48...

[RMG]

Originally Posted by 135Pats

BMW is not more reliable than Honda.

...depends. I've leased a Honda Accord ~10 years ago. It was a mess. The engine worked fine but I had noises coming from the right hand side suspension that the dealership tried to fix many times. I used to work across the street from the dealership and the car had numerous visits there, still when the lease came to the end the noise was still present. When it was humid outside there was a concert of noises coming from everywhere inside the car. I don't have the time to list all the issues I had. Since then I leased 3 e9x, two fitted with the infamous N54. I didn't have any problem with any of them.

[CivicSi_to_335i]

Originally Posted by aj555

BMW had 4 engines win at the international best engine of the year contest. The N54 was one of them.

The International Engine of the Year Award has very little to do with reliability. The contest is judged by a panel of automobile journalists from around the world on various parameters such as performance and fuel economy. If you look at the past winners, they usually award breakthrough designs that push the state of the art in terms of HP/liter or MPG for a given HP. The N54 was a very innovative engine which employed a lot of new technology and great performance at the time of release. Having owned one for the last 5 years and done a lot of research on the motor I can vouch for the fact that it's reliability while good is nowhere near world class, particularly during the first couple of years of production. There's a reason all the higher production number models use the N55 other than fuel economy and cost of course. It is the more reliable motor.

More on the scoring details:

SCORING
In every category, the panelists judged each shortlisted engine using their subjective driving impressions and technical knowledge, and took into account characteristics such as fuel economy, smoothness, performance, noise and drivability. The jurors each had 25 points to award to their five favourite engines in each category. A maximum of 15 points could be allocated to an engine, and the minimum was one point. An engine could not be tied for the top spot.

http://www.ukipme.com/engineoftheyear/rules.php

[Bingobimmer]

I'm surprised

[ManiacGT]

A voice from the UK agrees this list seems hard to believe. I too cannot believe BMW is more reliable than Honda. Honda is a beacon of reliability.

I've had no end of issues and continue to in my e89. If Honda made something I wanted, I'd move to them in a heartbeat as I know many owners that laugh constantly at my so called premium brand car when their run of the mill Honda never let's them down.

Here's another survey.. Who do you believe?

http://m.whatcar.com/car-news/what-car-reliability-survey-2012/263555

[bradleyland]

Building a statistically accurate model of an entire system from a sampling of data is incredibly difficult. Each of these data sets has their own issues. The biggest one is that all the sampling sizes are usually too small to make an accurate inference.

The USDoT says there were 5.6 million passenger cars sold last year, as well as 4.1 million trucks [1]. I don't know what portion of those trucks are for use by your average buyer, but I know that Ford sold a half million F-series pickups. If we said that about half of those 4.1 million were consumer truck sales, the total number of vehicles sold would be around 7.5 million (using round figures). 7.5 million is a HUGE number.

Looking at the total number of cars on the road, the USDoT puts that at around 250 million registered vehicles, 190 million of which are "Light duty vehicle, short wheel base" [2].

The CarMD data FAQ [3] gives some information, but I find it a little bit confusing. More to the point the report posted here says:

CarMD's network of thousands of certified automotive technicians and database of more than 3 million verified repairs. The November 2012 Index statistically analyzes repairs that apply to roughly 136 million model year 2002 to 2012 vehicles, taking place in the U.S. from Sept. 1, 2011 through Sept. 1, 2012.

Ok, so the CarMD index is based on 3 million verified repairs across 136 million vehicles in a 1 year period. That's over half of the registered vehicles on the road. Looking pretty good from a statistical standpoint.

The WhatCar survey posted above uses data from Warranty Direct; a company that sells extended warranties. I searched the source website at http://www.reliabilityindex.com but was unable to find any information on their sample size. They use only vague figures (they spend millions of pounds a year on claims). If we divided "millions of dollars" by the average repair cost in the CarMD data, you could be looking at a sample size in the hundreds of thousands. It's hard to say, but they're not forthcoming. That doesn't inspire confidence.

Let's look at JD Power:

The study, which is based on responses from more than 31,000 original owners of 2009 model-year vehicles after three years of ownership, measures problems experienced during the previous 12 months by those original owners.

So the sample size is 31,000 for a single year model. Ok, so you don't have to have a masters in statistical analysis to recognize the problems here:

1) It's a survey, so you're relying on PEOPLE to accurately report their experience. This is a bad idea, because people are extremely susceptible to bias.

2) The sample size is only 31,000. That's 0.4% of the number of cars sold each year. You'd have to have some serious science to back up your selection criteria in order for this sample size to reflect the full 7.5 million car data set.

So, for my money, the CarMD index is looking pretty damn good. It's certainly possible that their data is flawed as well, but the fundamentals look good.

This reminds me a little bit of the whole Nate Silver, 538 election prediction. No one wanted to believe him, but his models were based on simple mathematics, which is hard to refute.

1 - http://www.bts.gov/publications/nati...ble_01_12.html

2 - http://www.bts.gov/publications/nati...ble_01_11.html

3 - http://corp.carmd.com/Page/Detail/133

[shivaswrath]

O2 sensors and fuel caps... Lol.

[georgere]

Really? Does not sound right to me as BMW owner. Should be someplace after Japanese, Koreans, and some Americans; perhaps at 20th place?
P.s. just ridiculous......

[Beemw335]

Bmw and vw in top 10 reliability hahahaha this has to be a sick joke.
My family has owner over 4 recent bmws and I can say with confidence, BMW is nowhere near being top 3 reliability.
Nissan/Infiniti Honda/Acura Toyota/Lexus Gm Ford all make sense, bmw and vw...not so much

[debdoub]

Originally Posted by KingOfJericho

Best - maybe. Most reliable? No sir...

My parents have had Toyotas in various forms for 3 decades and over a million miles and have had zero failures. I had my BMW for nine months and it left me stranded.

I believe you entirely, BMW has many problems however I have 2

[Snaggletooth]

LOL @ this whole thread. The N54 is a beast, but it's just not as reliable as it should be. A lot of great cars aren't reliable, a lot of reliable cars aren't great.
I'll bet dollars to donuts that the N54 has more problems than all other BMW products. Is it the end of the world when a fuel pump or coil pack goes? naw. Is it an inconvenience? Absolutely.

That being said, I love the power and fuel economy. If only the components could keep up with the high bar set by the rest of the engine!

[cjh]

Originally Posted by SubzeroVR4

I used to have a 2004 Acura TL and it would visit the dealer multiple times. It didn't have the typical Honda build. I've went through two transmissions and numerous amount of electrical problems. I assumed I got a bad apple but my buddy purchased a 2007 TL and had to get a new engine already. My BMW has been more reliable.

The early model year (mostly 04, occasional 05) TLs experienced a fair amount of transmission failures. Aside from that they're as reliable as any other Honda...they essentially are an Accord V6 dressed in a suit. A Honda engine needing replaced is certainly an anomaly.

[Imola.ZHP]

I've only been stranded twice in almost 7 years...

Alternator while still under extended warranty, and fuel pump a few years after...

She's a maintenance whore, but very reliable and worth every penny...

My good friend with a 2007 335i however, is a completely different story... And not a good one...

[Frosty]

Originally Posted by Snaggletooth

they probably excluded the N54 from that list haha!

HA HA HA!!!!

[Falcao]

What a joke. They must have also excluded the "very reliable" s85 V10 monster engine and very reliable "SMG"

For those of us that own the cars, we know better.

[jon328i]

BMW #3 spot.....congrats...hope the numbers are accurate...

[aj555]

Originally Posted by bradleyland

It's like anecdote hell in here. Your individual experience doesn't dictate overall reliability. You are a single point of data. Reliability is measured as a probability, in which case a dataset of 1 is useless.

Exactly, I have a personal friend who owned a fairly large car maintenance shop that specifically worked on European cars. He never owned a BMW, but he always goes on about how BMWs are completely unreliable and break all the time because he sees them a lot. I always have to remind him that people with perfectly working BMWs usually don't come to him to have stuff fixed, and that his shop is in an area with a massive amount of BMW owners.

[Meeni]

Report on BMW includes Mini cars, so BMW can thank Peugeot/PSA/Citroen for the good ranking, it seems.

BMW gets most expensive repair per occurrence, so its like russian roulette, it's unlikely to shoot, but when it does it hurts "real bad".

On the VW front, Audi is included, and fares significantly better (why is a good question to ask VW executives, as many parts are identical, there must be some corner cutting involved in the assembly lines, maybe due to assembly in Mexico/USA compared to assembly in Germany for Audi ?).


Honda/Toyota and friends score big in term of "problem free" models. Some lines of car seem to just work forever according to the detailed report (list of most 100 reliable individual cars per model and per year). One conclusion that explain low overall rating is that while they have problem free models, the brand is plagued with lemon models that pull the overall grade low.

BMW individual models score just average, and the x28 and x35 cars are merged, which means the problematic turbos get diluted into the large mass of NA engines per year.


There may be a systematic bias due to BMW long factory maintenance, that is probably not reported (BMW dealership are ASE ? ). Problems arising in the first 4 years of the car are probably not reported while other brand see problems being reported after yr1 of ownership.

[Pappy]

Originally Posted by batislav

Didn't know if anyone else shared this, but BMW was ranked #3 in reliability by CARMD. Not sure about individual models though.

Never even heard of CarMD. Think I'll start a Pappy MD....

[suprman]

I don't know how reliable is that reliable list of reliable cars from CarMD???

My parents had a very reliable 80s Toyota and dependable 90 Nissan, but not so on their 80s Buick (transmission prob). My current 07 E92 been reliable on the most part minus a dead battery. Fuel pumps was replaced free under recall. Besides minor interior rattle noise, everything is holding up well, running 16k * knock on wood