As someone who has worked for AutoTrader.com in the past, I have seen both sides of this coin. I have seen some dealers that get 80% of their business off of AT and I have also seen dealers that list everything in the premium section and don't get much bang for the buck, at least not as much as it seems they should.
The common thread that ran through the dealers getting outstanding results on AT was that they had something unique about their vehicles. The most obvious one being price. There was/is a dealership that always has X amount of vehicles below $2000. They got created dozens of sales every month off of AT. The second uniqueness was in the vehicle itself. Cars that were limited edition, had rare options, or were rare themselves and hard to find tended to do very well on AT. The dealers that consistently carried these types of vehicles got nice results.
Consider what you're selling and where it will be listed on AT. Is your vehicle going to get lost in a sea of other similar vehicles? Will someone fly in from across the country or drive 3 hours for your vehicle or can they get the same thing at a very similar price 5 minutes from their home? Does your price stand out in any way? Does the vehicle stand out in any way?
If your vehicles are not going to stand out, then it's probably best to be doing an SEM/PPC campaign and drive traffic to your website. This way, when consumers are searching inventory, they are searching only your inventory, not every other dealers' as well. As someone mentioned earlier however, the website needs to be strong and created to provide an engaging user experience, which will translate into more leads/better conversion. As a general rule I think it's better to generate your own leads rather than using a third party, but you have to go with what works.
I hope this helps. I do know that AT does work if used correctly and the people there are great.