Disadvantages of buying a car from a non-local dealership?

Won’t get preference when you need to get car serviced, might not get a loaner

There isn’t really a downside other than transportation to get the vehicle, and the fact that dealers play games once they get you in the door. For instance the last car I went to buy was not actually there when I got there even though I explicitly said I was coming to buy it at the negotiated price. It was only a 30 minute drive and were able to work something out with one they had on the lot. I would have been pissed if I didn’t know I was getting a great deal on my trade, which I also negotiated beforehand.

Also, keep in mind that different states may have different distributors (ex gulf states Toyota vs southeast Toyota) that charge different destination fees, which won’t be waived. So you just need to know what all those fees are so you can compare apples to apples.