My manager insists on driving 4 hours in his own car instead of taking the KTX, which only takes an hour and a half

This is about work; it's a project I'm in charge of, and a business trip has been scheduled.
To be honest, I could go alone, but since we usually go in pairs, I'll be going with my manager.
The destination is in Jeonbuk. Since I live in Seoul,
it's a destination that takes only an hour and a half by KTX, followed by a 20-minute drive via Socar.
However, if we leave from the office together in his personal car, it takes 4 hours.
When I suggested taking the KTX and then using Socar, he insisted on driving his own car himself, saying he would drive.. Even though it's frustrating, I said okay since he's my superior.
When it's time to come back, would it be okay if I lied and said I was going to my parents' house and took the KTX back alone? Since it's Friday, if we finish work around 2 PM, the traffic will be even worse..
If I do this, I won't be able to claim KTX travel expenses, but I don't mind using my own money or my accumulated KTX mileage. If I use the excuse that I have to go straight to my parents' house because it's Friday,
is there anything my manager might find offensive or suspicious about this?
It's just frustrating; even if I'm not the one driving, being stuck in a car next to him is exhausting. I don't understand why he won't choose the faster, more comfortable way to save time for both of us.