So the Kennedy family wasn't originally a Boston elite family

Since they were an Irish Catholic family,
they must have been quite far from the WASP crowd,
but I thought they were at least a family that carried weight in Boston.
The traditional mainstream "Boston Brahmins"
were families like the Adams, Cabots, and Coolidges.