I think the character drift (and the random access humour that started just being skits to wait out the clock) was when I started tuning out of family guy.
Lois started off as an average mom (a little selfish, but at least self aware), to being this narcissistic bitter hag who was clearly waiting for Peter to die of heart failure.
Peter went from a dumb guy to barely functioning retard who only gets up to shit to create a plot.
Stevie used to be an evil genius and then just became gay jokes.
Meg and Chris were pretty standard, the writers didn’t know how to write daughters so Meg became a punching bag. But chris used to have somewhat relatable teenage boy drama episodes. He became such a background feature though.
And as mentioned above, Brian became a really over-done self-insert joke, but also completely stopped being enjoyable.
I think it got to the point where Peters adventures with Joe, Quagmire, and Cleveland were the only thing left that was entertaining.