As always, when posting such a message, you have an epiphany right after...
It seems the issue had a different origin that just coincided with the update to 12.3.
The actual reason for my Push 2 not working is the Scaler 3 setup. To enable multiple midi outputs, one now has to redirect the midi information, and it seems that this causes issues. I assume that Push 2 needs to have the tracks use "all ins", which you cannot use with the official setup for Scaler 3 suggested by the creators. No blame, as they could probably not foresee this very peculiar issue. My apologies for not realizing this before.
If I find a good setup for Scaler 3 together with Push, I will make a new post about it.