Some sort of penalty should be applied to people who don't accept matches in Ranked.
I'm assuming there is none at the moment as today I tried Ranked solo and for 3 times in a row it was cancelled because 1 player didn't accept. (after the 3rd time I decided to press "LEAVE PARTY" and try anew)
I would think it is not a common occurrence so I'm thinking the game kept asking the same guy and he probably was away from the keyboard (or trolling).
I remember seeing in another game the implementation of a time penalty, for example:
1st time failing to accept - 1 minute wait until it could try Ranked again;
2 times in a row - 5 minutes;
3 times in a row - 10 minutes;