While certainly, you’re correct that neither restriction is logically more obvious or less arbitrary than the other, in legal terms, one is significantly easier than the other. Allowing same-sex marriage only requires dropping the requirement that the two participants be of opposite sexes. All of the law surrounding marriage assumes two people, and in the more difficult cases like death of one person, or dissolution of the marriage, there is no logical difference whether there’s a man and a woman, or two women.
However, if you extend the law to more than two, things get complicated. Some things are obvious, at least in this day and age – all parties to a marriage would have to give consent (but would a highly-patriarchal society like the polygamist Mormons actually require wives to consent to a new wife?), But other things are less so. What happens when one person wants to leave a plural marriage? What happens when a marriage of 4 or more splits into two groups? Is there still a presumption of paternity? How does one write income tax schedules for marriages of 3, 4, or n?
Plural marriage requires same-sex marriage, because any marriage of n>2
will have at least two people of the same sex involved, but same-sex marriage does not automatically imply plural marriage. While it does seem to be the politically logical next step, there is strong antipathy to polyamory among some segments of the gay/lesbian community, so that next step may be a long time in coming.