That same logic can be applied to pretty much all contracts that are void for public policy, except for those applying to minors or invalids. Which is where this line of logic fails: if we legalize gay marriage contracts on the basis of “consenting adults should be able to enter into contracts” as an absolute maxim, then there is no reasonable means to object to contracts for prostitution, for polygamy, for one party to kill the other, for offensively usurious loans, for taking away vidya, performing trans surgeries, harvesting organs, etc.
Hell, depending on how far you want to push the line of logic, you can theoretically take it to require abolishing a fair number of criminal laws too.
And if we want to limit it to the domain of just “marriage,” then that requires admitting that marriage is something unique from all those other contracts, which undermines the premise that marriage is just another contractual relationship.
Just for context, I actually don’t disagree with the idea that many of the benefits of marriage, such as visitation rights, death benefits, etc should be freely contactable. But the line of logic that this should be the case because “consenting adults should be free to contract” is a poor one.