Religious convictions and moral panic aside, I think people tend to oppose same-sex marriage simply because the idea of it bothers them (for whatever reason). This is why the arguments I've most often heard against it - the slippery slope, the "institution" of marriage, etc - tend to lack any real substance. They are, as I said, excuses people use so they can argue against it without sounding like homophobes.
Of course, there's nothing wrong with not liking something. Just don't go out of your way to deny others the right to it, especially when it doesn't actually affect you.
