Is it ok to just guess my water pump is the culprit of leaking antifreeze and replace?

It's not a bad choice and the price is right. If the water pump is the original this is a good time to replace it anyway, and if it fixes the leak it is totally worth it. If not, at least you have a new pump and can rule the pump out as the source of the leak.