Stir water, sugar, ginger, lemon zest, and lime zest together in a saucepan over medium-high heat until sugar dissolves and mixture is bubbling, 5 to 10 minutes; cool slightly.

Stir the ginger mixture an orange juice together in a pitcher; pour into a 9x13-inch baking dish.

Place dish in the freezer; scrape and stir orange juice mixture with a fork once every hour until granita is evenly frozen and crystallized, about 6 hours.