Combine 5 cups water, sugar and ginger in heavy large saucepan. Bring to boil, stirring until sugar dissolves. Reduce heat and simmer 10 minutes. Strain liquid into large bowl; discard solids.
Return liquid to same saucepan. Add lemon and lime peels. Boil 2 minutes. Remove from heat. Whisk in lemon and lime juices. Cool completely. Pour mixture into 13x9x2-inch glass baking dish. Cover and freeze until solid, about 6 hours or overnight.
Transfer mixture to processor and puree until smooth. Return to same glass dish; cover and freeze until solid, at least 3 hours or overnight. (Can be prepared 3 days ahead. Keep frozen.) Scoop sorbet into glasses or bowls. Garnish with mint.
