In a saucepan simmer milk, peanut butter, and 1/2 cup sugar over moderate heat, stirring, until smooth, about 1 minute. Stir in vanilla and cool mixture to room temperature. Freeze mixture in an ice-cream maker according to manufacturer's instructions.
While ice cream is freezing, in a small saucepan simmer water and remaining 3 tablespoons sugar, stirring, until sugar dissolves. Add chocolate, stirring until smooth, and cool sauce to room temperature.
Spread one fourth ice cream in a 2-quart container and top with one third sauce. Layer remaining ice cream and sauce in same manner and with a spatula swirl sauce and ice cream together. Freeze ice cream, covered with plastic wrap, until hard.
