Bring milk to a gentle simmer in a saucepan over medium-low heat. Remove from heat.

Beat peanut butter and sugar together in a bowl until creamy. Stir in hot milk slowly until evenly incorporated. Cool in the refrigerator for 1 hour.

Mix heavy cream and vanilla extract into the peanut butter mixture. Pour mixture into an ice cream maker and freeze according to manufacturer's instructions, about 20 minutes. Stir in maraschino cherries. Transfer to an airtight container and freeze until firm, about 4 hours.