Preheat oven to 350 degrees F (175 degrees C). Line a baking sheet with parchment paper.

Beat unsalted butter, 3/4 cup confectioners' sugar, and 2 1/2 teaspoons vanilla extract in a bowl until smooth and creamy. Sift flour and salt together in a separate bowl; beat flour mixture into butter mixture to make a workable dough. If dough feels crumbly or dry, beat 1 teaspoon evaporated milk into dough, or more if needed.

Pinch off about 1 tablespoon of dough and form into a ball; push an indentation in the ball with your finger, insert a gumdrop, and wrap dough around gumdrop to completely enclose. Place cookie onto prepared baking sheet with the sealed side down. Repeat with remaining dough and gumdrops.

Bake in the preheated oven until cookies are set but not brown, 12 to 14 minutes.

Mix 1 cup confectioners' sugar, 2 tablespoons evaporated milk, and 1 teaspoon vanilla extract in a bowl to make a smooth frosting. Divide frosting into 3 or 4 small bowls and color each a different color, if desired. Spread frosting on cookies while they are still warm; set aside to finish cooling before serving.