Preheat oven to 375 degrees F (190 degrees C). Grease a 12-cup muffin tin or line cups with paper liners.

Combine 9 tablespoons butter, white sugar, and vanilla extract in a large bowl; beat with an electric mixer until smooth and creamy. Add eggs one at a time and beat until well combined. Fold in half the flour and 1/4 cup milk and stir to combine. Repeat with remaining flour and milk. Spoon batter into the prepared muffin cups, filling each 3/4 full.

Bake in the preheated oven until a toothpick inserted in the center comes out clean, about 18 minutes.

Remove tin from oven and cool on a wire rack for 5 minutes. Remove cupcakes from the tin and cool completely on a wire rack before decorating, about 1 hour.

Combine confections' sugar, 1 tablespoon butter, and 1 tablespoon milk in a large bowl; beat with an electric mixer until smooth and creamy. Thin frosting with a little milk or water, if necessary. Add food coloring and stir until frosting is uniformly brown. Add more food coloring, 1 drop at a time, until frosting is the color you like.

Spread a thin layer of frosting on top of each cupcake. Break vanilla cookies in half. Place a cookie half on each cupcake as a mouth. Use a bit of frosting to "glue" a jelly bean onto the cookie for Rudolph's red nose. Stick 2 candy eyeballs into each cupcake. Break off a piece from each pretzel so they look like antlers and stick 2 into the top of each cupcakes. Shape chewy candies into ears and stick 2 into the sides of each cupcake.