Combine hot cocoa mix, 1/2 cup butter, and eggs in a large bowl; beat with an electric mixer until creamy. Add flour, baking powder, and salt gradually; mix well. Drop teaspoonfuls of the dough onto ungreased baking sheets.
Bake in the preheated oven until edges are dry, about 8 minutes. Let cool on the baking sheets for 5 minutes. Transfer to wire racks to cool completely.
Mix confectioners' sugar and milk together in a small bowl.
Combine marshmallows and 1 tablespoon butter in a microwave-safe bowl. Microwave until marshmallows melt, about 30 seconds. Stir into confectioners' sugar mixture with a fork until smooth.
Drizzle marshmallow glaze over cooled cookies. Let stand until set, about 15 minutes.