Preheat oven to 375 degrees F (190 degrees C). Line 2 baking sheets with parchment paper.

Stir cake mix, eggs, oil, and vanilla extract together in a bowl. Shape dough into 1-inch balls. Arrange 2 inches apart on the prepared baking sheets. Coat a glass lightly with flour; flatten each dough ball into a cookie.

Bake in the preheated oven until cookies are slightly golden at the edges, 6 to 8 minutes. Transfer to wire racks to cool completely.

Place ice cream in the bowl of a stand mixer fitted with the paddle attachment. Blend on low speed until uniformly softened, 10 to 15 seconds. Add marshmallows; blend until incorporated.

Line a rimmed 9x13-inch baking sheet with aluminum foil. Spread ice cream on top. Freeze until firm, 5 hours to overnight.

Line a baking sheet with parchment paper. Spread 1 to 2 teaspoons frosting on the bottom of each cookie. Place 12 cookies frosting side-up on the baking sheet.

Remove ice cream from the freezer. Cut ice cream into rounds using a 2-inch cookie cutter. Place ice cream rounds onto the cookies using a spatula or by pressing them out of the cookie cutter with a spoon. Press another cookie on top, frosting-side down.

Place baking sheet in the freezer until ice cream cookie sandwiches are firm, 8 hours to overnight.

Pour sprinkles into a bowl. Roll each ice cream sandwich in sprinkles. Place 1 sandwich into each sandwich bag; fold the edge and crease to seal. Freeze for 1 to 2 hours.

Peel down bag as you eat, keeping the bag underneath to catch drips.