Combine 1/2 cup milk, almond bark, sugar, 2 marshmallows, and vanilla extract in a saucepan over medium heat. Cook, stirring often, until boiling. Reduce heat; add remaining 3 1/2 cups milk and stir until heated through, about 5 minutes. Add remaining 4 marshmallows.
Divide hot chocolate among four 6-ounce mugs, placing 1 marshmallow in each. Garnish with red and green sprinkles.