Heat milk in a saucepan until just boiling and remove from heat; add tea bag and let sit until tea has steeped, 2 to 3 minutes. Remove tea bag.

Whisk sugar, eggs, egg yolks, and salt together in a bowl. Pour 1/3 the milk mixture into the egg mixture, whisking constantly, until combined. Pour egg-milk mixture back into saucepan with milk. Cook over medium-low heat, stirring often, until custard thickens and easily coats the back of a spoon, about 10 minutes. Well defined trails should appear when you drag your finger across the back of a coated spoon.

Stir butter into custard until melted; pour into serving dishes or ramekins. Chill custard in the refrigerator, at least 1 hour.