Preheat oven to 325 degrees F (165 degrees C). Grease a fluted tube pan (such as Bundt®) generously.

Measure 5 tablespoons milk into a microwave-safe bowl. Heat in the microwave until warmed through, 30 to 45 seconds. Add lavender buds. Let steep to release their flavor, 10 to 15 minutes.

Whisk white sugar and butter together in a large bowl until creamy. Whisk in eggs, 1 at a time, whisking well after each addition. Add 1/2 cup lemon juice and lemon zest; mix until well-blended. Stir milk and lavender buds into the batter.

Sift cake flour and baking powder together in a bowl. Slowly fold into the batter, stirring well to prevent lumps. Pour into the greased tube pan.

Bake in the preheated oven until a toothpick inserted into the center of the cake comes out clean, 45 to 55 minutes. Cool in the pan for 5 minutes. Invert onto a wire rack to cool completely, at least 30 minutes.

Whisk remaining 5 tablespoons milk, 1/4 cup lemon juice, and confectioners' sugar together in a small bowl to make glaze; drizzle over the cooled cake. Let stand until glaze is set, about 30 minutes.