Preheat oven to 325 degrees F (165 degrees C). Grease 2 mini muffin tins.
Combine white sugar, butter, and cream cheese in a large bowl; beat with an electric mixer until light and fluffy. Add eggs one at a time, beating well after each addition. Beat in lemon juice, 2 teaspoons lemon extract, lemon zest, and vanilla extract. Add flour; beat until batter is just moistened.
Pour batter into muffin tins, filling each cup 2/3 full.
Bake in the preheated oven until a toothpick inserted near the center comes out clean, 10 to 15 minutes. Cool for 5 minutes. Transfer to wire racks to cool completely, about 10 minutes.
Whisk confectioners' sugar, 1/2 cup plus 3 tablespoons milk, and 3 1/2 teaspoons lemon extract together in a bowl to make glaze.
Dip tops of cakes in the glaze; place cakes on the wire racks until glaze is set, at least 1 hour.