Preheat the oven to 350 degrees F (175 degrees C). Grease 9 muffin cups or line cups with paper liners.

Whisk sugar, lemon juice, oil, lemon zest, and lemon extract together in a bowl until evenly blended. Add coconut milk and stir until well combined. Sift flour, baking powder, baking soda, and salt together in a separate bowl. Stir into sugar mixture. Fold in mashed banana and poppy seeds.

Spoon batter into the prepared muffin cups, filling each 3/4 full.

Bake in the preheated oven until tops spring back when lightly pressed and a toothpick inserted into the center comes out clean, 20 to 22 minutes. Cool in the tin for 5 minutes. Transfer to a wire rack to cool completely, at least 25 minutes.

Mix confectioners' sugar and 1 tablespoon lemon juice together in a bowl. Brush glaze on top of the cooled muffins.