Preheat oven to 350 degrees F (175 degrees C). Grease and flour a 9-inch square baking dish.
Whisk flour, graham cracker crumbs, brown sugar, and baking powder together in a large bowl. Mix banana, eggs, butter, and vanilla extract together in a separate bowl; stir into flour mixture. Stir milk into flour-banana mixture until batter is smooth. Pour batter into prepared dish.
Bake in the preheated oven until a toothpick inserted in the middle comes out clean, 30 to 40 minutes.