Preheat the oven to 350 degrees F (175 degrees C). Grease a 9x5-inch loaf pan with cooking spray.

Mix whole wheat flour, cornmeal, baking powder, cinnamon, salt, nutmeg, and allspice together in a bowl.

Stir sugar and yogurt together in another bowl. Beat in eggs, followed by avocados. Fold in flour mixture until batter is just moistened. Stir in milk and poppy seeds. Pour batter into the prepared loaf pan.

Bake in the preheated oven until a toothpick inserted into the loaf comes out clean, about 50 minutes. Cool for 10 minutes before serving.