Preheat oven to 350 degrees F.

Assemble topping by mixing butter, light brown sugar, oats, flour, and walnuts together in a bowl, and cream together using a fork to form large crumbles. Set aside.

In a large bowl, cream together the butter and white sugar with a hand mixer. Add light brown sugar, vanilla, eggs, and pumpkin puree. Mix until incorporated.

In a separate bowl, sift together flour, baking soda, baking powder, salt, and spices.

With mixer on low speed, add half of the dry mix to the wet ingredients and mix until incorporated. Continue by adding the remainder of the dry ingredients and the milk to the batter, mix together until the batter is smooth (be careful not to over mix).

Pour batter into a greased loaf pan and sprinkle with streusel topping.

Place on middle rack of oven and bake for 65 minutes or until a toothpick comes out clean. Remove bread from oven and allow to cool completely before serving.