Preheat oven to 425 degrees F (220 degrees C). Grease a 9-inch deep-dish pie plate.
Combine apple peels, white sugar, brown sugar, water, brandy, cinnamon, and vanilla extract in a saucepan; bring to a boil. Reduce temperature and simmer until fragrant, about 5 minutes. Remove apple peels with a fork.
Roll 1 crust into a 12-inch circle; place in the prepared pie plate. Roll out second crust into a 10-inch circle. Cut a few slits in the center to vent steam.
Mix corn muffin mix and milk in a bowl. Pour into pie plate.
Bake in the preheated oven for 5 to 8 minutes. Remove from oven. Layer apple slices and nectarine slices on top. Pour sugar mixture slowly over fruit; cover with second crust. Press crusts together to seal; flute edges.
Bake in the preheated oven for 10 minutes. Reduce heat to 350 degrees F (175 degrees C). Continue baking until top crust is golden brown, 35 to 45 minutes. Let cool to room temperature before slicing, about 1 hour.