Beat milk and pudding mix together in a bowl using an electric mixer until thickened, about 2 minutes. Measure 1 1/2 cup pudding and set remaining pudding aside for another use.
Sprinkle graham cracker crumbs into the bottom of each ice pop mold; add about 1/4 cup ice cream. Sprinkle another layer of graham cracker crumbs and top with about 1/4 cup pudding. Add a final layer of graham cracker crumbs. Freeze until completely frozen, about 3 hours.