Preheat oven to 425 degrees F (220 degrees C).
Thinly slice each potato several times, stopping just short of cutting all the way through.
Whisk olive oil, garlic, paprika, oregano, and celery salt together in a bowl; pour into a resealable plastic bag. Add the potatoes, seal the bag, and shake to coat completely. Transfer potatoes to a baking dish and reserve oil mixture in a small bowl. Sprinkle Parmesan cheese over the potatoes.
Bake potatoes in the preheated oven until layers start to separate and potatoes are slightly tender, about 30 minutes. Brush potatoes with reserved oil mixture, pulling layers apart if they are sticking together. Continue baking until potatoes are crispy on the edges and easily pierced with knife, 30 to 40 minutes more.
You might want to place each potato in a large serving spoon and use it as a guide for slicing.
For a richer flavor, leave the potatoes in the seasoned bag overnight.