Combine the butter, sugar, egg yolks, lemon or orange zest, nutmeg and salt. Beat until light and fluffy.
Sift the flour, then add it and the poppy seeds to the butter mixture. Mix until just combined.
Divide dough into 4 equal parts. Form each piece into a 2 inch diameter cylinder and wrap in plastic and refrigerate for 2 hours or until firm. At this point you can freeze the dough for later use.
Preheat oven to 350 degrees F (175 degrees C).
Cut dough cylinders into 1/4 inch thick slices and place on ungreased baking sheets. Bake at 350 degrees F (175 degrees C) for 9 minutes. Let cookies cool on sheets for 2 minutes, then transfer to wire racks to cool. Once cool, store in an airtight container.