In a medium bowl, beat the softened butter and granulated sugar with an electric mixer on medium-high until pale and creamy, about 3 minutes. Add the egg yolk and vanilla and beat until completely combined.

Sift the all-purpose flour, cornstarch, baking powder, pumpkin pie spice, and salt over the butter mixture. Mix on low speed just until a crumbly dough begins to form.

Add the milk 1 teaspoon at a time, mixing until the dough comes together into a ball and pulls away from the sides of the bowl. It should feel dry but slightly sticky. Add only as much milk as needed.

Shape the dough into a ball and set it on a sheet of parchment paper that fits a baking sheet. Roll the dough to just under 1/4 inch thick. Slide the parchment onto the baking sheet, cover with plastic wrap, and refrigerate for at least 2 hours or up to 2 days.

Preheat the oven to 350°F and position an oven rack in the center.

Remove the chilled dough from the refrigerator and cut out shapes with a cookie cutter. Save the scraps for rerolling. Bake for 12 to 13 minutes, until the bottom edges turn light golden. For crisper cookies, bake 2 to 3 minutes longer.

While the first batch bakes, gather the dough scraps and roll them onto the second piece of parchment. Place the parchment on a baking sheet, cut out more cookies, cover loosely with plastic wrap, and refrigerate until ready to bake. Bake for 12 to 13 minutes.

Cool the cookies on the baking sheet for 10 minutes on a wire rack, then transfer them directly to the rack to cool completely.

In a small bowl, whisk together the powdered sugar and maple syrup until the sugar is moistened. Add the milk or water 1 teaspoon at a time, whisking after each addition, until the icing reaches a smooth piping or spreading consistency.