Preheat oven to 350°F. Line 2 cookie sheets with parchment or silpat baking mats.
Cream butter, peanut butter, and both sugars in the bowl of an electric mixer fitted with the paddle attachment. Add egg, vanilla, milk, and salt and baking soda. Mix until combined. Slowly add flour and mix until dough comes together.
Scoop 1 tablespoon sized balls of dough and place them on prepared cookie sheets, 2 inches apart. Gently press each ball with the palm of your hand to flatten. Using a spoon, make marks to resemble a basketball, if desired. (You can also make the criss cross marks with a fork instead.)
Bake for 8-10 minutes, until the bottoms begin to look brown. Cool at least 5 minutes on cookie sheets before removing to a rack to cool completely.
Assemble the s’mores: turn cookies upside down on a cookie sheet. Place large marshmallow halves on half the cookies and a peanut butter cup on the other half. Broil until the marshmallow is toasty and peanut butter cup is melty, under a minute. Make sandwiches and serve warm.
Note: You can also toast the marshmallows the old fashioned way: over an open fire!