To make the brownies: Preheat the oven to 350°F. Spray an 8-inch square baking dish and a 9 x 13-inch baking dish with nonstick cooking spray.
Whisk the flour, cocoa, baking powder, salt, and baking soda in a bowl to combine.
Stir the melted butter and brown sugar in a large bowl to combine. Stir in the eggs. Stir in the red food coloring and vanilla. Stir in the dry ingredients. Pour 1 cup of the batter into the 8-inch square baking dish and the remaining batter into the 9 x 13-inch baking dish.
Bake until a tester inserted in the center of the brownies comes out clean, 20 to 23 minutes for the smaller pan and 40 to 45 minutes for the larger pan. Cool completely on racks. Remove the brownies from the smaller pan and cut into ½-inch cubes. Set aside.
To make the topping: Scatter the marshmallows over the top of the cooled brownies in the larger pan.
Heat the butter, cocoa, and milk in a medium saucepan over medium heat, stirring, until the butter is melted. Whisk in the confectioners’ sugar, and then stir in the vanilla. Pour the frosting evenly over the marshmallows. Sprinkle the reserved brownie cubes over the top.
Allow the topping to cool completely before cutting the brownies into 20 pieces. Store in an airtight container for up to 3 days.