My experience has been on the 3 day weekends that the Fort clears out by noon on Monday and Tuesdays are back to normal or less. I would definitely do the Sat-Tues if you can. You likely won't have much choice in loop arriving on Saturday, but you will still be at the Fort.
j