We stayed at POR in Building 85, and in 14 days of schlepping around on the buses, we only had problems twice - once where the phantom bus just never showed up on the way to DHS, and once when buses back from MK were just packed. In the first instance we just got an Uber. In the second, we jumped on a bus back to POFQ and walked up. Even though we were on the 'furthest' end of the resort from POFQ, it was still a very pleasant walk back.
As for buses, we always found it easiest to take buses to the park from the East Depot, right behind the Mansion buildings. On the way back, however, we'd get off at the South Depot and walk through - by the time it had gone South > West > North > East we'd usually be in our room...
HTH.