I would schedule at least a few hour layover in Athens. In my admittedly limited travel, the airports in Greece were some of the more stressful I've encountered. They were crowded, WAY too little seating, long lines at security, PITA rules about carry ons (restrictions on weight), etc. I can't remember where we were, but at one airport the check in desk employee had me shuffle around some carry on contents and then ended up fudging the numbers anyway, and then had me go to a different desk to pay for an additional bag since she had no way to charge me. They almost always tried to pressure us into checking carryons, saying they'd check them for free.
I'm pretty sure everyone else on our ABD flew out the morning the tour ended while we headed to Mykonos. We had to fly from Crete to Athens to Mykonos, and let them check a couple carry ons. When we arrived in Mykonos, one of our bags didn't make it. We waited in line with about 20 other people to file missing bag claims, and we were all told that our bags were left in Athens due to weight (despite being assured our bags would make it). When we left Mykonos, the check in person tried to get us to check our carry ons again but I said no thank you because of the previously lost bag. The check in employee said "they would never leave behind bags" like that (as if I were lying).