The 22-year-old, who spent the season with Bristol City has a wealth of experience in England's second-tier, could be set to leave Stamford Bridge this summer.
The Daily Mail report that the club are ready to let Palmer leave permanently to give him a permanent home instead of continuing to head out on loan.
They are ready to let him go for a free of £3.5 million, which appears to be a bargain following Lewis Baker's £10 million option to buy fee which was revealed earlier on this week.
Swansea City remain interested in the attacking midfielder, and this is what manager Steve Cooper had to say on the situation: "Kasey has been out in Japan with Chelsea and he's played some minutes for them.
"There are lots of players we have let our interest known on. It's difficult to make massive progress when the big Premier League clubs are away on pre-season tours.
"They're back in England now and hopefully that might help with the players we're looking at. Ideally you want to get players in as quickly as possible, but it's never easy as I'm finding out."
Palmer travelled with the Blues to Ireland, but is now expected to head out the door this summer if a fee can be agreed upon.