I was playing with your mod set up and my visitors are not leaving my map.
what do you mean by 'visitors'? guests added by hospitality, vanilla visitors or traders? I think this might be important but do they even behave differently? idk, sorry
is CoC the only mod that you have installed? I've experienced this issue before, it had nothing to do with CoC in my case... simply because I never played the CoC mods.
released prisoners would wander my colony's fields and eat raw rice right off the ground, and leave eventually. if I released too many at once, they'd all get stuck, if I did it in small groups or one by one, they were fine but it took them maybe a day or two to realise that they're free

rescued spacers' behaviour was identical btw. maybe this will help, you should try that. if you do have that issue.
as for visitors,
traders would get stuck in perpetual wait mode, with one senile cirrhosic caravaneer going to sleep for 12h+, another bloodthirsty girl attacking small game all around her,
BUT combat friendlies and Hospitality guests always worked like a clock for me, they never exhibited behaviour that could even remotely be described as weird or bugged
tl;dr from my experience having too many mods that affect non-colonist pawns' behavior
will bug out your game. this is something you should be prepared for as a mod user