I'm not entirely sure, but I think it really is just a random chance. It seems to happen more often when you melee raiders, but I think using a weapon that doesn't do a ton of damage might work too. Another way to acquire colonists is to trap the raiders until they give up and randomly start to "wander" (there'll be a message), or they collapse from starvation and you can capture them then.
But yeah, reloading works. I know for a fact that you can target someone when you're near them, pause, save, and just reload if you kill them. Eventually they get incapacitated and you can capture them. I'd give them a few shots so their hp was around 40-50 and then melee the rest and reload a ton. I haven't had this tactic work when the raiders loyalty has been over20% 30% though, and it seems like every time you hit them their loyalty increases. It's a little tedious so I wouldn't recommend actually doing it while playing through unless you really want some immediate new colonists. I wouldn't mind knowing if there were exact conditions that had to be met or something too.
But yeah, reloading works. I know for a fact that you can target someone when you're near them, pause, save, and just reload if you kill them. Eventually they get incapacitated and you can capture them. I'd give them a few shots so their hp was around 40-50 and then melee the rest and reload a ton. I haven't had this tactic work when the raiders loyalty has been over