Multi-Threading Without Multi-Threading

[Encrtia]

So I have 50 colonists constantly looking for a job to do. Check this, check that, can I do this, can I do that. Even with only 1 job ticked for each one, Speed 1 is jerking. The moment they all go to bed though (except those pesky Night Owls) my game runs super smooth - 105625 cells, dozens of mods, Speed 3 'n' all! So that got me thinkin'... The only other time it runs this smooth, is when I have 'em all drafted, or very close to this smooth when I disable every single Job.

The cure must be, Multi-threading. No one can deny that. Regardless of whether the task is too much work for certain people's opinions of the gain to be had. (Ultimately, different play-styles...)

Regardless, instead of "what will get me shot down" - can we have something in-game, that can "occupy the attention" of selected (being most) colonists to alleviate this dilemma? Effectively shutting down all their internal chatter to simulate the lag-free effect of shoving them all in Cryptopods?

I wasn't contemplating ideas to the solution here, but a template for one... so just making some stuff up on the spot as an example...

Alpha Concept: A simple tickbox for "Socialize Only", that'll effectively shut down All Jobs + Job Searching, in favour of just "chatting with the locals". Doubles up as a lag-reliever late game, along with a lacking social facilitator outside of getting people to chop blocks adjacent to one another, forever.

Beta Concept: Two work modes: "Efficient" & "Relaxed". Early game, especially with few colonists, the Efficient Work mode replicates what we have now. But late game, when your colony is very competent that strict efficiency for every single colonist isn't really needed (the same as in real life), queue Relaxed Work mode. Checks for jobs are reduced by a large amount, arbitrary lengths of time between checks (in order to avoid every colonist check-spamming-at-the-same-time lag? - remember, work-efficiency isn't key here, lag-efficiency is ) & an otherwise increased focus on merely socializing. Again, this could be a simple Tickbox to alternate between the two to get by the lag late-game.
^ I quite like my "thought-up-on-the-spot" idea now... not only will it allow us "Multi-threading PLZ!" players the chance to squeeze more time out of the game before deciding that Multi-Threading is the Holy Grail, yet again, it's a nice roleplay concept that once the Colony has set up, they naturally become more relaxed. A couple Colonists will be dedicated workers full-time, but others have earned the right to take a step back. Get the farmers on Efficient mode, tick Efficient Mode when I want something done quicker & only really need to rush around when repelling invaders. I'll probably be playing on a 2nd map most of the time anyway (Caravans invading, setting up a 2nd colony, etc.).

Add-On Concept: Train mode (or combat train mode). Just add it in there so we can get some dedicated (through roleplaying) guards/soldiers. Disables all other job checks, & allows them to simply "Practice Skill" every day as their sole job, ready to repel/invade. Nice roleplay, gets them off the jobs ladder & personally, feels like a really nice touch to add as I currently feel like everyone is just a Civilian in Armour.

I'm not a coder, so hell if I know what I'm talking about. All I know, is that I want to just "carry on playing" instead of conceding to a force my Colonists can't even fight against xD (Multi-threading rules...)

[TeraRaptor]

Removed the needless quote of the lengthy OP.

This needs to be a feature, late game lag is unbearable.

[Calahan]

Is bumping allowed on this forum? If so, bump.

No, content free bumping in not allowed on this forum.

And it might be a good idea for you to read the forum rules at your earliest convenience to avoid infringing any of them again in the future. https://ludeon.com/forums/index.php?topic=122.0

[Bergil]

When cooks are preparing fine meals it'd be more efficient to have them pick up both vegetables AND meat instead of making two trips to the pantry and back.

Speaking of efficiency and saving on lag why not implement say... Work crews?

Hear me out. Players band together their pawns into a work crew and during the working hours they band together and form a single pawn in terms of job searching. Whilst only one pawn with the right skilles will be doing a specific task the other two pawns could act as mules, carrying extra resources for builders, cooks, artisans etc. Or even carting finished items off to storage and then going off to get the resources and drop them back off with the pawn doing the task.

When they're finished, they look for the next task.

Get what I'm saying? Not sure if that would help with lag or perhaps just be a useful way of playing.

[Sbilko]

To fix this, Tynan could simply make some simple, low-quality graphics and include lazyness into the game.

Or you could just schedule all their "Work" time as "Joy time," so that they do not work. Or, as a last option solution, you could just exile most of your colonists out, so that you have only a few colonists left and it doesn't lag.

[qsdoosix]

As a student in computer science. I think there can be someway to keep the efficiency and reduce the lag.
Now, the game is lag in late game because it is checking every work for every pawn. So if you have x pawn and have y possible works to do, you need to check x*y times. And when you have many workbenches and many pawns it will cost a lot of calculations to do so.

The idea come to my mind is
make a queue for every job (hauling, crafting, etc., basically a queue for each work in work tab), updated and do the slow stuff (checking every possible work and push them in the related queue) in a relatively long period. (say,update every in game hour, even it takes a full second to do that you still can get a smooth game).And when a pawn needs a job, he just pull the first one in the queue with highest priority. Which takes almost no time.

And for emergency situation, we need be able to specify a work to be "urgent" so next pawn available will go for it. Or directly order some one to do it immediately.
Also this can work for personal needs like food/entertainment/sleep, all these stats should be consider as a type of job with priority. For example, set 5 queues for personal need with priority 0(which is highest) 1 2 3 4. if someone have more than 1/3 hunger, then the "work" eating will be put to the queue with priority 4, so he will not look for food unless he has nothing to do. And with the hunger decreases, it will be move to higher priority queue on update, until he is so hungry that he will ignore all other job queue (even it has a priority 1, is still lower than priority 0) and seek for food.

In general.
Before, we do the calculation to look for a job x*y times for each update.
now we do it y times for each game hour and x times for each update.


Moderator's edit (Calahan) - Not sure why you made a new post that was a quote of this post, and then edited the new post to add an extra paragraph to the quote of this post. Which is a bizarre choice, especially when you could have chosen the far far simpler option of just editing this post to add the extra paragraph. Anyway, I've taken the extra paragraph from the quote, added it here, and deleted the quote post.

emm, i think i am using the modify option in the thread and not making a new one. maybe some misop. Sorry for the inconvinence

[Nightinggale]

The idea come to my mind is
make a queue for every job (hauling, crafting, etc., basically a queue for each work in work tab), updated and do the slow stuff (checking every possible work and push them in the related queue) in a relatively long period. (say,update every in game hour, even it takes a full second to do that you still can get a smooth game).And when a pawn needs a job, he just pull the first one in the queue with highest priority. Which takes almost no time.

My impression based on experience with turning jobs on and off in the work tab is that hauling and cleaning are the most demanding tasks for the CPU when there is nothing to do.

Take for instance cleaning. It cleans in the home zone (There is a mod to make a cleaning zone. That would be nice in vanilla). Imagine having a counter for the amount of dirt in the home zone. This means the first thing to check when trying to see if cleaning is possible is to return not possible if counter is 0. A counter for dirt inside each room (which is also in the cleaning zone) would be nice because particularly with mods the cleaner might not be able to get to all the dirt, which will really makes the game lag. If the counter is not 0, check counters for the rooms the pawn can access. Particularly stuff like making prisoners clean with the prison labor mod makes the game lag big time when the prison is clean.

A counter for items to be hauled could be good too. If you somehow end up with items, which the pawns tries to haul to stockpiles, but the items aren't accessible, then the game will lag big time.

[qsdoosix]

The idea come to my mind is
make a queue for every job (hauling, crafting, etc., basically a queue for each work in work tab), updated and do the slow stuff (checking every possible work and push them in the related queue) in a relatively long period. (say,update every in game hour, even it takes a full second to do that you still can get a smooth game).And when a pawn needs a job, he just pull the first one in the queue with highest priority. Which takes almost no time.

My impression based on experience with turning jobs on and off in the work tab is that hauling and cleaning are the most demanding tasks for the CPU when there is nothing to do.

Take for instance cleaning. It cleans in the home zone (There is a mod to make a cleaning zone. That would be nice in vanilla). Imagine having a counter for the amount of dirt in the home zone. This means the first thing to check when trying to see if cleaning is possible is to return not possible if counter is 0. A counter for dirt inside each room (which is also in the cleaning zone) would be nice because particularly with mods the cleaner might not be able to get to all the dirt, which will really makes the game lag. If the counter is not 0, check counters for the rooms the pawn can access. Particularly stuff like making prisoners clean with the prison labor mod makes the game lag big time when the prison is clean.

A counter for items to be hauled could be good too. If you somehow end up with items, which the pawns tries to haul to stockpiles, but the items aren't accessible, then the game will lag big time.

Yes because this has most "space" can be worked on.
You may have 20 workbenches or more in mid/late game. And when working on a workbench, pawns are mostly standing still and don't need to do path finding.
But you will have over 200 blocks need to be cleaned and 100 items need to be hauled at same time. And you need find a path for each pawn doing it each time.

So it is very obvious which one will take more time to do.

As i don't know how the code is implemented, I just guessing on what i noticed.
The game gets much slower (I can keep smooth with 3x speed when alone, but when trader comes it become laggy) when trader group comes. Even your pawns are not going to interact with them. Also as i'm using a zombieland mod, 120 zombies didn't slow my game at all. And animals doesn't seems have too many effect (i had a save with about 30 animals still running smooth)

So my guess is, the game is searching for all possible work, including talking to all possible others, for every pawns in your base (kinda, active). When the population in your base increases, the calculation needed increase as n square.

[Nightinggale]

Yes because this has most "space" can be worked on.
You may have 20 workbenches or more in mid/late game. And when working on a workbench, pawns are mostly standing still and don't need to do path finding.
But you will have over 200 blocks need to be cleaned and 100 items need to be hauled at same time. And you need find a path for each pawn doing it each time.

That's not a worst case scenario. Try using Fluffy's worktab. You can open each job category to see each job. The game will go through the jobs (not job categories) right to left (assuming same priority) and once it finds something to do, it will stop searching, meaning everything to the right will be ignored. The colonist then completes the task before searching again.

The worst case scenario is when there is nothing to do and it goes through everything. Not only is this by far the slowest, it also mean the colonist is without a job and will search again next tick. It's possible that there is a number of ticks between each search, but they are very frequent.

Getting your colony to work with a bunch of workbenches is not really an issue from a search for work perspective if they can find something to do and/or you disable crafting in the work menu for the colonists you don't want to work there.

[qsdoosix]

Yes because this has most "space" can be worked on.
You may have 20 workbenches or more in mid/late game. And when working on a workbench, pawns are mostly standing still and don't need to do path finding.
But you will have over 200 blocks need to be cleaned and 100 items need to be hauled at same time. And you need find a path for each pawn doing it each time.

That's not a worst case scenario. Try using Fluffy's worktab. You can open each job category to see each job. The game will go through the jobs (not job categories) right to left (assuming same priority) and once it finds something to do, it will stop searching, meaning everything to the right will be ignored. The colonist then completes the task before searching again.

The worst case scenario is when there is nothing to do and it goes through everything. Not only is this by far the slowest, it also mean the colonist is without a job and will search again next tick. It's possible that there is a number of ticks between each search, but they are very frequent.

Getting your colony to work with a bunch of workbenches is not really an issue from a search for work perspective if they can find something to do and/or you disable crafting in the work menu for the colonists you don't want to work there.

That's why i'm suggesting a queue of works.
Search all works for every long time (like a in game hour) and push them to different queue by job type (like, crafting, planting, .etc), this can be done desynchronized, in other word, this can be done by another CPU thread. Then when a pawn needs a work, they just go to the queue with highest priority, pull one and then check if fits the requirement (like skill or accessibility), if it fits, then go for it, if doesn't, push this to a temporary queue and pull another one, until reach the end of queue or find one, then push the temporary queue back to the work queue.

By doing so you reuse the search result made before, and somehow keeps the manner that a work "created" earlier gets done earlier.

[mistomaxo]

you have 50 colonists and you want ways to turn off portions of their brains so they arent gobbling cpu power, is that it?

have you considered... allowing in/rescuing/capturing fewer colonists? you can work wonders with some turrets to help in defense