Ludeon Forums

RimWorld => Support => Topic started by: RayZoar-Z on July 10, 2019, 06:05:10 PM

Title: Small freezes / fps drops with increased frequency when speeding up game
Post by: RayZoar-Z on July 10, 2019, 06:05:10 PM
I don't remember when this started, but, starting immediately when my colonists spawn (and every time I start a new game), there's short freezes (less than half a second) with vanilla gameplay. At speed 3 they happen every 60 seconds, at speed 2 every 105 seconds, at speed 1 every 130 seconds and at speed 0 every 165 seconds. I can also see my fps drop in the main menu from 144 to somewhere around 130 every 105 seconds.

When I enable my mods: At speed 3 the freezes happen every 5 seconds, at speed 2 every 9 seconds, at speed 1 every 18 seconds and at speed 0 every 39 seconds.

There's no errors in the debug menu.
My gpu and cpu temperatures look fine.
GPU: Gigabyte GeForce RTX 2060 Gaming OC
CPU: Ryzen 5 2600

I tried halving the active mods over and over to see if one was a problem, but I think it's the number of mods that makes the freezes happen more frequently. I deleted all my Rimworld files and reïnstalled it. I verified the integrity of the game files. I deleted all pawns, animals and buildings on a map. I also started Rimworld on a different pc in the same house:

GPU: GTX 960
CPU: I7-4790

It was on the same steam profile with the same mods downloaded and it had the same freezes on vanilla and with mods. I don't know why the freezes happen, is it because I have too many mods downloaded (even when I don't have them active) or something with my steam account? It probably isn't hardware since it happens on both pc's right?

I would be ok with my fps lowering late-game because of all the mods, but I can't play 3 speed like I usually do now, because there's a freeze every 5 seconds.

[attachment deleted due to age]
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on July 11, 2019, 02:27:21 AM
Hi,
sorry there is nothing we can help you.
You use many mods, and serveral are real big ones.
The freezes comes because one or more mod's are calculating something.
Since no error's show up it is hard to impossible to tell which ones.

You can check the mod discussion of each mod about lag/freeze maybe you find something.
But i don't think the freezes don't appear right at a new colony, so you can't experiment with the modlist.

You can try a mod sorting tool, maybe it help's to arrange them it a different order but i wouldn't count on this.
https://ludeon.com/forums/index.php?topic=48518.0

Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: RayZoar-Z on July 11, 2019, 09:15:01 AM
Thanks for the reply and the tool, but as I said it happens with vanilla too. The mods only make the freezes happen even more. Or do you mean that I'm subscribed to too many mods from steam and they're causing the freezes even when they're not active (when I only enable 'core' in de mod list)?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on July 11, 2019, 10:26:13 AM
Sorry, that i overread the vanilla thing.
When it happen at plain vanilla gameplay i don't think it is a too many mod thing.

So it should have some other issues.
You should look more at your system. Any other tools that run at the background. Video capture,overlays.
Maybe update or downgrade the videodriver (i got the same like you and never got problems, but didn't update the drivers lately).

But when i look at the freezing interval you describe, rimworld is calculating something every 18 seconds at speed 1. But is is suspecious that it even freeze at speed 0.
And how you notice it freezes at speed 0 ?
That point stronger at the system then rimworld.
Long time ago i got overall freezes at my system that came from the mouse (don't ask my how/why), but with a new one they were gone.

Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: RayZoar-Z on July 19, 2019, 07:54:35 PM
After testing a lot of things, I found out that the memory usage displayed by the mod: RuntimeGC, shows it going up to around 950 Mb and then resetting to around 650 in a loop. When this happens, I get the freeze. (In speed 0 I notice it by panning with the camera which then stops moving for a while). The speed at which the memory usage goes from 650 to 950 are the intervals that I wrote in my first message. (so every 5 seconds with mods at speed 3). Also, when I click the 'Reclaim Memory' tool the same thing happens: It goes back to around 650 Mb and the freeze happens. Removing RuntimeGC doesn't help. Maybe this is useful: The memory usage in my taskbar stays around 2.8 GB.

How can I try to fix this? I don't think hardware is the problem since the same thing happens on a different pc I've tried it on (same mods and steam account). I'm only noticing freezes in Rimworld.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on July 20, 2019, 10:19:55 AM
Didn't you allready said, the same freezes happen on vanilla gameplay ?
So removing any Mod ofcouse don't help.

And when it happen without any mod's too and the majority of the player don't have similar freezes.
So the issue these problem is based on your system. Not the hardware but software.
Installed tools, driver.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: RayZoar-Z on July 20, 2019, 08:07:29 PM
I tried disabling all processes (also antivirus) besides Rimworld, but that doesn't fix it. I also checked my hardware again but that's all working fine. Then I found that some people had the same exact problem (by searching things like: 'rimworld freezing during garbage collection') and that it happened when Rimworld became 64-bit (B19). I think before that was also the last time I played (Rimworld ran fine then). Rolling back to B18 and A17 doesn't fix the freezes. Is there any way I could run Rimworld in 32-bit again to see if it works?  Or do you (or anyone else) know if and why 64-bit could be a problem with memory leaks or something?

With only Core and RuntimeGC (to see the memory usage) active, the memory usage is 1mb/sec at minimum and it keeps going from around 260 to around 350 (after a while from 270 to 360 etc.), meaning I get a freeze every 1.5 minutes. Also happens after clean install of Rimworld. Is this normal? If it isn't, my mods aren't the problem but they just make it worse right? And if it's also not my hardware or software (which I checked) and because the freezes also happen on a different pc I've tried it on, the problem is either in Rimworld, like the 64-bit thing or something with my steam account maybe?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on July 21, 2019, 02:15:14 AM
Maybe check what software you use on both PC's together. I still hardly believe there is the problem.
Currently you are the only one which report such issue.

If you use steam you can decide somewhere to use 32 or 64, like you can select to use unstable version.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: RayZoar-Z on July 27, 2019, 05:33:17 PM
I disabled all background processes (except steam because Rimworld would close and necessary windows processes) and nothing changes. Here is someone who might have the same problem (Frame drops in the main menu and when paused like me + more freezes when more mods, but for him the game speed doesn't make the freezes more frequent and for me it does a little bit). https://www.reddit.com/r/RimWorld/comments/cfnk31/amount_of_mods_causing_fps_drops_even_in_main/?utm_source=share&utm_medium=web2x

Something else that I found is that memory usage (displayed by runtimegc) goes from 1mb/s to 4mb/s when I open the work tab, to 12mb/s when I open the factions tab and to 15mb/s when I open the research tab (other tabs also change it). With no mods except runtimegc and after a clean install.
The memory usage then keeps going from 185mb to 230mb and resets with a freeze. (so a freeze every 3 seconds with research tab open on vanilla).

Also I can't find the 32-bit option, which I don't think exists anymore?
And could a steam reïnstall help?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on July 28, 2019, 03:09:31 AM
I wouldn't do a steam reinstall since that uninstall don't remove old files.
Just delete the rimworld installation folder and let steam repair/reinstall it.
You can delete the config folder too (if you don't find the saved data folder, there is a button at the option to open it with the windows explorer).
There is still a 32 bit version, but don't ask me where you can find it at steam. I think around the area you can select which version (1.0,B19,B18...) you want to play.

Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: dragonid1423 on August 13, 2019, 02:39:29 AM
I just want to check in with my experience, and I am having the exact same issue. I've tried reinstalling the game (including purging it from the computer and installing it on a new drive), which includes deleting all the config files straight from the folders as well. After reinstalling the game, and booting it again, still the same process has happened.
Exactly as Rayzoar's found, the game will stutter/freeze for about half a second, consistently, at the same time, in intervals.
I've looked through Dev-mode logs, Hugslib logs, and Verbose Logs, and by all accounts, the game is running fine. This appears to be an extremely rare issue that I can only put down as some form of Hardware issue.
For anyone who finds this that has this issue, or for Rayzoar, these are my exact specs, I'm sharing more than one would usually in attempt to find a single thing we have in common:
Motherboard: Strix Z27OE Gaming Motherboard
CPU: i7-7700k, 4.20HGz, x64-bit
GPU: GTX 960 (4GB VRAM)
RAM: 16GB DDR4
Disks: 7200RPM 1TB HDD, 1TB Sandisk SSD, 2TB Seagate Hybrid
OS: Windows 10 Enterprise Version 1803, Build 17134.829
Peripherals: j5create USB hub, Targus USB hub, Logitech C310 Webcam, Orion G910 Keyboard, Corsair Void Pro Wireless Headset, Logitech G502 Mouse. (Usually it'd be dumb to list peripherals, but Planetary Annihilation crashes with an Afterglow headset in, so you never know, right?)

Thanks

EDIT: How horrifying, we both use a GTX 960. I really hope that's not the issue.
EDIT2: I have a friend with a GTX 960, I gave him the same version I use, had him run it, and reproduced it immediately. If the issue stems from the GTX 960, we certainly need to contact Tynan.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on August 13, 2019, 05:45:08 AM
I would try to use different video driver at first.
Try to use the latest one, then try an older/much older one.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: dragonid1423 on August 13, 2019, 08:18:27 PM
Turns out I was on a very old driver in the 350s versions, and then I updated to 431.60, though, issues remained. I attempted to find old legacy drivers in attempt to see what I could do, but there's nothing before the 400s in drivers for the GTX 960 available to download.
Was worth a shot, I suppose.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on August 14, 2019, 01:43:40 AM
Yep it was.
Since you used a very old and a new driver, i don't think it is a driver issue.
It still can be an issue with the GTX 960 and Unity.
But it still could have some other issue.
Since you got someone you can compare it, what hardware/software/tools your friend and you got the same ? Maybe start to look there.
Long time ago i got an USB mouse which cause complete PC freezes too. Or when your HDD got a sleep mode, the wake up can do some strange things too, since window's wait for the HDD access.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: dragonid1423 on August 14, 2019, 02:28:05 AM
I asked, and it turns out we have the same keyboard, I am going to try and unplug it and use one of my backup keyboards instead...
Just checked, no cigar. He and I cross referenced our software on the keyboard as well, but we both use completely different driver dates. And of course, Rimworld only started having this problem recently, when I had the keyboard previously as well - granted, the same can be said for the GTX 960.
We're using different Screens, Cords, Motherboards, CPUs, Mice, Headsets, USB Hubs, etc. The only thing in common we have is Windows 10 64bit, GTX 960, and the G910 Orion Keyboard.
Haven't the foggiest on the source of the problem. He and I also went through the details list in Task Manager and compared background processes, we have little in common other than the big commons ones: Battle.Net, Steam, Malwarebytes, Nvidia, and of course all the Windows 10 things.
I feel like I'm on the end of the rope here, what else could possibly cause this?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on August 14, 2019, 04:45:42 AM
I am out if idea.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: dragonid1423 on August 14, 2019, 07:45:47 PM
I suppose the best we can do is say that the GTX 960 somehow is the source of this. It's probably a jump to assume, but I'm out of options.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: dragonid1423 on August 15, 2019, 05:52:28 PM
I tried poking around on reddit for some more info on this, and I found someone who was having the same issues with a GTX 1070, so the 960 is ruled out as the culprit. While that's good, it's certainly not great because now we know that the issue isn't caused by a known problem, which forces us to go back to the drawing board.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: dragonid1423 on August 18, 2019, 01:20:11 AM
For anyone finding this thread down the line: The fix is stupidly simple: Run Rimworld in Administrator Mode.

Thanks for helping everyone!
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on August 18, 2019, 05:55:36 AM
Simple fix, but still don't explain it why it works.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: jay124 on August 27, 2019, 11:59:53 PM
Sadly, I've been experiencing the same issue and running it as Admin did not help alleviate the issue. I was hopeful it would, but didn't. :(

It's the exact same thing this person was having.
https://www.reddit.com/r/RimWorld/comments/consfj/stuttering_every_couple_of_seconds_due_to_memory/

But in my case, it looks like it runs GC at 700MB and drops down to 500MB every 6 or so seconds.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: RayZoar-Z on August 31, 2019, 02:05:02 PM
Administrator mode also doesn't help for me. I gave up on finding a solution since my last post and haven't played since, but if anyone finds anything please post it.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on September 09, 2019, 02:54:42 AM
I've also been having this problem; it started around the 1.0 release last year.  I never could trace it down, but it becomes more obvious the more mods you add.  All I need to do to test it is start a new game and speed up time and watch a river freeze up and the FPS drop briefly every X seconds.  It happens in vanilla too, and it doesn't take many mods for it to become much more noticeable.  I reinstalled my OS entirely over the summer (finally upgraded from Win7 to Win10), have none of the same hardware as you (also a 1070GTX like the guy on reddit though) so I'm pretty sure it's a game thing.  From the descriptions in this thread, it sounds like an issue with the unity garbage collection mechanism. 

Edit: It's definitely the GC in game.  The RuntimeGC mod lets you watch it and the spikes correlate exactly with when the memory is automatically reclaimed.  I had a similar experience to RayZoar-Z with it.

The solution is likely a coding change to the base game.  One idea would be to prevent the GC from running automatically / as frequently and force it to run when a dialog is displayed instead.  I don't know if this kind of base level code change is possible via modding.

Alternatively, the issue could be caused by a memory leak, causing the GC to run frequently in an effort to try to keep up with the leak.  Again, a base game fix though.

Unfortunately, this issue makes the game super frustrating to play.  I haven't played it in months because of it, and was hoping to find that this had been figured out by now.

My best guess is that unity was updated to a newer version when Rimworld moved to 64bit (since that's when I first noticed the issue and started seeing other people post about it) and that this newer version somehow altered the way garbage collection works, or introduced a memory leak, though I could be incorrect.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: LWM on September 09, 2019, 08:22:28 AM
One other possible thing to try....

RimWorld does run on Linux.  You could get a (large?) USB drive and install a live version of some lightweight Linux on it, and see if you can fit Linux+Steam+RimWorld on it.  The garbage collection engine might behave differently.

I have read a little about the C# garbage collector, and it may be possible for someone to suggest a garbage collection on a faster/different schedule.  But I'm not 100% sure.

One other thing to consider: upgrade your system's memory, if it's low and you use a spinning HDD?

I sometimes get stuttering, but it's usually because I have 8 or 9 Firefox windows with 50-60 tabs open in the background :p  Closing those helps a LOT.  Good luck.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on September 09, 2019, 04:26:28 PM
Nothing else running, Rimworld's on an SSD and I have 32GB of DDR4 RAM.

I'm going to keep experimenting.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: LWM on September 09, 2019, 06:21:28 PM
Well, THAT should be promising.

One reason I was thinking Linux was b/c you would be (no choice) using a different set of hardware drivers...

--LWM
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on September 10, 2019, 07:16:36 PM
So I've tried a bunch of things, but nothing has really had any effect on it.

It's much more pronounced and faster to reproduce with mods running, so I've been running a dozen or so to make things easier for myself.  RuntimeGC is key though.

You can set up RuntimeGC to display the current .Net VM memory usage in the toolbar (and in the main menu).  In my case, it caps out around 560MB and then the garbage collection kicks in and it drops to 442MB. 

What's interesting is that this VM memory usage keeps going up, even in the menu or while the game is paused.  It goes up faster for each speed setting in-game, which is why the stutter happens more quickly at 3x than 1x.  What's unusual is that it is going up at all when things are paused and nothing is happening.  Even more so that it does it in the main menu.  This signals to me that there is some kind of leak happening somewhere -- some objects are getting constantly created and destroyed.

It would be interesting to see what other people see in the memory display in the RuntimeGC mod setting screen off the main menu. (Check 'Replace "RuntimeGC" with memory usage data' and set the update frequency to 'Realtime'.  Also uncheck "Replace/enhance vanilla GC" to view vanilla behaviour instead of the optimized collector in RuntimeGC.)   Is your memory usage also steadily increasing?

Edit: With no mods but RuntimeGC running with the settings changes from above, I can see the GC correlated stutter at 98MB before it drops back to 72MB in the main menu. 

I also tried unsubscribing from everything on steam and removing all manual mods, as I noticed they were still being referenced in output_log.txt, but it didn't make a difference.

I'm going to try it on a few other PCs and see what I get.

Edit 2:  3 other PCs exhibit the same behaviour.  The memory goes up continuously in the main menu from 72MB to 98MB before being reset.  When the reset occurs, the FPS counter drops 1-2 FPS on 60Hz monitors or between 10-20 FPS on 144Hz monitors.  This might be why only some people are noticing it.  It's much more noticeable at 144Hz than 60Hz.

I'm now poking through the source code for the RuntimeGC mod.  Since he was able to replace the GC with a more efficient one, I'm wondering if I can slightly modify it so that the upper limit / time before the reset can be increased to reduce the stutter, and/or what else I can do.

Edit 3: There doesn't appear to be anything obvious in there to control the timing of the GC unfortunately.  It looks like the setting to replace the vanilla GC just prevents the vanilla GC from doing any pawn cleanup, as it's already handled by the mod, but it still fires at the same time.

The good news is that it looks like we can just use the old System.GC object, since RunTimeGC is able to call System.GC.Collect().  I thought we were looking at some unity methods for GC, but since we can use the .Net object, we should (in theory) be able to reconfigure the GC settings via a mod.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: moyashii on September 11, 2019, 11:54:54 AM
I'm having the same issue as all of you. I've nearly gone insane from looking everywhere possible to solve it or at least see if anyone else had the issue. It doesn't even seem that rare, I've seen quite a few people report having the same issue except they never get "official" replies, or ever seem to get it fixed.

I'm beginning to wonder if everyone has this problem but the majority just doesn't notice it. It's so strange and I've already done every possible "fix" short of reformatting to no avail. Deleted the game, rebooted, deleted and unsubbed from all mods, deleted the ludeon folder and disabled steam's syncing, updated drivers, etc.

It honestly makes the game completely unplayable and not enjoyable for me, despite wanting to play it so badly. It stutters in vanilla as stated in this thread before, but it's a lot less pronounced. You can still clearly notice it though, even when you're distracted, when you're building things and panning your camera around for example, it's very noticeable and can cause you to mess up your construction plans often.

The worst part is that hardware seems to not be the culprit. Despite this, the few replies the posts claiming to have this issue get usually blame it on the specs or something on the individual's side, such as poor mod orders, when it's clearly more than that. Even with 10-20 mods the stuttering increases a massive amount, from the reasonable every 20-30 seconds in vanilla to every 6-12 seconds. Not even big mods like Psychology or CE either. Not that it matters, because big mods seem to affect the stuttering just as much as small ones, so long as you have more than 10-15 added. There are others with far more mods, and far worse specs that are able to run the game at a smooth 60fps all the way up to massive colonies, and claim to never have this sort of stuttering.

I've also used RuntimeGC, and noticed what's been stated multiple times about the memory problem. I wish I was able to simply remove all mods and not have this stuttering, or could blame it on something so that I'd be able to fix it by just getting rid of it and be on my merry way, but it isn't the case. It's clearly something I am, and most likely most users, unable to solve on their own. Running the game on administrator mode did nothing on my side too.

This is frustrating since the issue seems to only have appeared a year or two ago tops. I've played this game ever since the early alpha/Kickstarter days, and I've always been passionate about it. So it really saddens me that I'm unable to enjoy it because of something like this. I hope that a solution comes out of this, or at least for it to get some light.

Edit: Formatting and stuff.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on September 11, 2019, 02:49:21 PM
So while we might be able to do something using a new GC mod, I don't think it's a good solution. It doesn't really address the underlying problem - that something is creating lots of objects under the hood that need to be destroyed. Altering the GC would be a band-aid at best, and cause new issues at worst.

I have a hypothesis about what is happening.

When Rimworld moved from 32 bit to 64 bit, the unity engine was upgraded to 5.6.5f1.  My hypothesis is that there's a problem with that version of unity.

In the changelog for 5.6.6 of unity, I found 3 mentions of fixes to memory leaks. In managed code, a memory leak will be cleaned up by the GC, but can cause a performance issue, like we are seeing.

For reference -
https://unity3d.com/unity/whats-new/unity-5.6.6

There are likely more in further versions as well, though I haven't read past 5.6.6 yet. I'll do that if my hypothesis is proven.

So, to test this hypothesis, I propose the following experiment:

Install Rimworld B18 and the RunTimeGC mod for B18. Alter the settings of RunTimeGC as per my previous comment, and then observe the memory usage and FPS within the options screen in the main menu.  Note the version of unity in the output_log.txt file.

If the memory is stable, then we try B19.

IIRC B19 was when the 64bit switch was made, so B18 should be 32bit and on the old version of unity.

If it doesn't happen in B18, then we can make a couple of videos/gifs and send them to the powers that be showing them the difference, and request a unity version update to at least 5.6.6.

If it still happens in B18, then I'm going back to the drawing board...


EDIT: 

Ok, so first of all the B18 version of RuntimeGC doesn't have the memory bar in the options menu.  So I had to recompile the 1.0 version against the B18 unity & c# dlls to get that back.  You can find it here if you want to try it:

https://www.dropbox.com/s/idjk3p6cogn315l/RuntimeGC%201.0%20to%20B18.zip?dl=0

Note that it probably won't work in the game itself, as I had to comment out a whole bunch of features that aren't supported in B18, like battle logs and a bunch of the corpse removal stuff.

The memory usage in B18 _does_ still increase in the main menu, though there are differences.  For one, the memory goes from 22MB to 28MB before the GC kicks in.  Another difference is that on my 144Hz monitor, the game starts running at 137 FPS, not 144, and when the reset happens, it drops to 131 FPS.  So it's a 6 FPS difference, not 20.

While interesting, this doesn't really prove my hypothesis, as the memory still goes up in the menu in B18 too.

Unity version for B18 is 5.6.3p1.

I'm not sure what to try next...

What I'd love to try is recompiling Rimworld against Unity 5.6.6 or 5.6.7 and see if the issue persists, but that's not gonna happen.

EDIT 2:

So I've been playing around with things a bit more.  I discovered that I can swap out the UnityEngine dlls for the ones in Cities Skylines (5.6.6f2) and it still works.  No clue why, but it does, lol.  It should need a recompile or something.

Anyway, it didn't matter.  The stutter still happens.

After reverting back to 1.0 from B18, I somehow broke the steam FPS counter for Rimworld (and only Rimworld), so I started using Fraps, which indicates the FPS drop at 6-7 instead of the 20 or so I was seeing with the Steam counter.

I also starting playing with a mod called FrameRateControl.  This mod lets you force the framerate to 60 or whatever.  This reminded me of something that I forgot to note yesterday - on a 144Hz monitor, the memory usage increase will be much, much faster than on a 60Hz monitor.  For instance, at 144Hz, it goes up from 72-98 in 16 seconds (1MB/s).  At 60Hz, it's more like 0.1MB/s, so the GC starts its process nearly 10x later theoretically.  I say theoretically because I noticed the memory usage cycle is not linear.  It gets faster as it gets closer to the upper limit.

So I tried 300 FPS using FrameRateControl.  It's about 50% faster than 144 FPS.  In game is where it gets interesting...  It's about as noticeable as 144 FPS, but the speed of memory increase is the same regardless of game speed.

Conversely, if you force the FPS to 60 in game, the stutter happens less often, is linked to game speed and appears to be less noticeable, as it only happens every 40-60 seconds or so... though I haven't tried it yet with mods.

EDIT 3:

I added a dozen or so mods and tried it out again.  With this setup it's a noticeable ~1 second stutter at 60 FPS (drops to 54 in Fraps), and a ~1 second stutter at 309 FPS (drops to ~280 in Fraps; I don't know why the 300 FPS setting in FrameRateControl runs at 309).  However, the stutter occurs every 3-4 minutes at 60 FPS and every 1 minute at 300 FPS.

I just noticed that there's a "No Limit" option in FrameRateControl which makes it go up to ~1450 FPS, and then drop to 1200 FPS when the GC cleans up. 

I'm going to mess around with it some more.  30 FPS might be a workaround as the GC would probably only run every 10 mins or so... Nevermind it's not.  The memory builds up super fast anyway when you increase the speed of the game.  The stutter is noticeable regardless and the FPS doesn't have much of an impact on how long it takes for it to occur in-game. 

Btw, for reference my main PC:
Intel i7 4770k @ 3.5GHz
32GB RAM
1070 GTX
Samsung 850 Evo SSD
Win 10 Pro, recently installed from scratch a few months ago

This has been an interesting experiment, but I'm not really sure what to try next.  I can't prove one way or another that it's a unity problem or a game problem.  I can see that it happens on multiple PCs, but it's more pronounced for those with more mods and/or faster monitors, and it's definitely related to GC process.

Any ideas, folks?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Penchekrak on September 27, 2019, 03:57:42 AM
Probably, I have the same issue. My thread here with video demonstration: https://ludeon.com/forums/index.php?topic=49577
Since I have Linux systems on my work, I will try Gallium HUD to capture frametimes and check if issue is on Linux version too.

UPD: I was unable to run Gallium HUD, so no frametimes graphs. I have run Rimworld on Ubuntu system with Steam FPS counter and clearly saw the same issue - FPS drops from 60 to 56 in a loop.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on September 30, 2019, 06:29:47 AM
I have the same proble, guess not everyone is so sensitive like us for frameteime lags,

i use fps monitor and my frametime behaviour is exactly how people described them here

even on vanilla i have framespikes on a brand new colony?!



while in menue i can watch runtime gc net vm go to like 975 and my frametime goes to like 200 spiky from 16
while ingame i can watch runtime gc net vm go to like 1385 and my frametime goes skyhigh to like 650
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on September 30, 2019, 06:02:48 PM
attention pawns
check this

same fo 960 gtx i belive

https://www.nvidia.com/en-us/geforce/forums/geforce-graphics-cards/5/239928/gtx-1060-freezes-stutters/
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on September 30, 2019, 07:02:49 PM
Quote from: Kaelen Mosar on September 30, 2019, 06:02:48 PM
attention pawns
check this

same fo 960 gtx i belive

https://www.nvidia.com/en-us/geforce/forums/geforce-graphics-cards/5/239928/gtx-1060-freezes-stutters/

I have tested it on two PCs with 1070s, a laptop with a 765, and a PC with a 560 Ti, and there's no stuttering in any game except Rimworld.  All 4 systems stutter when the GC process kicks in.

Rolling Rimworld back to B18 fixes the stutter. 
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on September 30, 2019, 07:26:59 PM
Quote from: madd_mugsy on September 30, 2019, 07:02:49 PM
Quote from: Kaelen Mosar on September 30, 2019, 06:02:48 PM
attention pawns
check this

same fo 960 gtx i belive

https://www.nvidia.com/en-us/geforce/forums/geforce-graphics-cards/5/239928/gtx-1060-freezes-stutters/

I have tested it on two PCs with 1070s, a laptop with a 765, and a PC with a 560 Ti, and there's no stuttering in any game except Rimworld.  All 4 systems stutter when the GC process kicks in.

Rolling Rimworld back to B18 fixes the stutter.

oh wow this is kinda taking my hopes away :D
people kinda hostile if you point out that stutter - iam not sure, but maybe they do not even recognize their game stuttering as its really minimal but its annoying to me personally and b4 1.0 i had no stutter so..
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Jibbles on September 30, 2019, 08:42:55 PM
I don't have much to contribute. Just want to say I experience it too & jet.  Done all kinds of recommendations, performing clean install yadda yadda. Spent entirely too much time trying to solve it..

You're probably already doing it but make sure to run these tests on a save file for some consistency due to pawns and biomes. I've experienced these stutters since 1.0. Wouldn't doubt if there's a leak of some sort cause I have had some bad hitches when there's not much happening on screen. Hoping it would be fixed by now. I mainly just do tests when I booted up Rimworld since the release. Performance has made it unplayable for me in 1.0. 

It's noticeable on my 60hz monitor.  Not everyone is going to report it, and no bit of telling who all just puts up with frame drops these days. It could kick in with as little as 3 pawns, on one speed in a vanilla early game on my end. B18 performs significantly better for me both early and late game. The difference is hard to ignore.  I don't have performance issues with other games.

It's already noticeable but gets worse with mods. Enabling something like jecrells tools makes a big performance hit without other mods on my end. Had some other mods perform the same way but can't remember them. Sorry, I'm pretty much echoing everything.

If the engine was indeed upgraded to a different version in unity then it should be looked into.  Not trying to flame, but I've seen plenty of these threads pop up and not a response from Tynan or ludeon. Hopefully I just missed a response cause that doesn't make much sense to me.

Anyways, good luck figuring it out!


Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: k2ymg on October 01, 2019, 12:47:57 AM
If you use windows 7 and NVIDIA, try change display refresh rate to 59, and then 60.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Jennings on October 13, 2019, 01:45:09 AM
Quote from: Jibbles on September 30, 2019, 08:42:55 PM
I don't have much to contribute. Just want to say I experience it too & jet.  Done all kinds of recommendations, performing clean install yadda yadda. Spent entirely too much time trying to solve it..
Chiming in to say the exact same. I want to love this game so much, too, but I can't even play it when it's like this. Adding all the mods I want just makes the game stutter nearly constantly.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Elvenstar32 on October 13, 2019, 01:30:25 PM
Honestly just created this account to hopefully make the devs aware that this is a real problem.

With and without mods once the game reaches a certain amount of RAM used it stutters for a second while clearing some RAM.

It honestly completely ruins the game for me because playing without mods feels very empty by comparison to my full modlist setup and playing with the modlist makes the game stutter every 45ish seconds at x1 speed and every 15ish seconds at x3 speed.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on October 13, 2019, 09:19:26 PM
Hey sorry for arriving late on this one.

Just trying to get a feel for the consensus here: What are the reproduction steps for this issue? There's a ton of different information going around about RAM usages, different mod setups, and so on. I'm trying to figure out what the minimal steps are to see the problem.

If I follow these steps, will I see the issue?

1. Start RimWorld with no mods installed.
2. Start a new game with default settings, any biome, any map size.
3. Observe lag spikes every second or so.

Or is there something else you have to do?

Just as context, Unity uses a 'stop the world' garbage collector that does all its collection in a single frame. This means that the system will definitely have some spikiness to the frame time graph. We only consider these a problem if they're actually observable without a chart, though, since that's what affects players.

What could happen is if you're pushing the system near its limits - for example, playing with certain mods, or on very large maps, or on multiple home maps - these spikes are the first things you'd notice as being not smooth. But this isn't an issue of some particular bug or quirk - it's just the whole system being under strain (specifically memory pressure) and exhibiting this symptom as a response.

Thanks for any information.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on October 13, 2019, 09:38:22 PM
Thanks Tynan! I should have tweeted you sooner  :)

On my main PC (specs above, no other apps running), I can see it most obviously without any mods installed when I select a temperate biome with a river, all other settings default, including the scenario. Without mods, it isn't always immediately noticeable for me unless I centre the screen over the river and go to 4x speed (w/dev mode) active.  It's also more noticeable (for me at least) when my refresh rate is set to 144hz.  Others may have more extreme experiences without mods than I do.  IIRC, I usually notice the spike within 10 seconds.

If I add the GC mod, then I can set up so that it indicates the memory usage in the bottom toolbar. The lag coincides with the memory usage getting reset.

It becomes super noticeable once I've added a dozen or so mods, usually including HugsLib and JecsTools since so many mods require them.


Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on October 14, 2019, 12:02:04 AM
Yes, these are garbage collector spikes.  Their severity depends mostly on whether the code is allocating and releasing a lot of objects in a given time.

It's very easy to do this accidentally, and it takes some specific programming effort to avoid allocating. We use profiling tools to chase down excessive/unnecessary allocations and optimize them out.

What's insidious about allocation is that it doesn't create a performance problem immediately. Rather, it leaves garbage on the memory heap, and the problem only appears when the garbage collector has to go do its thing some time later. If there's too much garbage, the GC time gets long enough that the player perceives a hitch.

So basically if code is unoptimized in general, in ways that are not obvious, it'll produce GC hitches. But it won't produce these hitches when it runs; it'll produce them some time later. I think there's definitely some of this going on with these mods. So part of the solution will be simply having mod authors optimize their code to stop allocating and releasing so many objects.

The other aspect of this is frame rate. We designed the game to tick in 1/60 second intervals, accelerated to 3x speed. We never attempted to get it to be stable at unlimited frame rates, or using all of our dev tools (like 4x speed). So the vanilla game tries to keep GC times under 1/60 of a second at 3x speed. If you're running at extremely high frame rates or using dev tools, GC spikes will tend to pop out more, since the system has half (120Hz @3x) or even far less than half (144Hz @ 4x) of the designed time. This is an inherent problem with using a 'stop-the-world' garbage collector as used in Unity/C#.

The only games that can totally avoid this issue are games that don't use a garbage collector (i.e. Unreal engine games written in C++ with manual memory management), or if they are in a GC'd language, just never allocate anything during gameplay. We try to get as close as possible to never allocating, and we're pretty good at it, but Unity itself allocates and there was no need to incur the huge slowdown in features development requires to do that, in order to hit our 60fps target. And even if we never allocated, and in most frames we don't IIRC (though Unity does), mods still can and that'll cause this problem.

So to summarize, based on what I'm seeing here it looks like the game isn't broken in any particular way, so much as it's being pushed in ways that it was never designed to support, or there are mods which are generating memory pressure due to not being optimized appropriately.

Speaking generally (not just about RW), I do want to optimize our games' memory usage to alleviate such problems, and we've spent a lot of effort doing that to get the game to run as it does. This is a tradeoff against making features, though, so there's a constant need to balance between hitting technical targets and making gameplay features that people like - more stories, traits, characters, etc etc. 120Hz or 144Hz support aren't a given, they are distinct features with distinct benefits and costs, that must be evaluated against other features in terms of benefit and cost.

Most likely we'll stick to a 60fps spec, with 144Hz support not being a target. With the kind of tech we work with, and being an indie team, the sacrifices in core gameplay required to target such a spec don't seem worth it. This is doubly true considering the kind of game it is - not a fast twitch action game, but a play-pauseable story builder management sim.

We will definitely fix RimWorld if it is broken. I'd be happy to recieve more technical info about this to see if there really is an issue that hasn't been described in madd_mugsy's response. But we aren't planning on upgrading it to a whole new performance target (which may or may not even be feasible) because that would be a major feature add-on.

It might also be worth bringing attention to relevant modders if you notice GC spikes getting worse from specific mods. Because this is a sort of 'indirect' technical issue, where the cause isn't directly adjacent to the problem, it's not obvious what is causing the problem. But if you use a memory profiler it should become pretty obvious.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on October 14, 2019, 02:42:58 AM
Thanks Tynan.

I just wanted to clarify that my instructions in my previous response were the steps required to reproduce the issue immediately on my primary PC.

The issue still occurs at 60hz and 1/2/3x speed without mods, but it takes longer to become noticeable, at least on this PC. (It's been a while since I tested it... I'll run some tests this week.) While I agree that optimizing for 4x speed and 144hz isn't necessarily a good use of resources for the team, the issue is still happening within the 3x and 60fps target.

One other thing that I found interesting was that the lag doesn't occur in the Beta 18 build. Using the GC mod, I noticed that the memory footprint was a lot smaller in B18 and the GC didn't appear to build up as much before cleaning up.

Hopefully others will be able to chime in soon with their experiences and provide more data.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Pangaea on October 14, 2019, 09:01:28 AM
Figured I'd pipe in with my experience, which is a little different. I don't experience lags, at least not noticeable lags. There is a slight hickup when the game autosaves, and it can be quite noticeable when I watch the graphs (especially in long games). But in regular game play I can't say I suffer from lags. Well, except when you get 200 people raids and the game grinds to a halt for 5-10 seconds :D

I don't use many mods, though, basically just some QoL mods with few if any added items. So what Tynan writes appears to check out with my experience at least. And my computer isn't top notch any more. GTX 770 with 8 GB DDR3 RAM. It's on Linux Mint, though, which is lighter than the Windows monstrosities. No steam either, so that's another resource hog removed.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Elvenstar32 on October 14, 2019, 11:44:15 AM
The short stutter is present even with no mods installed at all. It does however take a fairly long time to happen (around 1min at x3 speed). It also feels slightly like it's lees obvious than when I have mods installed. It's definitely still visible without looking at graphs or anything but you kinda have to look for it.

It however happens way faster and in a more obvious way with mods (sometimes it even causes the audio to freeze). It's interesting to know that this is caused in part at least by the mods being poorly coded. This does lead to some disappointment in the quality of mods like JecsTools which is one of the biggest offenders of this problem despite being massively popular.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on October 14, 2019, 12:03:07 PM
Quote(sometimes it even causes the audio to freeze)
I got trouble with the audio too.
But this was because a harddisk got on autosleep mode, and the system tried to access it.
This cause the audio to freeze/loop for a few seconds until the system could access the files from that harddisk.

But i don't think that belong to these lagspikes.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Elvenstar32 on October 14, 2019, 01:18:23 PM
Quote from: Canute on October 14, 2019, 12:03:07 PM
Quote(sometimes it even causes the audio to freeze)
I got trouble with the audio too.
But this was because a harddisk got on autosleep mode, and the system tried to access it.
This cause the audio to freeze/loop for a few seconds until the system could access the files from that harddisk.

But i don't think that belong to these lagspikes.

Given that this only occurs during the stuttering and the stuttering only occurs when the GC is clearing itself, given that I have the game on an SSD and given that I have disabled the hard drive sleep thing in the windows powerplan I heavily doubt the 2 aren't linked in my case.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on October 14, 2019, 06:56:16 PM
So I've done a little bit of testing at 60Hz, 3x speed.  I noticed the lag spike for <1s every 55-60 seconds or so (approx 57 seconds usually), though there is also always a spike within 10 seconds of starting a new game.

I then added Apparello2 because it's a pretty big content mod with zero code, and while the memory usage was higher, the time to lag remained the same, as did the duration of the lag.

Then I tried running with just HugsLib, and it was basically the same as vanilla.

Then I tried running with HugsLib + JecsTools and it would lag every 19 seconds instead of every ~57 seconds.  The memory usage goes up very quickly with JecsTools, and I'd wager that it's exacerbating things quite a bit.  Has anyone mentioned this to Jecrell?

In these tests, I only ran the game for a several minutes each time, as I don't have a ton of spare time today.  Anecdotally, my past experience has been that the lag will become more pronounced (longer spikes) over time, though I don't have the data to back that up yet.

Also if anyone knows a good memory profiler that will work with Rimworld, please let me know.  RenderDoc was the one I saw recommended to use with Unity games, but I haven't had much luck with it so far.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Elvenstar32 on October 15, 2019, 11:01:41 AM
I have done more testing as well I can confirm everything madd_mugsy said.

Testing conditions were similar at 60hz and using x3 speed. However starting conditions are with 10 colonists instead of 3, just to have something of a midgame performance simulation.

On the hardware side of things, I have Rimworld on an M.2 SATA SSD, running the game on Windows 1903 64bit with an i7-9750h, a GTX1660ti and 16gb of DDR4 RAM @2666hz. No thermal throttling at any point.

So back to the results:

The spike is a lot shorter with vanilla or few mods but it ramps up as the number of mods increases. It however always stays a spike, the game does not freeze for several seconds in a row. Nevertheless the vanilla spike seems to be only a few microseconds whereas a fully modded spike seems to be closer to a full second.

Using RimworldGC to monitor RAM usage, vanilla rimworld RAM usage increases by 0.5MB per second. As you add mods this increases progressively and fairly linearly except for problem mods.

It seems like the bigger the RAM increase per second is big, the more frequent and the longer the spikes become.

There are however as madd_mugsy also mentioned some mods that increase the RAM usage per second a lot more than other mods.

So far I found 3 mods that are way over the line and are probably a big source of those issues and 3 mods that would need more testing but definitely use more than other mods on average

The 3 big ones were: "JecsTools", "What the hack" and "Achtung". Those all increase the RAM increase per second by at least 10 up to 30mb.

The 2 questionable ones were: "Giddy up" and all its modules, "Combat Enhanced" with all necessary patches that this mod always needs and the Humanoid Alien Race Framework (even without any additional factions installed). Those increase the RAM increase per second between 5 and 10mb.

As I have tested a wide variety of mods, here's the list of all mods I have currently enabled : https://pastebin.com/QYdHXwx9 (https://pastebin.com/QYdHXwx9)

I'm fairly confident that none of those mods are individually responsible for big spikes (power++ seems to be slightly more RAM hungry than average, using a whole mb per second on its own though). As it stands the RAM usage increases by 4 to 5mb per second for a spike every 50 seconds.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Ophaq on October 23, 2019, 07:32:00 PM
Hey all, I'm just going to put this out there and say that I also have been having this issue since around A18. I however, have over 250 mods installed. But my issue is probably the same thing, especially late game. Although I've been experiencing it in early game as well where the game will kind of hiccup for a split second regardless if I'm panning around the map or not. It gets way worse late game and I usually get sound stutters where the sound kind of "drags" like in a BSOD for half a second or second.

My GPU is an HD7970 and I run an i5 3570k OC'd to 4.2ghz.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Penchekrak on October 24, 2019, 04:30:00 AM
I think there is a silver bullet for that issue - disable garbage collector and trigger it to run before autosave. We always have a lag before autosaving, it will change nothing if it will be 1 sec more but make gameplay much more smooth.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: LWM on October 24, 2019, 12:53:30 PM
Quote from: Tynan on October 14, 2019, 12:02:04 AM
So basically if code is unoptimized in general, in ways that are not obvious, it'll produce GC hitches. But it won't produce these hitches when it runs; it'll produce them some time later. I think there's definitely some of this going on with these mods. So part of the solution will be simply having mod authors optimize their code to stop allocating and releasing so many objects.

For mod authors who are unversed in Unity programming or unfamiliar with C#, is there anything you can recommend in terms of practices to avoid, practices to follow, tools to use, perhaps reading about the problem that you can recommend?

For everyone reporting problems, there's also the question of memory pressure in general - what's going on in the background?  Many of us (including the dev apparently) don't ever see the distracting stutters, so this has got to be a very frustrating problem to work on.

One other question for Tynan: is it possible to force the GC to run more often - and might that help at all anyway?

Thanks!
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Ophaq on October 24, 2019, 01:57:44 PM
I also have an issue late game where the game speeds become slower late game. For example, speed 3 will look like speed 1 or 2. But I think that is just due to RimWorld being only a single core game since so much is going on, especially with over 250 mods.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Ophaq on October 24, 2019, 01:58:02 PM
Quote from: Penchekrak on October 24, 2019, 04:30:00 AM
I think there is a silver bullet for that issue - disable garbage collector and trigger it to run before autosave. We always have a lag before autosaving, it will change nothing if it will be 1 sec more but make gameplay much more smooth.
How do you do this?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tassatir on October 24, 2019, 07:05:38 PM
Dunno if this will help but I've had this issue recently and I've realized it's because my AI (Colony Manager mod) had its tasks set to check things too often. (Such as checking forestry and hunting among other things)
I had it set to check every hour IGT which was too much for my game to handle, so I cranked it back to every 8 hours and daily and it solved my issue. Hope this helps.

Edited to mention the specific mod, since I noticed it on your mods list and the symptoms seemed to match up with mine, since the faster the game runs the more often it can check tasks. I know it's a bit late but I hope it can help you.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Penchekrak on October 25, 2019, 06:24:55 AM
There is a fix in new version of unity - incremental garbage collector: https://www.youtube.com/watch?v=5Fks2NArDc0
Looks like that should solve that problem in Rimworld.
Tynan, may you please comment about that feature, please. Tested, worth it?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Pangaea on October 25, 2019, 08:55:02 PM
The video says that .NET 4 is a requirements, which I believe means this GC wouldn't work on Linux or Mac systems. Not good.

Since the base game runs well, however, do we know that this really is a GC issue, and not a modding issue?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Pressensaft on October 30, 2019, 08:07:29 AM
For me, the base game exhibits the same behavior, though. The lag isn't as prominent or as often but it´s there and when calling the GC with RuntimeGC you can reproduce the lag. That being said, it´s a lot harder to notice in vanilla as it only happens every 50-60secs. I played with B18 and the lag didn't appear, no matter the number of mods I used.
The issue is now in the game since around 1.0 and I´ve been looking for a fix since then, as the game is unenjoyable for me. There has to be a difference within these two versions that cause the lag.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 01, 2019, 09:16:31 PM
If there is code spewing garbage into the memory heap:

-- Running GC more often won't help for a variety of deeper technical reasons. Otherwise we'd just GC every frame.

-- Incremental GC would likely smooth out the spikes, but would just spread performance degradation more among all the different frames. This may be an improvement but I'm not sure I'd call it an acceptable solution. No way of knowing without testing.

This isn't a problem you're going to solve with a simple band-aid, it's just a matter of engineering code not to spew unnecessary garbage into the heap. It's a continuous process of profiling, optimization, and complex technical decision-making that lasts for an entire project. We did it and modders need to do it too to produce quality, performant code.

Without looking at code it's hard to say how mod authors can do better in terms of writing code specifically. However, the one critical thing they should try to do is profile their code so they can see where the problems are. (I believe there are some .NET profilers that could work, though I'm not an expert here since we use the internal profiler.)

Quote from: Pressensaft on October 30, 2019, 08:07:29 AM
For me, the base game exhibits the same behavior, though. The lag isn't as prominent or as often but it´s there and when calling the GC with RuntimeGC you can reproduce the lag. That being said, it´s a lot harder to notice in vanilla as it only happens every 50-60secs. I played with B18 and the lag didn't appear, no matter the number of mods I used.
The issue is now in the game since around 1.0 and I´ve been looking for a fix since then, as the game is unenjoyable for me. There has to be a difference within these two versions that cause the lag.

If you're using RuntimeGC, you're not using vanilla. If you're using RuntimeGC to induce performance problems deliberately, it's definitely not vanilla.

I'm not sure how it's working now, but we took a look at RuntimeGC a year or two ago and found that it didn't do anything useful; it's perfectly possible this mod is degrading your performance by unnecessarily checking and shuffling data. Not everything that 'looks' or 'feels' like an optimization actually is, and sometimes they're de-optimizations.

If RuntimeGC is forcing garbage collection more often (not sure it is, but if) it's almost certainly degrading performance. GC timing is an extremely complex subject because of the generational nature of the GC and all the non-obvious things it's doing internally to speed itself up. It does not work the way you would think, and the way it does work does not produce results the way you would think. This is a hyper-optimized system designed by people with massive expertise in their field backed by years of formal research, analysis, and performance data. The smartest thing to do is to leave it alone. This is a hard point to make without actually teaching a course on garbage collection and, cache lines, pipelines, branch prediction, etc. But it's just not a system that bear simplistic non-expert reasoning.

The core game of 1.0 is more optimized than B18, but a lot has changed around it in the ecosystem that could affect performance if you're using mods. There are more mods, bigger mods, more ambitious and complex mods. Modders are more numerous and may be less experienced on average than they were before.

---

Regarding core count, even if we made the vanilla game code use multiple cores it wouldn't help GC problems at all. The current situation is several seconds of smooth framerates with hitches at GC time. Moving code to multi-core just speeds up the already-smooth sections without helping the GC whatsoever. So you're solving a problem you don't have, without addressing a problem you do have.

Also note that even if we put a ton of the vanilla game code on multiple cores, it wouldn't mean that mods had to do the same thing. And if mods need more performance, mods already have everything they need to run their code on multiple cores. Even if vanilla ran on multiple cores, mods might not. Mods can still spew garbage on any number of cores, producing the same GC hitches as now, because as noted above the speed of the GC has nothing to do with how many cores the game is running on. It has to do with how much memory garbage the game is producing, which is a figure we optimized quite a bit in vanilla because it's the one that really matters.

Honestly there's a lot of voodoo reasoning about optimization and performance out there. When you actually profile a real game in real-world situations on a real and flawed tech stack (like Unity and the Mono GC), the things that actually matter can often turn out to be completely different than what you might have thought. In this case, what actually matters is reducing memory garbage. Multi-core doesn't help this. Messing with the default GC works against this.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ReZpawner on November 10, 2019, 10:56:37 AM
This is a bit of a shot in the dark, but during my own development in Unity, I came across someone mentioning that strings could cause stuttering and tiny lagspikes due to garbage collection, and that the way around it was to use stringbuilder instead of strings. It just stuck with me since it describes the problem that a lot of people are having in Rimworld at late game. Could this be relevant?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 10, 2019, 08:05:58 PM
Quote from: ReZpawner on November 10, 2019, 10:56:37 AM
This is a bit of a shot in the dark, but during my own development in Unity, I came across someone mentioning that strings could cause stuttering and tiny lagspikes due to garbage collection, and that the way around it was to use stringbuilder instead of strings. It just stuck with me since it describes the problem that a lot of people are having in Rimworld at late game. Could this be relevant?

Building complex strings with repeated string allocations is one out of a thousand things that can spew memory and make GC spikes. Vanilla doesn't do it, but some mods probably do.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on November 10, 2019, 10:54:51 PM
Iam happy that i found this thread.
I recently returned from a rimworld break since spring 18.
Back then i used like 150 mods and had no stuttering.
I used fps monitor to visualize frametime spikes and even in an unmodded vanilla new game i get them on 3x speed like every 40 seconds?
if i load 2 mods - mod manager and combat extended the frametime spike at 3x speed occur more frequent and it is also visible to the human eye as a micro stutter.
guess what the more mods, the more insane the stutter.

I reverted to b18 and loaded some b18 mods, no stuttering <3
Don't get me wrong but something must happen. I don't understand how people are able to play these days. I even see up to date let's plays with stutters :D and these guys like they dont recognize it.

Specs i7-8700k and a nvdia 1060 first i thought my cpu crapped itself and testet it on another machine with identical specs, same results.
So any ideas, any fixes? I want to enjoy my old rimworld again but i can not stand this stutter.
Thanks
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ReZpawner on November 11, 2019, 10:55:49 AM
Quote from: Tynan on November 10, 2019, 08:05:58 PM
Quote from: ReZpawner on November 10, 2019, 10:56:37 AM
This is a bit of a shot in the dark, but during my own development in Unity, I came across someone mentioning that strings could cause stuttering and tiny lagspikes due to garbage collection, and that the way around it was to use stringbuilder instead of strings. It just stuck with me since it describes the problem that a lot of people are having in Rimworld at late game. Could this be relevant?

Building complex strings with repeated string allocations is one out of a thousand things that can spew memory and make GC spikes. Vanilla doesn't do it, but some mods probably do.
I have to disagree there. Very late game (we're talking 10-20 years in), the game slows down considerably. Some of it can be attributed to job-spam for cleaning\hauling, but it still has the microstutters when those jobs are turned off. I can't be certain of what actually causes it, but it would make sense if it had something to do with string garbage collection, since the number of excellent artpieces\weapons\gear are sure to generate a LOT of names. It could also be the history\events that are stored, but I can't be sure of this. Others have theorised that too many storage areas are to blame.

Some useful things I have found to help are:
When setting up crafting tasks (until X), specifying storage helps, so the game doesn't have to check every single storage area.
Have 1-2 pawns do cleaning, remove the rest.
Have 3-4 pawns do hauling, remove the rest. (these numbers will obviously vary depending on the size of the colony).
Using a cleanup mod to remove the dead pawns also helps significantly, but this requires modding (in a modded playthrough I had literally thousands of these taking up resources, responsible of about 50% of the savegame size, which at the time was 70+MB. ).

None of this is meant as criticism btw, I absolutely love the game and the work you've done, from kickstarter to post release. I just would really love it if the lagging would go away, as it diminishes the enjoyment we get from the game.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 11, 2019, 07:08:29 PM
If you've got a vanilla savegame with performance issues can you just email it to me? [email protected]

It'd be useful to be able to take a look; these cases are hard to reproduce in-house.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ReZpawner on November 12, 2019, 04:53:59 AM
I tend to delete the saves when starting new colonies, but I did start a new unmodded game yesterday, so I should have one in a day or two that I can send to you. Hopefully it'll help!
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 12, 2019, 10:24:59 PM
Quote from: ReZpawner on November 12, 2019, 04:53:59 AM
I tend to delete the saves when starting new colonies, but I did start a new unmodded game yesterday, so I should have one in a day or two that I can send to you. Hopefully it'll help!

Much appreciated :)
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ReZpawner on November 14, 2019, 08:36:53 PM
Quote from: Tynan on November 12, 2019, 10:24:59 PM
Quote from: ReZpawner on November 12, 2019, 04:53:59 AM
I tend to delete the saves when starting new colonies, but I did start a new unmodded game yesterday, so I should have one in a day or two that I can send to you. Hopefully it'll help!

Much appreciated :)
No problem. It'll take a few days I think, since I'm only at around day 450  now, and real life is interrupting the playthrough a bit. The good news is that there's no stuttering to report yet.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 16, 2019, 01:14:00 AM
Quote from: ReZpawner on November 14, 2019, 08:36:53 PM
...The good news is that there's no stuttering to report yet.

..thats actually bad news in that context here.

My "Story" w the game regarding performance / stuttering is kinda vanilla as well...
Played the game through A16-17-18 and some B19 before meh, sorta burned out and let it rest for a year - my biggest colony during that time had 25 Colos, some 100+ Animals on top of 50+ Hauler Huskies and aside from the Idle Pawn stutters, which were fixed by setting day/night shift schedules the game ran fine, that is with mods of course, at that point i had some 150 installed because its just too enticing "Ohh, i need that...and that...i WANT this OMFG...givegive!"

Imagine my surprise when i finally started to get back into it w 1.0 and with a culled&updated modlist down to about 80 of the same mods i had before, just some superfluous stuff removed or stuff i never actually used before either and - once you hit like, year 5 and between 12-15 Colonists (im picky...) the game becomes a Stutterfest - no matter how you set up schedules, no idle pawns, NO animals whatsoever and i couldnt figure out why considering pre-1.0 i had much more going on and it was smooth-ish sailing...

The funny thing is this tho - when you load a save, after starting the game, obviously...and just let it sit there and run on normal speed stuttering along and all - after about 20-30mins of this stutter festival it clears up and becomes somewhat smooth-ish again for a few mins - OR something "big" has to happen within those 20-30mins like a Raid or mech ship crashing or whatever..all of a sudden the game runs w/o stutters for like 10-15mins, like the game engine has to take a shat first before its ready to go temporarily and then gets constipated again

Anyway - the base game has a problem somewhere - Mods can and do matter obviously, esp the ones adding simple Walls w "tickertype-NORMAL" for every segment of wall / pipe placed or some other nonsense like that BUT the problem is the base game, considering that even before those aforementioned mod hitches were fixed on my end ie before 1.0 the game was smooth-ish w much larger colonies and modlists
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on November 16, 2019, 01:12:58 PM
thank you for your 2 cents on this,

its pretty the same situiation i expirienced, played b17-b19 builds, even earlier ones
burned out

Came back installed game
grabbed all mods i wanted (was used to)
started playing and it took me a few minutes to recognize that something is wrong, it became so obvious that i discontinued to play the game in this state.

i mean vanilla is ok, but it becomes really pretty with this nifty mods out there.
Maybe tynan can tell what he changed from b19 to 1.x versions
And what changes can affect this, maybe he can find out whats makin our pretteh game lag

And i appreciate your long time test very much.



Check these,
look at the top left frame time bar

https://www.youtube.com/watch?v=uhO8aVsqT4E

https://www.youtube.com/watch?v=iMPiV6-OXY8

https://www.youtube.com/watch?v=NBd1DhsBc9o
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Pangaea on November 16, 2019, 05:06:01 PM
There are clearly very different experiences about this. But one thing I don't get is the people who claim there are lag problems, and the proof is a tiny spike in a graph from a third party program. Meanwhile, the videos show that the actual game runs smooth as a baby's bottom. I'm sure others experience observable lag, otherwise so many people wouldn't write about it, but unless you actually see/experience lags...?

I don't have a vanilla game either, but run with relatively few mods. The colony is almost 10 years old, has 44 pawns and over 200 animals. And some exceptions aside (huge raids spawning for instance), I don't experience lag. There may very well be spikes here and there if I used a third party program like that, but if I don't actually see lags, I honestly don't care.

Unless there are big lag issues with pure vanilla games, it's not necessarily something that has changed in the vanilla game that causes these issues either (the real ones). Perhaps one or more mods that many people use have changed or expanded in ways that cause (more) lag.

For those who experience lags, I suppose the best suggestion is to send those save files to Tynan and let them check it out themselves. If there is an issue with the vanilla game, that will help them track it down.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 17, 2019, 11:34:22 AM
Another one reporting in.

1080ti here, stutters on both vanilla and modded. Obviously more severe with modded, but still extremely frustrating with vanilla.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 18, 2019, 01:51:57 AM
Who is using Win10 here - pre 1809 version please if anyone?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 18, 2019, 04:05:14 AM
Quote from: atewithouttabIe on November 18, 2019, 01:51:57 AM
Who is using Win10 here - pre 1809 version please if anyone?

Windows 10 here, but currently on the 1809 version.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on November 18, 2019, 01:09:24 PM
Quote from: atewithouttabIe on November 18, 2019, 01:51:57 AM
Who is using Win10 here - pre 1809 version please if anyone?

win 10 18362 build 64bit
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 18, 2019, 03:52:05 PM
Hmhm....

Alright story time:

Yesterday, working, futzing around w Mods (adding, not removing because, game runs rather...not optimal anyway so pfht) in between real-life stuff, restarting the game like 20 times back and forth, restarting windows and all that...
Always the same: game runs, max FPS 27-30, GPU at idle speeds, same FPS when forced to max, Windows + game total memory usage hovering between 4.2-4.6GB etcetc..it was playable, sure but something seemed to throttle everything despite the fact nothing else was running cept windows (..doh)

So at some point, after i was done w the whole work / Mod futzing / rebooting windows and such stuff i say FINE, lets start it and just let it chug along for a while...and now it runs w 55-60FPS again / 50-60TPS, 3.2GB total phys memory usage and all peachilily to the point im loathe to turn it off if it werent for that nagging voice as well as a wet towel on head sometime around 1am from a certain female whom shall remain anonymous.

So, remember, i save the game w everything running fine sans the obligatory 1/3sec microstutter every 6-9secs alright...

A few hours later, turn PC back on do the usual stuff emails/work etc - alright, lets restart windows and then load Rimworld...load my save from 1am, the 55-60FPS/TPS 3.2GB usage one...and back to crap again - 27FPS/20-60TPS* and 4.3-4.5GB mem used,  no matter what - NOTHING had changed between 1am and NOW - wtf?

So..what now? Remember the "game has to take a dump first" thing i mentioned above?

Ya, on account of being a tad "irritable" at this point, i decide to make a save using words not repeatable here and then spawn a mild 4000 point raid - wait 5 mins - still 27FPS/20-60TPS and 4.3GB memory usage total (windows and game combined)...

...then the Raid spawns (aka the Game takes a "DUMP"...) and instead of the FPS going down further as well as the TPS...
...both go back up to NORMAL instantly ie 60/60
...the GPU finally starts to use something other than P5 state (324Mhz)
...total MEMORY usage drops instantly by 1GB down to 3.2GB (900MB Windows - 2.3GB Rimworld) despite there being 30 pawns more on the Map pathing, snorting flake, telling dirty jokes, getting attracted to each other, hashtagging selfies ("#RAIDYOLO #metoo #tablesrock") etc..

Is Windows10 / Memory / Resource management etc being doofy?
Is the game being a nimrod?
Are both working in concert to purposefully display a level of incompetence usually only associated w public servants or uncoordinated preschool children trying to "wahs ur car w rocks, daddy LOOK!!"?

What...?

* Ticks Per Second - 60 being normal
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 18, 2019, 04:09:05 PM
Quote from: atewithouttabIe on November 18, 2019, 03:52:05 PM
*snip*

I'm curious if you were to do the same thing and let the game take its dump then load a previous save, would it be running smoothly still? My assumption would be yes, since it's still the same instance running, thus it would continue to run as it should. I might do some testing myself later today and see what happens.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 18, 2019, 04:25:37 PM
As it happens i just did that...

The craftily worded save i made w the game running at a glacial pace (27-30FPS etcetcetc) BEFORE i summoned the raid via DEV is still running like...excrement

The save i made just AFTER the raid, w everything suddenly running peachy (55-60FPS/TPS etcetc) is still running spiffy...

So i can load 2 saves of the same colony now, same in-game day, same modlist / mod order, same all w the exeption that the first save runs "constricted" before Raid and the 2nd save, after raid, runs nominal...srysly what changed w the raid that makes the game run fine suddenly?

Explain...
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 18, 2019, 09:42:50 PM
We have collected a few mature colonies (1,000,000+ wealth) and we're going to do some profiling and look at them. If there is an issue it'll be fixed.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 18, 2019, 11:11:23 PM
Spawning the raid made no difference for me, sadly. Still hanging on GC processing.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 19, 2019, 12:52:12 AM
At this point i wouldnt rule out Win10 being an asshat w regards to resource management either, either allocating or releasing to be re-allocated for the game - sure enough theres enough threads on various mature forums w titles such as "Win10 stutter / Laggy / Unpossible to play xyz on Win10" etc esp with regard to the standby / file caching mechanisms in particular and such...
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 19, 2019, 04:09:39 AM
Fresh, up-to-date install of Win 10 complete but the same problem persists. I'll be testing on a windows 8 install later to see if it exists there too.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 19, 2019, 04:25:26 AM
There will be small GC spikes in every version of the game, because they're inherent to C# and Unity. They should be small enough not to be perceptible without tools, but if you're using tools you'll always see them.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 19, 2019, 04:41:07 AM
Quote from: Tynan on November 19, 2019, 04:25:26 AM
There will be small GC spikes in every version of the game, because they're inherent to C# and Unity. They should be small enough not to be perceptible without tools, but if you're using tools you'll always see them.

This simply isn't the case, though. At least for me, they're consistent and extremely noticeable. I can't pan the camera without a spike causing it to fly off in a random direction as the game is still taking the camera movement input during the spike. This happens at least once a minute on a purely vanilla save.

Edit: This can also be seen in recent, albeit modded, youtube let's plays as well as the videos linked earlier in this thread.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Pangaea on November 19, 2019, 05:45:39 AM
Maybe a non-issue, but a thought that came to me while trying to set up a new game: what sized maps do people use?

Personally I use 275x275, and I would imagine that bigger than that has a greater chance of causing lags, especially in the late game.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 19, 2019, 06:35:33 AM
Quote from: thejazzmann on November 19, 2019, 04:41:07 AM
Quote from: Tynan on November 19, 2019, 04:25:26 AM
There will be small GC spikes in every version of the game, because they're inherent to C# and Unity. They should be small enough not to be perceptible without tools, but if you're using tools you'll always see them.

This simply isn't the case, though. At least for me, they're consistent and extremely noticeable. I can't pan the camera without a spike causing it to fly off in a random direction as the game is still taking the camera movement input during the spike. This happens at least once a minute on a purely vanilla save.

Oh, I was talking about a new game situation as in some videos that were posted earlier.

How old is your colony? How many colonists? How much wealth? Which mods are you using of any kind?

What you're describing doen't sound like GC spikes. If the system is under low memory pressure it'll create tiny GC spikes (not perceptible) every 20-30s. If it's under high pressure it'll make big spikes every 5-10s. It's hard to see  a case where it'll make big spikes every 60s.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 19, 2019, 06:44:15 AM
Quote from: Tynan on November 19, 2019, 06:35:33 AM

Oh, I was talking about a new game situation as in some videos that were posted earlier.

How old is your colony? How many colonists? How much wealth? Which mods are you using of any kind?

What you're describing doen't sound like GC spikes. If the system is under low memory pressure it'll create tiny GC spikes (not perceptible) every 20-30s. If it's under high pressure it'll make big spikes every 5-10s. It's hard to see  a case where it'll make big spikes every 60s.

Ah, my apologies in that case. I'm also talking about a new game situation. As far as a vanilla/no mods run is concerned, this is from the very beginning. Immediate lag spikes beginning within the first minute of the colonists (3) landing and happening every 30-45 seconds forever. Minimum wealth, whatever the original three colonist scenario starts you with. It's exacerbated by time acceleration, exponentially.

On a modded run of 3-5 colonists it begins immediately and happens every 5-10 seconds, perhaps more frequently.

In both scenarios, even when no orders have been given and the game is left to sit, it still occurs.

Edit: I've also now run this on a 2017 Macbook pro running 10.14.6 Mojave (arguably not the best hardware to run anything on) completely vanilla and the same issues occur. I don't believe it to be an OS or purely windows based problem.

Edit #2: I reverted to B19 just out of curiosity and it runs flawlessly. So it would seem like something between B19 and 1.0 release, although you likely already know that.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 19, 2019, 12:01:29 PM
So...OS's out of the loop, happening across multiple platforms - back to focusing all pummeling attention on game proper, exclusively? Fair enough... =)
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ForeverZer0 on November 19, 2019, 02:08:49 PM
Quote from: atewithouttabIe on November 19, 2019, 12:01:29 PM
So...OS's out of the loop, happening across multiple platforms - back to focusing all pummeling attention on game proper, exclusively? Fair enough... =)

The only problem with this theory is that if was a core problem with the game, it would typically be a more widespread problem that everyone experienced, not a just a handful. When there are only a few people experiencing the same problem, and not the overwhelming majority, it often does indicate OS, hardware, or driver issues.

Either way, hopefully it is able to be resolved, it would indeed be irritating to play with it studdering in such a way.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 19, 2019, 10:07:04 PM
A small amount of further testing: The 32-bit (x86) version of the game also runs without issue, zero spikes.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 19, 2019, 10:11:45 PM
Quote from: thejazzmann on November 19, 2019, 10:07:04 PM
A small amount of further testing: The 32-bit (x86) version of the game also runs without issue, zero spikes.

Now that's really interesting. It would be amazing if you could actually tell me how many milliseconds the spikes are on each version.

We're looking at possibly doing an update that comes with a new Unity version update, and Unity updated their Mono runtime which includes an improved garbage collector, so it may solve these issues. But, don't hold your breath - it isn't planned for the immediate future.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: mjcox02 on November 20, 2019, 01:13:03 AM
I just reinstalled the game recently, after a hiatus for a month or two. I ran into the same problem others are having in this thread.

I stumbled upon this post, and thank you for looking at the issue! Really, Thank you!

I've been playing a lot of paradox games recently.. And, well one of those games, is a performance hellhole. I love the game, but it runs terribly. And the community is up in arms about it, but the developers are simply ignoring the issue at hand. The game is unplayable because of it.

So I was surprised to see someone caring about performance! Thank you, this game is a masterpiece, and so is the community. Let's hope there's a fix to this stuttering problem!
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 20, 2019, 11:23:07 AM
Okay, so I've done some further testing (again).

Using the steamdb manifest information and steam's depot download function through the console, I downloaded several previous versions of the 1.0 release:

Version 1.0.2096 (November 23, 2018, Manifest ID 2990835901037261399) - No Spike
Version 1.0.2150 (January 20, 2019, Manifest ID 4677681625388671945) - No Spike
Version 1.0.2231 (April 7, 2019, Manifest ID 3518976739099334473) - No Spike

These three were run completely vanilla and no lag or spikes occurred. I purposefully spawned in several different events including a number of raids per colony.

The versions following this, however, did spike.

Quote from: Tynan on November 19, 2019, 10:11:45 PM

... It would be amazing if you could actually tell me how many milliseconds the spikes are on each version...


While measuring the exactly millisecond would be difficult (unless you know some easy way for me to do this?), I can confirm that on all versions I've tested so far that have spiked, it's been 1/4 of a second. This is true of both modded and vanilla, the spike duration doesn't change.

I believe I've exhausted all possible testing abilities for myself here, unless anyone has any further ideas. Hopefully this will shed some light.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 20, 2019, 11:35:48 AM
All x64 versions or 32?

Also, did you try a handstand while balancing a frisky gerbil on one foot during a full moon on an odd day not ending in Y yet while further testing?

...you axed  :)
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 20, 2019, 11:47:10 AM
Sorry, should've mentioned that! All tests above done on x64 versions. Given that the x86 version is basically relegated to 32-bit systems (and that it worked on the latest version without issue) I didn't bother testing it.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on November 20, 2019, 01:02:54 PM
maybe post your hardware specs aswell o/
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: thejazzmann on November 20, 2019, 02:33:31 PM
Yeah, probably a good point.

CPU: Core i7 7700K
GPU: MSI GeForce 1080Ti Gaming X 11gb
RAM: 32gb (4x8gb) Corsair Vengeance DDR4
Running Rimworld from a Samsung 960 EVO NVME 500gb SSD on Win10 Home
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on November 20, 2019, 09:35:56 PM
That's some hardcode Steam hacking. Nice.

The versions are:

Oct 1, 2019    1.0.2408     main default branch
May 27, 2019    1.0.2282       
Apr 7, 2019   1.0.2231   
Jan 20, 2019    1.0.2150
Jan 16, 2019     1.0.2096     beta branch: catalina_test

So basically you're saying something happened between April 7 and May 27 which causes lag spikes.

We didn't change any content during that period, the only change was optimization to game loading as described in this blog post (https://ludeon.com/blog/2019/05/rimworld-update-1-0-2282-released/). However that code shouldn't in principle do anything during play at all; it only made loading faster. But, bugs can be weird.

What I'd be interested in is if anyone else can try those same observations and see what happens to you. The beta branch catalina_test is version 2096, so you don't need to use the Steam console to get it.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Asero on November 21, 2019, 08:51:40 AM
Well, I didn't test with without any mods installed or tried any different versions, but I did stumble across something that may be relevant. (Or perhaps it's a fluke and it's only working for me.)

I recently started playing Cities Skylines again and as a result I use a lightweight memory optimizer especially since the game can be extremely resource hungry with all DLCs and mods installed. Either way, I ran the memory defrag beforehand as per usual; but decided to play Rimworld instead :) Rimworld suddenly had almost no fps drops, which was pretty surprising since I'm playing on an i3 and the lag used to get pretty extreme in my case. Could these stutters perhaps be caused something that changed how Windows 10 manages system memory?

Please note that I did not run the memory optimization at any point whilst playing the game. In fact it's a pretty bad idea and can really mess things up--which is why I'm using Wise memory Optimizer as it's lightweight, free, easy to use and doesn't try to constantly screw with the memory when you're busy playing the game. Either way, I doubt this will work for everyone but it's probably still worth a mention.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Geist3 on November 21, 2019, 04:00:48 PM
Hi everyone,

I have 172 hours in Rimworld since 2018, played mostly with mods without any problems.

Recently I reinstalled the game but found that the game stutters every few seconds as described by the posters in this thread. This happens in vanilla right at the start of a game. I can deal with  low fps or stutters due to mods but this seems like a persistent 'baked in' stutter that kills enjoyment of the game.

I kept checking forums and reddit every few weeks to see if anyone came up with a solution, just signed up now since the head dev seems to be investigating personally (awesome, thanks for listening :))

My specs:
Windows 7
i7 3.5ghz
R9 270
8 gigs of ram

Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: atewithouttabIe on November 21, 2019, 04:11:29 PM
Quote from: Asero on November 21, 2019, 08:51:40 AM
... Could these stutters perhaps be caused something that changed how Windows 10 manages system memory?
...

Discussed at lenght here https://forums.guru3d.com/threads/fix-game-stutter-on-win-10-1703-1809.420251/page-36

been futzing around w ISLC as well as exploit protection toggling on and off, so far its been "promising" to a certain degree esp w regards to the aforementioned "game has to take a dump first" analogy i used - worth a try at this point huh

What your Mem Optimizer did was most likely nothing more that is being discussed in that thread ie stopping windows from filling all free memory w standby stuff so theres nothing left for apps that actually need it right now thus causing stuttering and abysmal performance in general - just flick through that thread and you'll see
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kassenschlager on December 01, 2019, 07:36:52 AM
Hey, wanted to thank you for this post.

For the last few weeks i have had the same problem as everyone described here.
I thought the problem were too many mods, worktypes and pawns which seemed to exacerbate the problem. Prioritizing Wortypes etc. helped a little. In the end it was the standby list though. When i start a savegame and purge the standbylist, the freezes are gone completely. To anyone here, this is the program mentioned in the thread: https://www.wagnardsoft.com/content/intelligent-standby-list-cleaner-v1000-released (https://www.wagnardsoft.com/content/intelligent-standby-list-cleaner-v1000-released).
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on December 01, 2019, 12:55:42 PM
any updates on this matter?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ReZpawner on December 01, 2019, 02:45:50 PM
Kind of.
A number of savegames are(or will be) analysed by the developers.
I've tested the vanilla game for 137 hours for the past two weeks, and while the vanilla game does slow down (at times significantly), and there are stutters, the microstuttering is not at all present in a 1.5 million wealth colony at the current time at year ~12 ish. Other than that, you'll have to ask the devs, as they are sure to know more than the testers.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: reuven on December 25, 2019, 08:32:19 PM
I too did some mod enabling and disabling because there was a noticeable 1 sec freeze between intervals or around 20 or 30 seconds, and I chalked it up to only two activated mods: Hugslib and Jecstools. It does seem that after deactivating Jecstools it seemed to improve the stutter (it was so bad that it seemed to stutter when I was in the main menu switching between options. It kind of sucks because the mod is required for a lot of other ones. I'll do some more experiences with my modlist.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on December 26, 2019, 06:00:45 AM
Quote from: reuven on December 25, 2019, 08:32:19 PM
I too did some mod enabling and disabling because there was a noticeable 1 sec freeze between intervals or around 20 or 30 seconds, and I chalked it up to only two activated mods: Hugslib and Jecstools. It does seem that after deactivating Jecstools it seemed to improve the stutter (it was so bad that it seemed to stutter when I was in the main menu switching between options. It kind of sucks because the mod is required for a lot of other ones. I'll do some more experiences with my modlist.

yea well thats bad bec hugslib and jectools pretty needed for everything <3
but thanks for your intel
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ruPal on January 16, 2020, 05:13:29 PM
Quote from: Tynan on November 19, 2019, 10:11:45 PM
Quote from: thejazzmann on November 19, 2019, 10:07:04 PM
A small amount of further testing: The 32-bit (x86) version of the game also runs without issue, zero spikes.

Now that's really interesting. It would be amazing if you could actually tell me how many milliseconds the spikes are on each version.

We're looking at possibly doing an update that comes with a new Unity version update, and Unity updated their Mono runtime which includes an improved garbage collector, so it may solve these issues. But, don't hold your breath - it isn't planned for the immediate future.
Don't like to be that person, but are there news about that update?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Varius on January 17, 2020, 05:56:52 AM
Hey folks.
I've only literally created an account today to post and mention than I to do experience the same issues which lead me to do an unmeasurable amount of troubleshooting in the past.
To be frank, I've experienced these issues since the inception of this thread and I merely watched silently in hopes that a solution would be found or possibly the mention of stone unturned that I had missed. I wanted to make an account and subscribe to this thread in hopes that one day, metaphorically - salvation knocks.
I've been playing this game since it's the release of 17 & 18 and when it hit 1.0, I immediately noticed the stuttering and the plummet of performance.
To sum up - I'm posting to aid in further visibility of this issue, voicing my concern and eagerly waiting for a solution!
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ruPal on January 17, 2020, 10:22:23 AM
I think that most of us experience issue when we move camera. In that same moment camera jumps and that causes functional issues. Hope that it is possible to stop all input tracking milliseconds before GC and activate input tracking as soon as GC done his job. That will atleast make this issue less annoying since camera stops jumping.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on January 22, 2020, 05:54:06 PM
Quote from: ruPal on January 17, 2020, 10:22:23 AM
I think that most of us experience issue when we move camera. In that same moment camera jumps and that causes functional issues. Hope that it is possible to stop all input tracking milliseconds before GC and activate input tracking as soon as GC done his job. That will atleast make this issue less annoying since camera stops jumping.

it has nothing to do with camera movement, you may recognize it better if moving but it is happening regardless if you are moving the camera or holding it static check youtube rimworld stutter or freeze etc.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: kelmvor on February 06, 2020, 09:57:59 PM
another person jumping to say i have this issue and it really makes the game hard to enjoy, happens all speeds, vanilla game, modded game. Recently started playing again is when the issue first arose. been looking all over to roll back my version of the game because whatever version I was playing last summer worked fine.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: kelmvor on February 07, 2020, 01:54:54 PM
Whether its good or bad news for some, I fixed the issue I was having with the stuttering every few seconds. After testing everything I could think of I searched some old hard drives for an old install of the game to check the previous version I had on there. It ended up being 1.0.2282. Tested it with no mods running it independently without steam. No stutters. Threw all my mods on there from my steam mod files. No stutters. Not the ideal fix but after suffering for days I can finally play the game again.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: kelmvor on February 10, 2020, 10:13:17 PM
yah problem still there. has this unity thing always been an issue and I seriously just never noticed the massive stutters?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Jibbles on February 11, 2020, 12:13:53 AM
Quote from: kelmvor on February 10, 2020, 10:13:17 PM
yah problem still there. has this unity thing always been an issue and I seriously just never noticed the massive stutters?

In my experience, only started after b18 -outside late game colonies.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: kelmvor on February 12, 2020, 08:21:49 PM
For me it's juat since I tried playing again after a year. Been trying to fix it for a week or more
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Brightsideguy on February 15, 2020, 09:19:03 PM
Quote from: Tynan on November 20, 2019, 09:35:56 PM
That's some hardcode Steam hacking. Nice.

The versions are:

Oct 1, 2019    1.0.2408     main default branch
May 27, 2019    1.0.2282       
Apr 7, 2019   1.0.2231   
Jan 20, 2019    1.0.2150
Jan 16, 2019     1.0.2096     beta branch: catalina_test

So basically you're saying something happened between April 7 and May 27 which causes lag spikes.

We didn't change any content during that period, the only change was optimization to game loading as described in this blog post (https://ludeon.com/blog/2019/05/rimworld-update-1-0-2282-released/). However that code shouldn't in principle do anything during play at all; it only made loading faster. But, bugs can be weird.

What I'd be interested in is if anyone else can try those same observations and see what happens to you. The beta branch catalina_test is version 2096, so you don't need to use the Steam console to get it.

Anyone know if the catalina_test branch was removed?  I wanted to test this out for the thread, but can't tell if its gone or I'm just blind.

[Also dealing with the stutter problem for a while with vanilla 1.0. Clearing windows standby, as suggested by others, did nothing.]
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: SergeshD123 on February 16, 2020, 12:46:57 AM
Now that this thread has mentioned it, I can definitely say at least for me that something goes wrong between the older versions and the full release. I can't remember the exact version, but I played roughly the same mods between A16-B18 as I do during the early days of 1.0 and I never suffered any stutter. As soon as 1.0 hit (can't comment on B19 as I took a hiatus during this), the game started stuttering. This can't be due to the mods because as I said, I used the same mods on the release version as I did in the Alpha and Beta versions.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: kelmvor on February 16, 2020, 09:07:52 PM
I wont say I'm glad its happening to others cuz this sux I,wanna playit again. But at least I know I'm not crazy
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on February 17, 2020, 08:26:29 AM
The game now uses an incremental garbage collector, which should remove the periodic frame hitches that would appear when a lot of memory was allocated and released. However, note that there is still a cost to memory allocations, so modders should still try to reduce allocation wherever possible

Omg
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on February 18, 2020, 12:39:06 AM
Curious to know how 1.1 has affected/will affect this issue.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on February 18, 2020, 09:55:36 AM
I will for sure be testing as soon as enough mods will be updated to 1.1 and 1.1 leaving unstable <3

But i thought just this point in the changelog aimed straight towards this issue that people expirienced between after flawless B versions of the game.

So if it was aimed towards thank you if not.. what the hell is goin on :D
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: moyashii on February 19, 2020, 11:30:17 PM
I've done some light testing every now and again in Vanilla and so far, the stuttering due to GC seems far less noticeable, if not completely absent when compared to before. Even Vanilla prev. to 1.1 still had some noticeable stuttering every ~60 seconds. After a few (admittedly not very long) games and experimenting without any mods it's still buttery smooth.

I almost can't believe it. We'll need to wait for more complex mods to be updated to see how much of an effect it had on them I suppose, but if the issue is resolved, then I'm very thankful! I'm glad we had a lot of people to shine some light on this issue, and for your quick action on this matter, Tynan.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on February 25, 2020, 02:32:34 AM
Initial tests are very positive for me as well!

I haven't noticed any appreciable amount of lag or spikes since trying a "quick" new game today with the new dlc and 30 or so mods.  By "quick", I mean that naked brutality handed my ass to me after only a couple of hours: while suffering from malaria and heat stroke, my lone dude rescued someone from an angry duck and had his base set on fire by raiders, only to be kidnapped just before he died in another raid.  ;D
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: madd_mugsy on March 11, 2020, 03:49:56 AM
Another quick update: after several dozen hours of playing with a few hundred mods, I feel confident in saying that this issue has been resolved, at least for me.

Any lag issues I have now, I can attribute to various mods + using the unstable branch.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Tynan on March 11, 2020, 04:02:06 AM
Awesome!
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Edrin on March 11, 2020, 07:53:26 AM
USB hubs (and/or other USB thingys) are known to sometime cause problems with hard drives, causing freezes. It's about drawing too much power or something like that. motherboard manufacturers are often not following 100% of the USB standard. Maybe you can try to unplug them?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: tobi1449 on March 16, 2020, 03:43:10 PM
I'm currently having the same problem (tons of mods, latest Rimworld). Is there any way to attach a memory profiler to Rimworld to see what is actually causing this?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on March 16, 2020, 05:05:51 PM
Maybe this ?
https://github.com/Dubwise56/Dubs-Performance-Analyzer
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: tobi1449 on March 17, 2020, 01:01:48 PM
Already tried it, but unfortunately it doesn't help me find out what's causing all the GC freezes (i.e. what's creating so many objects that the GC is triggered so often and needs so much time).
And the usual memory profiles like dotMemory or Unity's own memory profile don't seem to work with non-debug builds :/
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on March 18, 2020, 03:46:32 AM
Maybe this one ?
But it isn't for 1.1.
https://ludeon.com/forums/index.php?topic=50324.0
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Kaelen Mosar on March 19, 2020, 06:31:23 PM
Quote from: tobi1449 on March 16, 2020, 03:43:10 PM
I'm currently having the same problem (tons of mods, latest Rimworld). Is there any way to attach a memory profiler to Rimworld to see what is actually causing this?

pc specs`?
modlist?
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ZetaJ on June 22, 2021, 04:11:31 PM
First of all, sorry to open this old discussion.
...
After purchasing the game and playing version 1.2.3005, my game immediately started with the problems that are explained in this forum.
Since I found no solution, I uninstalled the game from Steam and refunded it.
But then I had an idea, to test the game pirated.
And it works perfectly !!!! wtf
I don't know why it does it, but it seems that ... the problem is Steam?
Has anyone tried looking at Steam settings for this game?
Maybe Steam is "bugging" Rimworld.
Please have someone check this,.....this could be a possible solution.
Thanks.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Canute on June 23, 2021, 03:02:08 AM
Steam steam, ahh isn't that the big bug which drain the developers money and user informations to get fat ? :-)
You can buy the game over the website or GOG for a DRM free version, and geting even a steam key when you use the website.
I don't think it is a generell steam issue because much more people would have it. It is maybe just some special circumstances you got at your mashine with installed apps and settings.

Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: ZetaJ on June 23, 2021, 02:26:22 PM
Quote from: Canute on June 23, 2021, 03:02:08 AM
Steam steam, ahh isn't that the big bug which drain the developers money and user informations to get fat ? :-)
You can buy the game over the website or GOG for a DRM free version, and geting even a steam key when you use the website.
I don't think it is a generell steam issue because much more people would have it. It is maybe just some special circumstances you got at your mashine with installed apps and settings.
I already have the money put into steam, I can't buy the game on the Ludeon website.
And what do I do now? xD
If I buy the game on steam, I will have problems ...
I can't buy it on the website because the money, which was supposed to go to Rimworld, is in Steam ....
...
The universe definitely doesn't want me to play Rimworld.
Title: Re: Small freezes / fps drops with increased frequency when speeding up game
Post by: Luftraumm on September 21, 2021, 12:12:46 PM
Still an issue, logs show no major errors

i7, 16gb ram, gtx 1080, win10
under 20 mods, mostly QOL

debug log message spamming:

--- Main thread ---
0.0106ms (self: 0.0100 ms) ExecuteToExecuteWhenFinished()
- 0.0006ms (self: 0.0006 ms) Verse.Sound.Sustainer -> Void <.ctor>b__15_0()


Hotspot analysis
----------------------------------------
1x ExecuteToExecuteWhenFinished() -> 0.0100 ms (total (w/children): 0.0106 ms)
1x Verse.Sound.Sustainer -> Void <.ctor>b__15_0() -> 0.0006 ms (total (w/children): 0.0006 ms)

UnityEngine.StackTraceUtility:ExtractStackTrace ()
Verse.Log:Message (string)
Verse.ThreadLocalDeepProfiler:Output (Verse.ThreadLocalDeepProfiler/Watcher)
Verse.ThreadLocalDeepProfiler:End ()
Verse.DeepProfiler:End ()
Verse.LongEventHandler:ExecuteToExecuteWhenFinished ()
Verse.LongEventHandler:ExecuteWhenFinished (System.Action)
Verse.Sound.Sustainer:.ctor (Verse.SoundDef,Verse.Sound.SoundInfo)
Verse.Sound.SoundStarter:TrySpawnSustainer (Verse.SoundDef,Verse.Sound.SoundInfo)
RimWorld.CompTerrainPumpDry:CompTickRare ()
Verse.ThingWithComps:TickRare ()
Verse.TickList:Tick ()
(wrapper dynamic-method) Verse.TickManager:Verse.TickManager.DoSingleTick_Patch1 (Verse.TickManager)
Verse.TickManager:TickManagerUpdate ()
(wrapper dynamic-method) Verse.Game:Verse.Game.UpdatePlay_Patch1 (Verse.Game)
Verse.Root_Play:Update ()