Ludeon Forums

RimWorld => General Discussion => Topic started by: Kenneth on December 18, 2018, 07:11:59 PM

Title: Looking for infos regarding long loading times with modded games
Post by: Kenneth on December 18, 2018, 07:11:59 PM
We've recieved some reports regarding very long loading times (30+ minutes) for (heavily) modded games and we're currently investigating these reports, but we're lacking concrete reproduction info.

Everyone who is experiencing this or experienced it in the past please post your full mod list, hardware specs (CPU, Memory, HDD/SSD, OS) and your logfile (see this thread on how to get your logfile: https://ludeon.com/forums/index.php?topic=513.0) in this thread.
Any other like potential leads are very well appreciated aswell!

Please enable "Verbose logging" in the settings before you post your logfile. With this setting enabled there will be additional profiling information for us to analyze. See the attached screenshot on how to enable verbose logging (enable both marked settings).

Please also make sure that you have "Run in background" enabled or keep the game window in focus while its loading. If this option is disabled and you're not having rimworld in the foreground while loading the loading process will stop entirely until you bring it into focus again.
See this post for more info: https://ludeon.com/forums/index.php?topic=47478.msg449091#msg449091

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Canute on December 19, 2018, 03:14:57 AM
My longest loading time allways was with the Hardcore SK modpack. But never around 30 min.
But i could imagine with old hardware could can get this.
But this modpack don't got released for 1.0 yet, latest version was for B18.
https://ludeon.com/forums/index.php?topic=12996.0

Title: Re: Looking for infos regarding long loading times with modded games
Post by: Symile on December 19, 2018, 10:08:25 AM
Having almost entirely played only with mods, I can say my load times are in order when I actually have the game in focus.
Load times tend to be 30 minutes, just cause I keep doing stuff on the side, and forget the game is inactive.

My actual load time comes down to about 7 minutes at the most, and that's with plenty of mods installed, about 50 or so last time I checked.

General Specs:
Windows 10
Intel i5-2500k
AMD Radeon R9 200 Series
16 GB of Ram
Title: Re: Looking for infos regarding long loading times with modded games
Post by: sidfu on December 19, 2018, 10:17:53 AM
for hardcore sk mod pack on the development 1.0 build  current load times on a 4ghz quad core is around 7 min. some people have around 30+ min long times
it doesn't seem to be related to cpu speed as some people said their computers are 4ghz also. but us modders know the long load times are due to the game taking ages to read the xml.

a rough rundown of my load times over past few updated
this all with same computer
a16 2-3min
a17 3-4min a bit more mods than a16
a18 5-10min. amount of mod is actually half what a17 was
1.0 7-15min. even less mods now than b18 and still longer load times for the amount of mods.

the load times got higher as more of my mods are now xml based over dll based.

intel I7-6700k 4ghz
64gb of ram
windows 10
geforce 1080 8gb founders edition vid card
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Ashnal on December 19, 2018, 10:40:23 AM
I've had this issue while trying to cook up my perfect mod cocktail. I have a bad tendency to grab a lot of small tweak mods and only a few content mods to ensure no overlap. I've also been working on my own mod which is in here as well.

That said, I'm at 282 mods, you can find the list in the modlist.txt

This set of mods takes ~20 minutes to load to the main menu.

CPU: i7 6700k @ 4.5 GHz
RAM: 32 GB DDR4 @ 3000 MHz
SSD: Samsung  850 EVO 1TB
OS: Windows 10

EDIT: I had Run in background enabled for this, and the game was in the background while I was browsing the internet. Also, I'll have to look at that network adapter disable fix. I have 8 network adapters between Ethernet, WiFi, VPNs, and WMware.

EDIT 2: Re-loaded with verbose logging. Forgot to time it, but it felt similarly long. Only changed my loaded mods a little bit.

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Mehni on December 19, 2018, 10:45:04 AM
One dumb thing:

When the game is disallowed to run in background, that also means it's disallowed to load in background.

Perceived load times may be improved by allowing the game to start up in the background, but not actually play.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Mark4102 on December 19, 2018, 11:09:20 AM
Just checked. Took me 13 minutes to load with 207 mods.

Ryzen 5 2600 @ 3.9ghz
MSI GTX 970 4 gb
16 gb ddr4 3200mhz
Windows 10
Title: Re: Looking for infos regarding long loading times with modded games
Post by: chardog on December 19, 2018, 11:23:09 AM
Here's a lead if you haven't heard it yet: some people report shorter load times after disabling their internet and putting steam in offline mode.
https://www.google.com/amp/s/amp.reddit.com/r/RimWorld/comments/6gntfg/how_to_vastly_improve_load_times/
I've heard some theories that it has to do with unity pinging network adapters for each mod file. Whatever the case, it does seem to have a measurable difference.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Kenneth on December 19, 2018, 12:39:30 PM
Thanks to every reporter so far!
Please, when you are posting reports attach your logfile so we can see your modlist and potential leads to why it loads so slowly for you.

I've tested Hardcore SK development branch on my dev machine but it loads within a bit less than 5 minutes for me (i7-4790, 16gb ram and SSD).

As Meni mentioned, disabling the game to run in background will cause the loading process to stop aswell when in background.
To everyone having issues with this, please try enabling "Run in background" and retest your loading times (A screenshot where to find this option in the settings is attached).

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Canute on December 19, 2018, 12:44:13 PM
Or maybe "Run at background" only start to work once the game is loaded and a safegame is loaded.
I allready made this suggestion at B18 :-)
Title: Re: Looking for infos regarding long loading times with modded games
Post by: greyfade on December 19, 2018, 12:50:48 PM
I haven't been seeing tens of minutes (if I were, I'd trim the hell out of my mod list), closer to 1-2 minutes, but:

I did notice a significant 2-3x load delay after adding only two mods to the other 38: Humanoid Alien Races 2.0 and one race mod.

Freshly-created log with nothing very interesting in it attached.

Core i5-6600
GTX 970 4GB
32GB DDR4
Linux

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Ratys on December 19, 2018, 01:19:18 PM
I don't have access to my install right now, so, no logs, sorry.

This problem might not be RimWorld-specific, but, rather, Unity-specific: Kerbal Space Program seems to exhibit identical behavior, and, apparently, people have found a workaround (https://www.reddit.com/r/KerbalSpaceProgram/comments/4gpc8r/i_just_solved_my_20_minute_plus_load_times_with/). I tried this trick with RimWorld, and it seemed to help somewhat - but, being lazy and in a "just let me play already" kind of mood, I haven't clocked it. Others reported mixed success.

EDIT: that seems to be the wrong thread, and I can't find the right one. The fix was to put Steam in offline mode and disable all network adapters.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Ressing on December 19, 2018, 01:35:21 PM
ooooo now this is something I've had issues with since I FIRST started playing back in A14. Always had 30+ minute start up time, mods or not. And I used two computers to play, my desktop and my laptop which is brand new.

Only laptop specs thou since thats what I use the most to play.

CPU: AMD E-300 APU with Radeon HD Graphics 1.30 GHz
RAM: 4 GB
HDD: a pos preinstalled 296 GB hard drive
OS: Windows 10

Logfile attached! Hopefully this info helps because I LOVE this game but the load times kill me. I can walk off and cook a turkey before its done loading.

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: notfood on December 19, 2018, 01:53:15 PM
Kenneth, investigate caching the patched XML. This one works for me to cut 2-3 minutes from my load of 7-10 minutes.

https://gist.github.com/notfood/893b31b1a32b2043d8def1dbbc177a91
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Zaltys on December 19, 2018, 02:03:32 PM
Oh man, if I only had 20 minute load times... My modlist takes 70 minutes to load at the moment. I am running about 240 mods, though.

Recently upgraded from Windows 7 64-bit to Windows 10, and flashed the newest bios version (after not doing that in two years). Clean install on a new drive. No noticeable difference in the load time. Moving the game from HDD to SSD didn't seem to do much either.

MB: Z170 Pro Gaming
CPU: i3 6100 @ 3.70Ghz
RAM: 16 GB, DDR4
GPU: Geforce GTX 750 Ti 2Gb

Yeah, I know that the CPU/GPU are getting old. Still good for the genres that I mainly care about (RPGs, strategy).

Of the mods that I have, the ones that take noticeably long to load are the race mods, and SF Grim Reality. The latter is just a bunch of patches to the vanilla thoughts.

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Sosltice_The_Tech_Marine on December 19, 2018, 03:46:05 PM
It normally takes about 46 seconds for me to load vanilla rimworld, but about 18.07 minutes to load my current mod list.

MB: Intel 0GDG8Y
CPU: I5-2320 @ 3.00 GHz
RAM: 16 GB DDR4
GPU: GTX 750 TI
SSD/HDD: SSD
OS: Windows 10


[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Auraknight on December 19, 2018, 04:08:35 PM
14 minute load time, spent whole thing with Rimworld in-focus. This is a reduced modlist, mind. With the full list, it takes 40 minutes. (also previously timed) You'll excuse me for not wanting to spend an hour for a log mid-week! If you want the full mod list, gimmie a shout, and I'll do it when I have time to check back up on it. I have a feeling the problem is exponential? I only trimmed off 1/5th of my mod list, and reduced the loading time in half from the 40 minute one.

32gb ram
SSD
64bit Windows 10
Intel i7 @ 2.80 GHz
GTX980M

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Canute on December 19, 2018, 05:23:42 PM
Kenneth,
maybe you should add at your first posting that people should enable Verbose logging at the options.
Then the log contain the time how long Rimworld take for earch step.
To see the difference check this log.
https://gist.github.com/HugsLibRecordKeeper/c82fb0ebec1d2396ca26fa902d701880
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Coomby on December 19, 2018, 05:26:46 PM
It currently takes around 18 minutes to access the main menu with run in background enabled. After loading into a game, I typically have to wait another 10-15 minutes for my research tab to load due to Fluffy's OP research tab. Worth the wait imo  ;D

300+ mods

Intel I7-3770k(not overclocked)
16gb RAM
Intel DZ77GA Mobo
Titan X 12gb GPU
Rimworld is currently installed on a Samsung SSD(EVO)

My Logfile and modlist are attached to this post.

***Added logfile with verbose logging enabled***

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Kenneth on December 19, 2018, 05:35:13 PM
Quote from: Canute on December 19, 2018, 05:23:42 PM
Kenneth,
maybe you should add at your first posting that people should enable Verbose logging at the options.
Then the log contain the time how long Rimworld take for earch step.
To see the difference check this log.
https://gist.github.com/HugsLibRecordKeeper/c82fb0ebec1d2396ca26fa902d701880

You are correct, i edited the first post to include infos on how to enable it. Everyone having this issue, please rerun the game with verbose logging enabled and reupload your logfile.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Zaltys on December 19, 2018, 06:51:10 PM
Updated mine to verbose (previous page). And noticed this in it:
Quote
<...>
- 60ms Load language metadata.
- 5644ms Loading language data: English
- 1329ms Copy all Defs from mods to global databases.
- 9849ms Resolve cross-references between non-implied Defs.
- 204ms Rebind defs (early).
- 390ms Inject selected language data into game data (early pass).
- 1008ms Generate implied Defs (pre-resolve).
- 0ms Resolve cross-references between Defs made by the implied defs.
- 171ms Rebind DefOfs (final).
- 4099ms Other def binding, resetting and global operations (pre-resolve).
- 2055693ms Resolve references.
- 22ms Generate implied Defs (post-resolve).
- 6ms Other def binding, resetting and global operations (post-resolve).
- 4209ms Error check all defs.
- 9ms Load keyboard preferences.
- 429ms Short hash giving.

As you can see, the resolve references part is by far the slowest. There are other things that take a few minutes, but that took 34.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: ambivalence on December 21, 2018, 01:35:29 PM
would be so great to solve it somehow, to run the initialization multithreaded or something. even though it's possible to way half an hour to run the game if you want to play it (half an hour, Karl), it's completely impossible to experiment with the load order of mods, adding something new, bugchecking etc. thanks for trying to solve it.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: eksynn on December 25, 2018, 01:28:39 PM
took me about an hour to load the game with 216 mods in the folder xD; (i was timing it, but i literally fell asleep waiting)
attached are my dxdiag results (system specs), along with the output log (i didn't close the game, so the last few lines might be strange) and my mods list

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Ira671 on December 28, 2018, 06:30:15 PM
CPU:AMD A10-46000 APU WTH RADEON(TM) HD GRAPHICS  2.30GHz
Memory: INSTALLED MEMORY RAM 8.00GB (7.47 GB USABLE)
SSD: HITACHI HTS547575A9E384 (686 GB)
OS:64-BIT OPERATING SYSTEM, x64-BASED PROCESSOR

I am one of those who it takes more than 30 minutes to load into a game (at least the main menu or when I'm done fixing my mods.) Loading to generate a world takes a minute or two. I actually tried to make new games but end up adding more mods which end up making my load take longer.

EDIT: I finally timed it. Takes at least 2-3 hours to load my game. I also don't know how to upload my mod listing if someone can give a step by step process that would be much appreciated. Thank you.

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: NobodyImportant on December 30, 2018, 08:43:27 AM
CPU: Intel Core i5 M480 2.67GHz 2.67Ghz
RAM: 8GB (7.8GB usable)
HDD: HGST HTS541010A9E680 931GB
HDD Read: 0.3-80MB/s
HDD Write: 1-76MB/s
OS: Win7 Home Premium x64
ModsActive: 160
ModsInstalled: 189
ModFolder size: 808MB
Loading time: 30-40 minutes in background, 20-30 minutes when only application and in foreground.
Hugslib Log: https://gist.github.com/HugsLibRecordKeeper/4608b55c359b77d601aacbedab9289c8

The real kicker here is that i used to be able to run Fallout 4 with 4GB RAM and 180 mods that were 20 GB altogether and at the very worst i'd get load times of 5-15 minutes.

Meanwhile the 2D Rimworld with a measly 800MB in 160 mods, is loading for half an hour.


Oh and this part "Please also make sure that you have "Run in background" enabled or keep the game window in focus while its loading. If this option is disabled and you're not having rimworld in the foreground while loading the loading process will stop entirely until you bring it into focus again." is inaccurate.
The loading process will only stop twice without focus. Once right at the start around when the steam overlay activates, and once before building the research tree when memory is being reclaimed although this second one doesn't always happen. Other than these points the game loads basically just as fast in the background as when i am watching those dots blinking.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Canute on December 30, 2018, 09:57:01 AM
QuoteThe loading process will only stop twice without focus. Once right at the start around when the steam overlay activates, and once before building the research tree when memory is being reclaimed although this second one doesn't always happen. Other than these points the game loads basically just as fast in the background as when i am watching those dots blinking.
Maybe the steam version is different.
I modify my desktop shortcut, so it minimize on start.
When i disable the "Run at background" option, the game don't start up. Doesn't matter how long i wait.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: forahilarm on December 30, 2018, 08:47:22 PM
Been meaning to chime in here, but I had dev mode off for the longest time and it pains me to boot this game up at all anymore Lol.

Intel Core i5-6400 @270GHz
SSD
16GB ram
Nvidia GeForce GTX 970

256 Mods installed
230 Mods Active
Load time is about 40 minutes.

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: ozzymandias on January 01, 2019, 04:47:55 AM
Quote from: Canute on December 30, 2018, 09:57:01 AM
QuoteThe loading process will only stop twice without focus. Once right at the start around when the steam overlay activates, and once before building the research tree when memory is being reclaimed although this second one doesn't always happen. Other than these points the game loads basically just as fast in the background as when i am watching those dots blinking.
Maybe the steam version is different.
I modify my desktop shortcut, so it minimize on start.
When i disable the "Run at background" option, the game don't start up. Doesn't matter how long i wait.

I can confirm, I have the steam version and i'm seeing the same thing as he is.


I'm very happy this thread was started as I play a heavily modded rimworld and it's been taking several hours for rimworld to load up to the main screen. It barely takes 2 mins to load with no mods but i've got about 200 running. I'll be posting the information required soon and i hope it'll help with find a solution to this issue.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Tynan on January 03, 2019, 01:24:30 PM
I moved this thread to General since it's fair critical and we want to get as much info as possible. Thanks to all those giving info so far!
Title: Re: Looking for infos regarding long loading times with modded games
Post by: YokoZar on January 04, 2019, 06:17:16 AM
Quote from: Canute on December 19, 2018, 12:44:13 PM
Or maybe "Run at background" only start to work once the game is loaded and a safegame is loaded.
I allready made this suggestion at B18 :-)
I think this would be a better experience in general.  Players aren't expecting to have the game not bother loading when they alt-tab out of the load screen, especially by default.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: rdz1122 on January 04, 2019, 01:47:07 PM
Generally about 10 minutes from cold start or mod restart.  In game seems fine.
Operating System: Windows 10 Home 64-bit
Processor: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz (8 CPUs), ~2.6GHz
Memory: 32768MB RAM
Card name: NVIDIA GeForce GTX 1070
Native Mode: 2560 x 1080(p)

[attachment deleted due to age]
Title: Looking for infos regarding long loading times with modded games
Post by: Irritatedcone on January 04, 2019, 02:46:28 PM
I'll add mine as well.
The logs I'll add are made with "Run in Background" on. (In my case it doesn't change the load times either way.)

Number of active mods: 282
Load time: around 40 min
CPU: i9-9900k
GPU: RTX 2080Ti
RAM: 32GB DDR4 (3000Mhz, CL 14)
SSD: 960 EVO 1TB

Files added:
output_log.txt - debug log with verbose enabled
time-snip.txt - a snippet from output_log.txt with only the Thread 2 load times
DxDiag.txt - system information
modlist.txt - Modlist generated by the mod "[1.0] Modlist"
ModsConfig.xml - Modlist generated by Rimworld

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: da_dark_derp on January 08, 2019, 07:17:46 AM
I'm also experiencing long loading times (fastest one is 20 minutes so far) though usually it's at 30 or more.
Tried with "run in background" on and off. no real difference was noticed.

this is my list with subscribed items: https://steamcommunity.com/sharedfiles/filedetails/?id=1510307437 (https://steamcommunity.com/sharedfiles/filedetails/?id=1510307437)

These are my specs:
CPU: Ryzen 1500x,
GPU: Asus Rog Strix GTX 1060 O6G,
SSD: Samsung Evo 970 NVME 250 gigs,(I tried it on this device, load times were relatively the same(2-3 minutes faster sometimes)),
HDD: WD Blue WD10EZEX, 1TB(this is the device that rimworld currently is on)
RAM: 2x Kingston ValueRAM KVR21N15S8/8,
OS:   Windows 10 Professional.



[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: vale.maio2 on January 08, 2019, 04:50:49 PM
Description:
I've got very long loading times, up to 60+ minutes at one time. As mentioned in this Reddit post (I'm OP) (https://www.reddit.com/r/RimWorld/comments/a9n8ad/save_file_486mb_absurd_loading_time/?st=jqo9xkji&sh=971bf1db), the save file was around 480 MB big. Mod RuntimeGC was telling me I had 56,000 dead pawns somewhere in the world, and the line count for the save file was 19 million. I've cleared them with RuntimeGC and loading time went down to 15 seconds (even tho right now is around 1 minute).

Computer specs:

Mods list:

    Core
    HugsLib
    [SYR] Set Up Camp
    Rimatomics
    [SYR] Prosthetic Table
    Blueprints
    RIMkea
    Mad Skills
    Medical Tab
    Allow Tool
    A Dog Said... Animal Prosthetics
    Glitter Tech
    Tilled Soil
    [KV] RimFridge - 1.0
    Expanded Prosthetics and Organ Engineering
    Conduit Deconstruct
    [T] MoreFloors
    VGP Vegetable Garden
    RuntimeGC
    Dubs Bad Hygiene
    Sometimes Raids Go Wrong
    Real Ruins
    Project RimFactory - Core
    Project RimFactory - Drones
    Project RimFactory - Insanity
    Project RimFactory - Storage
    Gear Up And Go
    Better Pawn Control
    Locks
    Prison Labor

This is my log output (https://gist.github.com/HugsLibRecordKeeper/380665823c5cdf786f245cff4791c881)

I'm not sure I've done the log thing right, if not please let me know.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: 2.71828 on January 09, 2019, 06:29:58 PM
I'll add another piece of info:

Loading times are 20 minutes (±1 m), I'm on (Arch) Linux, or, more precisely:


$ uname -srvmo

Linux 4.20.0-arch1-1-ARCH #1 SMP PREEMPT Mon Dec 24 03:00:40 UTC 2018 x86_64 GNU/Linux


I attached "rimworld_log" from directly after I loaded up my current savefile (that took between 30 and 60s) and unpaused briefly. I only censored my actual username, otherwise the file is unedited. Since I'm also a bit concerned about memory usage, here is the output of "free", from the same time as my "rimworld_log":

$ LANG=en_GB.utf8 free -h

              total        used        free      shared  buff/cache   available
Mem:          7.8Gi       6.6Gi       135Mi       135Mi       1.0Gi       774Mi
Swap:            0B          0B          0B


At the same time, the Xfce taskmanager reported that Rimworld used "3,7 GiB" of memory, and, somehow, "26%" CPU... Since Rimworld tends to get killed by my OOM-killer (usually because I told him so, since Linux's OOM-Killer is pretty much the opposite of aggressive) after (very) long-ish play sessions (today it was after about 2h 15m), because my system gets unusable, I thought I'd report this, too. Maybe that's just how it is, because I use way too many mods, but maybe there's a memory leak, or something? Another thing: Rimworld seems to have some trouble starting up around the time that "BodyDef Squid has BodyPartRecord of SquidBody whose children have more coverage than 1." is written to the log, and it seems to take a few minutes for Rimworld to write any new entries into the log. I have no idea whether that's significant, a mod issue, or whatever, but I thought I'd mention it either way. 

Furthermore, I attached my modlist ("015h.xml") in the form of a "ModList" from Fluffy's Mod Manager. This way you get to see the mod names and Workshop IDs at a glance, and can easily import it, if that's needed.

My hardware, as output by "inxi", shortly after I quit a Rimworld session, so memory is still quite full (I might be a bit paranoid, but I censored my hostname...):

$ inxi -Fx

System:    Host: HOSTNAME Kernel: 4.20.0-arch1-1-ARCH x86_64 bits: 64 compiler: gcc v: 8.2.1 Desktop: N/A
           Distro: Arch Linux
Machine:   Type: Desktop Mobo: ASRock model: H67M-GE/HT serial: <root required> BIOS: American Megatrends
           v: P2.10 date: 04/27/2012
CPU:       Topology: Quad Core model: Intel Core i5-2400 bits: 64 type: MCP arch: Sandy Bridge rev: 7
           L2 cache: 6144 KiB
           flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 24755
           Speed: 1596 MHz min/max: 1600/3400 MHz Core speeds (MHz): 1: 1596 2: 1596 3: 1596 4: 1596
Graphics:  Card-1: Advanced Micro Devices [AMD/ATI] Barts XT [Radeon HD 6870] driver: radeon v: kernel
           bus ID: 01:00.0
           Display: x11 server: X.Org 1.20.3 driver: ati,radeon unloaded: modesetting,vesa
           resolution: 1920x1080~60Hz
           OpenGL: renderer: AMD BARTS (DRM 2.50.0 / 4.20.0-arch1-1-ARCH LLVM 7.0.0) v: 3.3 Mesa 18.3.1
           direct render: Yes
Audio:     Card-1: Intel 6 Series/C200 Series Family High Definition Audio driver: snd_hda_intel v: kernel
           bus ID: 00:1b.0
           Card-2: AMD Barts HDMI Audio [Radeon HD 6790/6850/6870 / 7720 OEM] driver: snd_hda_intel v: kernel
           bus ID: 01:00.1
           Sound Server: ALSA v: k4.20.0-arch1-1-ARCH
Network:   Card-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169 v: kernel port: d000
           bus ID: 07:00.0
           IF: enp7s0 state: up speed: 1000 Mbps duplex: full mac: 00:25:22:bf:37:45
Drives:    HDD Total Size: 1.82 TiB used: 287.00 GiB (15.4%)
           ID-1: /dev/sda vendor: Samsung model: HD154UI size: 1.36 TiB
           ID-2: /dev/sdb vendor: Western Digital model: WD5000AAKX-001CA0 size: 465.76 GiB
Partition: ID-1: / size: 212.25 GiB used: 148.85 GiB (70.1%) fs: ext4 dev: /dev/sdb3
Sensors:   System Temperatures: cpu: 27.0 C mobo: N/A gpu: radeon temp: 34 C
           Fan Speeds (RPM): N/A
Info:      Processes: 181 Uptime: 10h 09m Memory: 7.75 GiB used: 3.59 GiB (46.3%) Init: systemd Compilers:
           gcc: 8.2.1 clang: 7.0.1 Shell: zsh v: 5.6.2 inxi: 3.0.10


[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Kayrah on January 15, 2019, 10:49:24 AM
i run the sk modpack and have loading times around 10min
3.5ghz 8700k
SSD
32gig RAM

the overall performance of the game is pretty poor. It slows down a lot during the game and only reloading will bring it back to normal speed. Not sure its the modpack or the core though.
I wish there was a multicore client support. Its 2019!
Title: Re: Looking for infos regarding long loading times with modded games
Post by: 2.71828 on January 16, 2019, 02:28:04 PM
Quote from: notfood on December 19, 2018, 01:53:15 PM
Kenneth, investigate caching the patched XML. This one works for me to cut 2-3 minutes from my load of 7-10 minutes.

https://gist.github.com/notfood/893b31b1a32b2043d8def1dbbc177a91

Is there any way I could use that bit of code to speed up my loading times? Could I add this as some kind of mini-mod somewhere?

Quote from: Tynan on January 03, 2019, 01:24:30 PM
I moved this thread to General since it's fair critical and we want to get as much info as possible. Thanks to all those giving info so far!

I don't know whether this question makes any sense, but should the loading times be shorter than they are, i.e. are they longer than expected?
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Ira671 on January 21, 2019, 04:18:22 AM
Hello just back again will there be updates to bring down loading times. I still have the same amount of time from when I last post to just get into the main menu.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Zaltys on February 03, 2019, 10:00:09 PM
Already posted my log on page one, but now that the BetterLoading mod is out:

Noticed that it spends a large chunk of the loading time on step 'Reloading DefDatabase: RecipeDef'. That seems to be because I have a lot of race and animal mods, plus patches that inject recipes into those.

So I can imagine the 'allowed' list getting huge for each recipe. Which may be an ineffective way of handling it when most recipes are along the lines of "available for all animals" or "for all humanlikes".
Title: Re: Looking for infos regarding long loading times with modded games
Post by: notfood on February 08, 2019, 03:28:01 PM
Here is another way that halves loading times dramaticaly. Caching the Assembly lookup:

https://github.com/Zetrith/Multiplayer/blob/master/Source/Client/Optimizations.cs#L98-L114

Creadit to Zetrith for finding it.

Basically, RimWorld does reflection every time it finds a type in XML. The more XML custom types the more reflection and reflection is slow. All mods use it. This patch makes it so there is a cache lookup before returing the type.

Many report 2-4 times faster loading.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: 2.71828 on February 08, 2019, 04:13:33 PM
Quote from: notfood on February 08, 2019, 03:28:01 PM
Here is another way that halves loading times dramaticaly. Caching the Assembly lookup:

https://github.com/Zetrith/Multiplayer/blob/master/Source/Client/Optimizations.cs#L98-L114

Creadit to Zetrith for finding it.

Basically, RimWorld does reflection every time it finds a type in XML. The more XML custom types the more reflection and reflection is slow. All mods use it. This patch makes it so there is a cache lookup before returing the type.

Many report 2-4 times faster loading.

So that is what speeds up loading times so much? Do you think these optimisations could be made into their own mod (unless the Devs decide to put this into Rimworld proper)? When I tried Zetrith's Multiplayer mod out, loading times went down from ~25min to less than 10, but I don't need the multiplayer part, and it doesn't play well together with my current modlist/save, anyway, so I can't use it.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: notfood on February 11, 2019, 03:25:01 PM
Another optimization trick:

using Harmony;
using Verse;

namespace FixGamePlz
{
    public class HarmonyPatches : Mod
    {
        public HarmonyPatches (ModContentPack content) : base (content)
        {
            HarmonyInstance.Create ("rimworld.fixgameplz").PatchAll ();
        }
    }

    [HarmonyPatch(typeof(ModMetaData), nameof(ModMetaData.PreviewImagePath), MethodType.Getter)]
    static class HideImages
    {
        static bool Prefix(ref string __result)
        {
            __result = string.Empty;
            return false;
        }
    }
}


Preventing RimWorld from loading the preview images of mods saves a ton of ram, dramatically. It loads the preview files and keeps them in memory even if you aren't using the mods. It explains why my Steam takes so long to load while the SendOwl takes very little time. I have a ton of subscribed workshop mods.

When categorizing the mods for the Multiplayer mod, Parexy had to subscribe to the 2000 available for 1.0, his ram would go up to 6GB! With this patch, his ram on load is 700mb. Dramatic difference!

The ideal solution would be to load those files on demand, not every available mod at start up.

Included the tiny compiled mod.

[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: forahilarm on February 13, 2019, 11:05:39 AM
Quote from: notfood on February 08, 2019, 03:28:01 PM
Here is another way that halves loading times dramaticaly. Caching the Assembly lookup:

https://github.com/Zetrith/Multiplayer/blob/master/Source/Client/Optimizations.cs#L98-L114

Creadit to Zetrith for finding it.

Basically, RimWorld does reflection every time it finds a type in XML. The more XML custom types the more reflection and reflection is slow. All mods use it. This patch makes it so there is a cache lookup before returing the type.

Many report 2-4 times faster loading.

I can confirm that my loading times drastically decrease with this, however, I've found that my world generation junks up pretty badly with my other 200 mods. Is there possibly a way to yank out that code and make it a mod in of itself for the purpose of self caching?
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Ronburgundy on February 15, 2019, 08:15:10 AM
My game with mods loads fine till this point then it sits here for  30+ minutes
here is the config any help would be helpful.
https://pastebin.com/HAnMwSAc (https://pastebin.com/HAnMwSAc)
(https://i.imgur.com/SDdDfGB.png)
Title: Re: Looking for infos regarding long loading times with modded games
Post by: 2.71828 on February 16, 2019, 03:09:19 PM
Quote from: forahilarm on February 13, 2019, 11:05:39 AM

[...]

I can confirm that my loading times drastically decrease with this, however, I've found that my world generation junks up pretty badly with my other 200 mods. Is there possibly a way to yank out that code and make it a mod in of itself for the purpose of self caching?

I guess it's not that easy, since I already asked for it, as well, and notfood hasn't reacted to that, either. This is of course understandable, since that is neither her mod, nor should it be her that has to fix the game, because that's what this would basically be, if I understand it correctly, i.e. a bugfix mod.

Additionally, none of the Devs (especially Kenneth, who has created this thread), have reacted to any of the three (!) possibilities to speed up loading times that notfood proposed, and Kenneth hasn't been seen in this thread since December 19, although Tynan himself has written that this thread is "fairly critical", because they want to fix this themselves. I guess they're busy doing something else, but it would still be nice if one of the Devs dropped in and maybe told us whether the info that gets posted here is even helpful to them, whether we are supposed to continue posting logs (even if we have posted one already), etc. No one, except myself, has looked at my log and my modlist, by the way, assuming the "downloaded x times" thingy is counting correctly.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Irritatedcone on February 21, 2019, 03:00:43 PM
Quote from: 2.71828 on February 16, 2019, 03:09:19 PM

I guess it's not that easy, since I already asked for it, as well, and notfood hasn't reacted to that, either. This is of course understandable, since that is neither her mod, nor should it be her that has to fix the game, because that's what this would basically be, if I understand it correctly, i.e. a bugfix mod.

Additionally, none of the Devs (especially Kenneth, who has created this thread), have reacted to any of the three (!) possibilities to speed up loading times that notfood proposed, and Kenneth hasn't been seen in this thread since December 19, although Tynan himself has written that this thread is "fairly critical", because they want to fix this themselves. I guess they're busy doing something else, but it would still be nice if one of the Devs dropped in and maybe told us whether the info that gets posted here is even helpful to them, whether we are supposed to continue posting logs (even if we have posted one already), etc. No one, except myself, has looked at my log and my modlist, by the way, assuming the "downloaded x times" thingy is counting correctly.

I noticed that too, I haven't seen any dev activity here nor any file downloads that weren't done by me :(
Managing to fix this would increase my enjoyment a lot (and give me about an hour extra playtime lol)
Currently I have to pretty much plan when to play Rimworld instead of just launching it when I feel like it.

Please give us an update - even if it's just a simple "working on it".
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Trentennisfigati on March 08, 2019, 04:55:51 PM
Hello, this is my modlist. https://pastebin.com/dRrH66WJ
My load times are extreme. I've tried on several machines and and the wait is around one hour.
I think some mod might cause ram or cpu issue, I havent figured out yet.
Also to me RecipeDef 94 is a point where the game loads for a lot.
The game has no big red errors and is somewhat stable, but at some points with bigger events freezes to death.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Jonchan1997 on April 07, 2019, 11:53:37 PM
I went ahead and put nofood's github snippet into my vs 2017 and built it with zero errors using .NET Framwork 3.5 and the latest version of harmony. Hopefully this helps i am going to recompile in the other framworks and test it when I have more time.
https://drive.google.com/file/d/1KTYTRlsCvDme9Qmn9rszUlnnlcNYDnL_/view?usp=sharing
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Jonchan1997 on April 13, 2019, 08:25:00 PM
Another solution is use Nirsoft.net produkey program to grab your pcs product key, then buy a samsung portable ssd t5 from amazon and use wintousb by haslo to put win 10 on it thru windows installion media iso from Microsoft.com(free downloads) from there should your ram suffice you can boot from that drive in legacy mode and put just your steam program on it.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Xaviien on April 19, 2019, 10:42:52 PM
Quote from: Jonchan1997 on April 13, 2019, 08:25:00 PM
Another solution is use Nirsoft.net produkey program to grab your pcs product key, then buy a samsung portable ssd t5 from amazon and use wintousb by haslo to put win 10 on it thru windows installion media iso from Microsoft.com(free downloads) from there should your ram suffice you can boot from that drive in legacy mode and put just your steam program on it.

Was that just a really awkward way of saying run Windows and Steam on SSD?
Title: Re: Looking for infos regarding long loading times with modded games
Post by: TastyCookies on April 23, 2019, 04:19:31 PM
Quote from: Ronburgundy on February 15, 2019, 08:15:10 AM
My game with mods loads fine till this point then it sits here for  30+ minutes
here is the config any help would be helpful.
https://pastebin.com/HAnMwSAc (https://pastebin.com/HAnMwSAc)
(https://i.imgur.com/SDdDfGB.png)
I also want to point out that I too experience that part of the loading time. Once the game reloads the recipe def database at 94, the loading screen takes 30+ minutes for it to progress. I have a total of 241 mods installed. Total load time was approximately 51 minutes.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Moeppel on May 05, 2019, 11:49:37 AM
Log file is attached.

1700X (undervolted, stock frequencies)
16gb DDR4 3200CL14
SSD
Antergos, Linux

Load time(s) approx. 20 minutes.

Haven't attempted a Windows install to make OS comparisons.

259 active Mods:

https://pastebin.com/z5UpMVdm


[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Nuclearmayhem on May 06, 2019, 02:13:28 PM
All these people complaining about 30 min load times :)  I wish my game loaded in 30 mins.

Rough estimate of my load times based on the last time I actually sat by the computer.  4 hours.

Mind you, I'm trying to run with 630 various mods.  I like variety in my game.  If I do just people/animal mods I load quick (well under an hour), if I do just items mods also fairly quick.  Combine the two together I'm loading for hours.

Laggy to play some times but usually stable.   Running steam version, really thinking of getting a stand alone version to load on a second pc, so that if one crashes I can switch to the other while the first PC reloads.


[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: legendary on May 21, 2019, 02:50:34 AM
Quote from: Nuclearmayhem on May 06, 2019, 02:13:28 PM
All these people complaining about 30 min load times :)  I wish my game loaded in 30 mins.

Rough estimate of my load times based on the last time I actually sat by the computer.  4 hours.

Mind you, I'm trying to run with 630 various mods.  I like variety in my game.  If I do just people/animal mods I load quick (well under an hour), if I do just items mods also fairly quick.  Combine the two together I'm loading for hours.

Laggy to play some times but usually stable.   Running steam version, really thinking of getting a stand alone version to load on a second pc, so that if one crashes I can switch to the other while the first PC reloads.

This is a bit unbelievable if you mind. Please provide your modlist. So many mods can't be compatible and not overlap. Simply impossible (?)
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Nuclearmayhem on June 05, 2019, 04:18:53 PM
Quote from: legendary on May 21, 2019, 02:50:34 AM
This is a bit unbelievable if you mind. Please provide your modlist. So many mods can't be compatible and not overlap. Simply impossible (?)

See attached for mod list.  exported from mod manager  I've cut down on some of mods to get me below 600 (I had a lot of gun mods and decided I really didn't need all of them)  Not counting Core, 585 mods in this list.

It was working with only some errors.  It had some issues with minions chemdef, werewolves not having a peaceful option and some warnings about Adeptus Mechanicus Magos possibly not having weapons due to money limits.

However as of a couple days ago the whole things been borked and my list will reset.  I started to rebuild, got 130+ mods in and something changed and borked again.

When I manage to get a full list going again that works, I think I'm going to make copies of the mods and use my DRM free version so that auto updates on steam don't bork things.

loading time with the new unstable cut down loading to 1 hour 20 mins, and most of that is from Research Pal doing its thing.


[attachment deleted due to age]
Title: Re: Looking for infos regarding long loading times with modded games
Post by: NobodyImportant on August 10, 2019, 12:53:01 AM
Just confirming that indeed the game still loads 800MB of mods in about half an hour.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Kaelen Mosar on November 10, 2019, 02:34:01 PM
Do you recently recive a lot of reports that this game is having a lagspike with a few mods like every 4-5 seconds on max gamespeed, the slower game speed the more time between the spikes. reproducable on different machines with same specs, reproducable on different spec machines.
Steam discussions and comments are full of guys talking about the same problem. all since 1.0 it seems?
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Tynan on November 10, 2019, 08:07:16 PM
Quote from: Kaelen Mosar on November 10, 2019, 02:34:01 PM
Do you recently recive a lot of reports that this game is having a lagspike with a few mods like every 4-5 seconds on max gamespeed, the slower game speed the more time between the spikes. reproducable on different machines with same specs, reproducable on different spec machines.
Steam discussions and comments are full of guys talking about the same problem. all since 1.0 it seems?

Discussed here: https://ludeon.com/forums/index.php?topic=49133.0
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Diana Winters on February 26, 2020, 02:33:17 PM
Ever thought about handling mod lists the same way paradox does? Have a separate menu that launches before the game is loaded where you can select what mods you want to have loaded. That would definitely reduce the need for modifying the config files so I don't need to launch my game, wait for it to load, then re-launch it to add a single mod.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Tynan on February 26, 2020, 07:17:49 PM
Thought about it; given the technical challenges it won't happen any time soon.
Title: Re: Looking for infos regarding long loading times with modded games
Post by: Canute on February 27, 2020, 03:17:12 AM
Someone made an external Modmanager tool for such thing.
RimPy
https://ludeon.com/forums/index.php?topic=49371.0
But not working for 1.1 i think.
Because he wrote
https://ludeon.com/forums/index.php?topic=50482