Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - hoho

#1
QuoteLoading screen now displays present and active expansions and mods.
Is this a hint of what is coming down the line?
#2
General Discussion / Re: Steam Trolls!
July 18, 2016, 04:54:59 PM
Quote from: Nareese on July 15, 2016, 08:46:26 PM
Quote from: Elixiar on July 15, 2016, 06:11:17 PM
The early reviews for this game are impressive.
They are in fact so impressive that people think they are fake. 
Exact same thing happened a few months ago when Factorio came to Steam and was the #1 top rated game for a few weeks because of massively positive feedback.

There will be people who get Rimworld just because it's rated high, try it and see that it's not for them. Rimworld isn't meant to be loved by everyone. Pretty much no game is.
#3
I'm almost certain prepare carefully won't be added to game (unless as a developer-mode) but interface should definitely belong in vanilla. I've been playing on a13 for a few hours and not having the portraits makes it rather tedious.
#4
Ideas / Re: potatoe power
April 04, 2016, 09:21:21 AM
Might be useful as a temporary power source for some low-powered light. A handful of potatoes should be enough to keep a bulb on for a while.
#5
Ideas / Re: Multi-core support
February 19, 2016, 06:04:38 AM
"Multi number parameters" could be having to check for all body parts for injuries-upgrades to figure out how fast someone runs, having to check for positions of animals to figure out who to hunt, having to check objects in path of a bullet to figure out where it hits etc.

Another HUGE issue with multithreading is when objects can be removed or created at pretty much random. For example, if one is applying medicine to another character and the char dies in the middle of it (and the data structures describing their health go with it), the one doing the healing *must* know that the person died before it attempts to do anything with the data of the character they were healing.

Essentially, even if one thread wants to only read data that is managed by another thread it's not enough to just check if the data exists to start reading it. It must be enforced that the data doesn't get deleted before all the reading is complete. Trying to read data that has been deleted generally results in application crashing. Same with trying to write to structures that have been deleted. Only protection against such crashes is, again, locking them leading to bottlenecks.


Yet another issue with threads are deadlocks. Deadlock is what happens when you have a circular dependency between locks and threads. Here is a simple example code demonstrating deadlocks: http://stackoverflow.com/a/1385876 That sync() call there is locking the variable. If a deadlock occurs, the application has hanged - it can't proceed further as two (or more) threads are stuck waiting after each other and none can proceed.

Avoiding deadlocks is by far the hardest thing in threading.
#6
Ideas / Re: Multi-core support
February 19, 2016, 02:56:19 AM
Quote from: Vas on February 18, 2016, 03:37:27 PMI assumed that if it can upload a thing to memory, the thread may lock something but the memory value can be read at any time.
Technically, you are correct - everything can be read at any time.

Problems emerge when some parameters are more complex than the simpliest variables (single integer or floating point value, boolean). In those cases, it's likely that when someone is changing that multi-number parameter then the one reading it gets first half from old state and second half from new state. In those cases, all write *and* read accesses must check for the locks.

Also, the locking doesn't work automatically. Programmer has to specifically write stuff like:

check if lock is already locked by someone else
If yes, put the thread to sleep with wake condition of "this lock is released"*
If not, lock it yourself
change or read the variable
remove the lock

If theread does happen to get thrown off the core while it's in sleeping state then that alone will take hundreds of CPU cycles, getting it back there again costs hundreds to move data, thousands if some data was thrown out of caches completely.

Quote from: Vas on February 18, 2016, 03:37:27 PMNo one here gives a shit about my original intentions of posting here.
"Problem" is, everyone with half a brain knows that hypothetically, multithreading can make things faster. There aren't much reason to point that out as I'm quite certain Tynan is fully aware of it already.

As has been said here, multithreading can at best increase speed lineary by at best 2-3x. Optimizing algorithms will often give much more performance than that and will make the game scale vastly better. It makes MUCH more sense to optimize algorithms than spending huge amounts of time to make the game run in threads.
Quote from: Vas on February 18, 2016, 03:37:27 PMIts difficult, but possible, and most games these days support it.
I can assure you, almost every AAA game out there is *not* parallelizing their core logic to run in more than one thread.

What they probably are doing is separating things out by tasks. E.g rendering in one thread, physics in other, preparing data for rendering in third, AI in fourth etc. They also usually use third-party libraries for some tasks, especially physics, and those libraries are often internally multithreaded. As most of those games are rather graphics-intensive, the threads dealing with rendering and preparing data for rendering take a LOT of computing time so spreading them out to different threads makes sense.

Rimworld is essentially nothing like those games. Its rendering is trivial (I'd think it could run at several thousand FPS if we'd turn off all other processing) and it has no real physics. Almost all the computing time is taken by the "AI" and general game logic and it's extremely hard to split that to run in several threads. In those AAA games, that part of the game barely takes a couple dozen percentages from a single core to run.
#7
Ideas / Re: Multi-core support
February 15, 2016, 04:50:19 PM
Quote from: Vas on February 14, 2016, 02:41:40 PMYou do realize the game doesn't need to wait for any reason to get a temp right?  Temp can entirely be separated into a different thread, nothing needs to wait on it.  Heat/Cold pushers while powered, will be processed by the temperature thread.
So, when will food on ground be updated depending on ... temperature? What about feelings of pawns and animals? What about plant growth? When will the rooms decide how to change their temperature depending on heat sources and heat "bleeding" through walls? All these things depend on each other at *some* point and *all* these points have to be synchronized between threads.

Quote from: Vas on February 14, 2016, 02:41:40 PMPath finding can also be separated I believe, and path finding is quite the intensive event
I have no idea how the pathfinding has been implemented in this game but I do know that on 2d grid, there are some extremely efficient algorithms for doing it. Again, look up Factorio and how they optimized their pathfinder by several orders of magnitude. Having thousands of *actively moving* mobs on a map size of several million squares doesn't really make a dent in game performance there and *everything* but rendering is running in a single thread in that game.
[/quote]It is. One can do thousands of calculations when thread hits a mutex lock, millions if that thread gets thrown off the core to pull in another and that doesn't even take into account the time it takes to propagate mutex changes between cores.

Quote from: Vas on February 14, 2016, 02:41:40 PM
Thread 1 (main thread) : Pawn starts cleaning? == True
Thread 1: Push to Thread 4
Thread 4: Path find to nearest dirt spot.
Thread 4: Clean spot.
Thread 4: Loop 10x times.
Thread 4: Check if pawn needs to change jobs.  If False, loop 10x times.  If True, push to Thread 1.
"Push to thread 4" - what exactly does that mean? You mean main thread builds a "command" for the "clean dirt" handling thread, adds it to some global queue (that can be modified by a single thread at any time, anyone wanting to add/remove stuff from it *must* wait until previous actions are complete) and sends a message to the thread 4 that it has some work to do?

Sure, it can be done like that. Only problem is that the cleaning thread, as you described, does a ton of stuff that depends on all sorts of external variables that can be changed at any point and the job it does is spread over several seconds and several dozen "ticks" during each everything can change (e.g the pawn gets hit by a bullet). Also, that would mean you *can't* separate pathfinding to a new, separate thread.

Also, that "push to thread 1" at the end seems rather weird. Do you mean that it gives over the CPU for the main thread to run? I sure hope not as that's nothing like multithreading works in reality.
Quote from: Vas on February 14, 2016, 02:41:40 PMYou explain why x feature can or can't be done, and you leave it at that.
Sadly, you are trying to offer ideas that are, well, useless. Parallel programming doesn't seem to be a topic you know much about.

Quote from: Vas on February 14, 2016, 02:41:40 PMI've had large raider spawns land on my map, mostly Tribals, that do it, causing massive lag as they all path find to their idle spot, then the lag settles slightly, then starts up again when they start path finding to my elaborate base setup
So, what makes you think it's pathfinding instead of the pawns updating their mental states to changing surroundings?
Quote from: Vas on February 14, 2016, 02:41:40 PMIf an object in the game needs to check the temp that is using Thread 1, and Thread 6 is handling temperatures.  Thread 6 does the temp calculations and sets the temp of a room into memory so that any Thread 1,2,3,4,5,7,8 object can check it any time
... and every time something tries to read that temp value, they need to check if mutex protecting it is locked or not.


Quote from: Vas on February 14, 2016, 02:41:40 PMIf such object relies on power, the power thread, thread 4 (for example) can keep a list of powered objects in memory, and alter their state like that so thread 6 simply needs to check that Item624722 Power == 1 before processing it's heat/cold pusher stats.
So, powered items have to additionally check if they are allowed to access the powered state before even beginning their heat calculations. That happens for *every single powered item* that produces heat.

I assume the "simultaneously" there meant that threads 1, 4 and 6 are ran in parallel and not that each thread is somehow duplicated.
- Thread 1: Pawn Jobs (+30 more lines)

What does that "+30 more lines" mean? That thread 1 handles pawn jobs? What does "handles a job" mean, exactly? That it takes into account the pawn mental state, resources availiable and jobs availiable, pathfinds to gather stuff up and do whatever else is required?

- Thread 1: Countdowns for story teller events (+5 lines)

I'm almost certain that story teller stuff is triggered at best once a second and takes less than 1000 CPU cycles to run. I wouldn't be surprised if it's usually even much less than that. In other words, I believe storyteller takes almost no CPU power at all.

- Thread 1: Set memory values for various things so other threads can access quickly (+12 lines)

Will you lock each element individually or everything together during the time they get updated?
If you do it individually it would mean a ton of mutexes are required (yay increased cache load) and everything has to checked every time things are accessed. Though at least you can have one thread handling object N in the array while another thread handles object N+1.

When locking everyhting you'll make everything stop while those updates are made but at least you won't have to worry about checking for each object mutex and can just assume that if the object array is usable, all are. Less mutexes at cost of less granularity. Though I'm extremely doubtful that object-level mutexes would make much sense in this game.

Thread 4, as you described, seems to handle a single powered element. I would guess that would take just a few hundred CPU cycles without accounting for mutexes or locking. Throwing that sort of thing to a separate thread makes little to no sense.


I highly doubt separating electricity handling to a separate thread would make sense. Rimworld has *significantly* simpler power networks than Factorio and it's clearly possible to optimize them to take neglible CPU time in single thread. I wouldn't be surprised if hitting just a single mutex in there would slow it down by 2x vs having it run in main thread.


- Thread 6: Check Item624722 power state - assuming per-object locking it would mean checking for that lock before checking for it.
- Thread 6: Item624722 pushes 16 units of heat to Room662; - meaning it has to lock the data structure where it writes to


Quote from: Vas on February 14, 2016, 02:41:40 PMI'm just, making a rough guess here for this stuff
That you do :)

Most games don't even attempt to separate threads by task-types all that much. Sure, input, rendering, sound and physics are generally
each in a different thread but each one of them has drastically different CPU requirements and that alone rarely has much impact on performance. I've yet to hear any game getting anywhere near 2x speedup from just that going from single core to any number of cores you throw at it - Amdahl's law.

What most games do is to figure out what *actually* takes CPU time and parallelize that. Usually it's physics and often preparing data to be sent for rendering on GPU. In rimworld, neither is anyhwere near taxing on CPU (or GPU). Not knowing the details of inner workings of Rimworld it's really hard to make suggestions on how to parallelize it successfully but I can say for a fact that the scenarios you proposed would be rather inefficient for any type of game if the aim is to gain actual performance increase.

Quote from: Vas on February 14, 2016, 02:41:40 PMBut Power Net (thread 4) uploads the state of the power grid for each item to a memory value, while temperature (thread 6) checks those states before executing the heat/cold pushing power of that thing and altering the temperature before uploading the temperature of that room to memory both doing this simultaneously and synchronization doesn't matter here because the temperature from the previous check 1 tick ago is still there and it is doubtful to have changed dramatically enough in a single tick to matter if it checked 1 nanosecond too soon
Sure, double-buffering can work in some scenarios. Problems emerge if you have feedback loops and if several things can effect same things - depending on what sequence threads get executed you can get wildly varying results. Not to mention that you'll have to synchronize everything at every tick/frame anyway. Again, Amdahl's law hits hard here as everything will be running at the speed of the slowest part.


Adding in multithreading to anything that was written without having that in mind from the early designs is pretty close to insanity. The ones that have done that have generally ended up rewriting almost entirety of their codebase. Again, there is a reason why Kerbal Space Program took *years* to update to Unity 5.

In addition, it's always preferred to first optimize the algorithms as much as possible. It helps both single-threaded and multi-threaded cases and while parallelizing over infinite threads usually doesn't give more than 2x speedup in games, optimizing algorithms tends to often give exponential speedups.

For example, if one were to optimize whatever it is that makes huge swarms slow to run at (being optimistic here) 3x the speed over 8 cores, fine, you'll get 30FPS instead of 10 you had before. Now, if one manages to replace a quadratic algorithm with linear or even constant speed one, you can essentially increase the swarm size by several orders of magnitude long before you'll even feel any slowdowns.

Sadly, without knowing where are the bottlenecks in Rimworld it's impossible to even guess what might benefit from optimization or if parallelizing it would help at all.


Rant on Factorio:
QuoteAgain, looking at Factorio, my current factory is just a beginning one with lousy 5000 solar panels, 2000 accumulators (think batteries) and ~100 coal fired power plants. They power ~500 machines + ~10000 inserters (things that move items from one place to another, I use bullet-based turrets and each requires one inserter, I essentially have a wall of them around my base and have killed over 300,000 enemy units).

The game has no problems running at 60 updates per second. According to the debug screen, electric network takes about 1/10'th of entire update per-tick and at less than 1ms. I'm producing on averaeg about 20,000 items per minute or ~180 per second. For almost every item, inserter has to move it from one place to another and majority of items move on conveyor belts hundreds of tiles long. My pollution cloud covers around 5 million squares (and gets absorbed/moves/spreads according to forests/winds) and influences the actions of all the mobs in it's area (though pollution gets calculated by chunks, each 256 squares). Overall I've explored an area of around a billion squares (equivalent of Rimworld map size of ~32,000 times 32,000) = thousands of enemy spawners generating mobs to throw at me as pollution hits them or just wander around idling (but still requiring path finding).

In addition to all that, the game is *massively* modded with mods written in Lua - zero compiled code in mods as there is in Rimworld. The game does have 2 threads but one of them is purely for rendering while the other deals with all the heavy-lifting.

In other words, it's perfectly possible to optimize a game to be able to handle huge scale and loads of entities/pawns without too much problem. Obviously, it's not easy. Those guys have had several optimization passes, each bringing significant increase to the size of factory one can make before it slows down too much.


Also note that my factory is tiny compared to what some people have created after spending over thousand hour building on one. I know some require nearly an hour of real-time to travel between the mining outposts at opposite sides of the map using an in-game train


TL;DR

It is possible to multithread Rimworld. I don't think it's required nor do I think it'd give a measurable performance increase. I'm absolutely positive that there are MASSIVE gains to be had from improving the algorithms that are currently used.

Also, sorry if I sound harsh. It's just that your suggestions aren't really helping (you can be quite certain that Tynan knows *much* better than any of us what and how would make sense to parallelize, if anything) and make claims about parallelization that simply don't work in real world.
#8
Ideas / Re: Multi-core support
February 12, 2016, 02:34:40 AM
Quote from: lude on February 05, 2016, 01:39:37 PMYeah, like the things propagating per cell is very elaborate, compared to simulators like DF2, I just upped my CPU to 4.2ghz with an FSB of 240, the "mysterious IPC" is more affected by FSB increase than by multiplier, keeping the other busses at their vanilla speeds with an increased FSB netted me very nice results with most single core limited games
That seems to point to the game/mods being inefficient in their data layout in memory. Increasing FSB lowers memory latency and thus helps in scenarios where either precaching doesn't work as well or where caches get trashed.

Quote from: Vas on February 06, 2016, 10:17:00 AM
Quote from: Ectoplasm on February 05, 2016, 11:20:09 AM
Quote from: Vas on February 05, 2016, 10:15:51 AM
All game developers should be doing multi-core support by default in their game engines. 
To be fair, when Rimworld started this wasn't an option in Unity. Rimworld will be 3 in November. I think you need to consider this legacy, multi core just wasn't there in Unity.
Yes, but other engines had it and 64 bit too.
At the time, there pretty much were no other engines availiable for the price of Unity.

Also, 64bit support is trivial compared to multithreaded stuff. Pretty much the only thing you need to look out for is doing pointer arithmetic.

Also, there are scenarios where 64bit instead of 32bit computing causes performance losses.

QuoteBy the way, as stated before some things in this game CAN be offloaded to other cores.  Path finding, for example.
I'm almost certain that pathfinding in this game is one of the least expensive algorithms, unless awfully inefficient algorithms are being used right now. Maps are relatively small and there aren't many pawns running around. No, 100 isn't many for half-decent pathfinders on a 500k square area.

Again, bringing Factorio as an example, they have infinite map sizes, kind of like Minecraft - their size is limited to how far you've explored. They improved their pathfinding speed by several orders of magnitude by tuning the algorithms.

QuoteAlso, temperature calculations can be offloaded to another core in order to speed up temp checks.
As I said earlier, those checks aren't independent and depend on other things leading to having to synchronize things leading to nowhere near as fast speedups.

For example, if temperature check has to alter parameters of something it means nothing else can access that something during the operation. The data of the "something" has to be locked, modified and unlocked. Every single time anything else access that object, that lock has to be checked no matter if it is locked or not in reality. It means often times the locked status has to be transferred from one core to another, through RAM in worst-case scenarios. If something does have it locked then everyone else are just dead in the water busylooping and checking for when the lock is released. In other words, everything else waiting behind the lock are wasting CPU time while doing nothing useful.

QuoteI was saying that had I been the one choosing a game engine, I would have chosen one that had multi-core support to begin with, as well as 64 bit support.
Can you point to an engine from 3y ago that supported those things and didn't cost half a million?
Quoteand UNITY, should have been putting support for this in to begin with
Considering it took two YEARS for KSP to adapt to unity5, how long did you think unity devs had to spend to make their engine to support multithreading? IIRC, they were essentially forced to rewrite almost everything.


For the record, I'm playing the game on a CPU from 2007, Core 2 Quad 6600. A 2.4GHz CPU. The game runs roughly as fast as on my laptop with the latest I7 running at 2.8GHz. In pretty much every synthetic test, the laptop CPU is almost twice the speed of the older CPU in single-threaded loads. This, again, hints to me that isses are in algorithms and/or data layout. Most likely it's not the game but the mods used. They are probably trashing caches and thus depending heavily on memory latency. Those two are almost equal between the two PCs I have.


Last time I had any serious issues was when I was using zombie apocalypse, aparello and something that modified weather calculations. Problem was that first mod added huge waves (seeing >100 zombies was common), second added a TON of clothes that are *actively* worn (leading to several times more calculations to be made for temperature/combat and most likely a lot of other things) and last one simply made weather calculations take longer. Each mod individually probably wouldn't have made much of a problem but combined they increased some calculation requirements by at least an order of magnitude. In addition, they didn't add a constant scaling but exponential.
#9
Ideas / Re: Multi-core support
February 12, 2016, 02:08:29 AM
Quote from: Vas on February 04, 2016, 08:09:01 AM
Quote from: hoho on January 27, 2016, 05:08:57 AMI'd be extremely surprised if Rimworld running on 4core/8virtual core CPU would get 2x faster than running singlethreaded.
How exactly do you come to this conclusion?
Experience from writing software running in parallel. I'm almost certain there is some "main" thread that runs stuff that can't be parallelized effectively and will limit the overall speed the game can run:
https://en.wikipedia.org/wiki/Amdahl%27s_law

Quote from: Vas on February 04, 2016, 08:09:01 AM
MultiCore/Thread: Game is able to offload some work to different threads.
Emphasis on "some". 
Quote from: Vas on February 04, 2016, 08:09:01 AM
Thread 1 handles temperature changes.  Thread 2 handles pawn hauling jobs.  Thread 3 handles Pawn Cleaning jobs including snow and calculates whee they should go next each job.  Thread 4 handles pawn hunger and tiredness levels, beauty calculations and other such.  Thread 5 handles mental levels and thought defs.  Thread 6 handles power grid states.  Thread 7 handles random event calculations.  Thread 8 handles Animals and Plants on the map.  Game runs at normal/fast speeds because there is nothing waiting in line to be executed.
Nice hypothesis. Too bad all these things are depending on each other meaning there is tons of synchronization and waiting for intermediate results between threads meaning it that parallelization won't give anywhere near linear increase in speed.

Quote from: Vas on February 04, 2016, 08:09:01 AM
It really sucks everyone here has taken up to trolling the community for requesting multi-core support all the time.
Most people I've seen "trollin" here are people that actually know a bit or two about programming.

Quote from: Vas on February 04, 2016, 08:09:01 AM
Even he must know by now his game has too much data to process on a single thread
I *highly* doubt that. Take Factorio - it has little to no problems handling hundreds of thousands of objects in-game at 60 updates per second without multithreading. All it takes is to use efficient algorithms, possibly with redesigning one's data structures to match those algorithms. As simple things as how variables in a structure are laid out can mean massive speed difference due to trashing of L1 cache, having not-so-good data structures (e.g linked lists instead of arrays/vectors) can mean you'll break the processor precaching leading to it waiting for hundreds of cycles for data to arrive just to spend a dozen cycles to do the math on it.
#10
Ideas / Re: Multi-core support
January 27, 2016, 05:08:57 AM
Quote from: Nickname34 on January 19, 2016, 12:43:34 PMthe cause is the non-multi-core support engine.
How do you know this to be the root cause and not e.g algorithms that simply don't scale well?

Not to mention that going multicore rarely, if ever, gives linear increase in games. I'd be extremely surprised if Rimworld running on 4core/8virtual core CPU would get 2x faster than running singlethreaded.
#11
Ideas / Re: Water Need!
January 27, 2016, 05:05:20 AM
I like the idea in general but I'm afraid it'll mean colonists will be spending a ton of time just staying alive (running after food/water and sleeping) instead of doing stuff.

Obviously, if plumbing was added it would make things easier. Alternatively, just make days last longer while keeping the per-day food and sleep usage the same so more stuff can be done within a day.
#12
Mods / Re: [Mod Request] Harvest organs dead cropse
December 18, 2014, 05:37:34 AM
I approve of this idea.

Especially how I felt it's weird how I couldn't harvest any other organs from my subjects after I had taken out their heart or both lungs. Sure, they'd die in minutes but they are already on the table with their body opened up and all the machinery ready for storing their parts. In that state I should be able to harvest EVERYTHING they have.