Ludeon Forums

Ludeon Forums

  • November 30, 2022, 03:07:07 AM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - Nightinggale

Pages: [1]
1
Bugs / [1.0] [fixed] modding patch loading is no longer possible
« on: June 18, 2018, 10:19:01 AM »
EDIT: build 1939 added full mod support to the code in question, meaning everything mentioned here can now be modded.

I'm not 100% sure where to post this, but I ended up calling it a bug because it works in B18 and it's most likely not intentional.

The issue is Verse.LoadedModManager.LoadAllActiveMods(). In B18, it loads xml def files and apply patches to each before merging. In 1.0, it load def files, merge and then patch. Apparently this is done purely for performance reasons and it will likely patch noteworthy faster when using vanilla operations only.

The problem is that it clashes big time with ModCheck. It's not about the need to update the mod, more like the whole concept is now locked out from working at all regardless of modded code.

What ModCheck does in B18 is use Harmony to patch Verse.ModContentPack.LoadDefs(), which was called for each mod from Verse.LoadedModManager.LoadAllActiveMods(). Here it calls the singleton in ModCheck to inform modname, name of file being patched as well as start and stop for applying each patch. No change to the code, just reading more data.

This is no longer possible. Verse.ModContentPack.LoadDefs() no longer handles patching. Instead patching is done inside Verse.LoadedModManager.LoadAllActiveMods(). ModCheck use an instantiation of a Mod type class to start Harmony. This is done from LoadAllActiveMods() and Harmony doesn't patch methods already in the stack, or at least I couldn't get it to work. However LoadDefs() is called after applying Harmony patching, meaning it's fully patchable with Harmony.

ModCheck aims to speed up patching and profiling has told me that XPATH searching is by far the dominant part concerning time spend patching. ModCheck solves this with ModCheck.FindFile where the patch creator adds two strings, modname and filename. ModCheck will then compare those against strings already in memory and only return true if both are true. This way it's possible to write a patch file, which only makes an XPATH search on a single file and it discards wrong files virtually instantly. Some mods are full of small files, like adding 50 animals with one animal in each file. Not only will FindFile result in only using XPATH once, it will do so on a file with just a single def.

Another major feature is profiling. ModCheck is able to measure time spend on each root PatchOperation. Downloading a bunch of mods from steam and profiling them has revealed that there are patches there, which takes a few seconds per root PatchOperation when loading a lot of mods. The ability to locate such slow patch files is the first step in getting them up to speed.

Another lost feature is log writing, particularly the 5 arguments, which are quite useful when generating error messages, both for ModCheck and for patch creators.

Now what?
If ModCheck would be rendered obsolete by an update in RimWorld, then I wouldn't mind. The problem is the vanilla change is most likely slower than proper use of ModCheck and a number of ModCheck features are lost without a replacement. For users of ModCheck, this is a major step backward.

I propose two changes:
One is to split Verse.LoadedModManager.LoadAllActiveMods() into two methods. Move everything after the foreach (Type type in typeof(Mod).InstantiableDescendantsAndSelf()) loop to a new method and end LoadAllActiveMods() by calling the new method. This way the new method will be compatible with Harmony and some functionality can be restored (particularly profiling).

The other change would be to restore patching ability to Verse.ModContentPack.LoadDefs(). Not as a replacement to the current 1.0 system, but rather it should load patches from a different folder. This way people can use the current patch folder or knowingly relying on FindFile, they can revert to the old approach. This part might be possible to add using Harmony, but I aim to make ModCheck as little intrusive as possible on vanilla patch loading and as such I prefer to just read variables and not actually change functionality in vanilla loading.

2
When loading mods, if the same DLL file exist in multiple mods, the one from the first mod in the loaded mod list will be used. This is a problem for tools, which are distributed as a DLL file to add to the mod itself (ModCheck, Harmony etc) because if a new version is released, mods can't use the new features because then it will fail, possibly even crash if a user happens to to load a mod, which didn't update the DLL. The solution would be to not include the mod in multiple mods, but then the game crash if the user doesn't include the DLL mod, or include it in the wrong order.

The problem can be avoided if all mods always update to include the newest DLL. However why should a mod update if it doesn't use the newly added feature? This mean the mod creator who has to do the work will not be the one making the DLL or the one whose mod breaks due to lack of update.

Obviously this is a big problem for any DLL designed to be included in multiple mods.

The problem appears to be in Verse.LoadedModManager.LoadAllActiveMods().

Code: [Select]
foreach (Type type in typeof(Mod).InstantiableDescendantsAndSelf())
{
if (!LoadedModManager.runningModClasses.ContainsKey(type))
{
ModContentPack modContentPack2 = (from modpack in LoadedModManager.runningMods
where modpack.assemblies.loadedAssemblies.Contains(type.Assembly)
select modpack).FirstOrDefault<ModContentPack>();
LoadedModManager.runningModClasses[type] = (Mod)Activator.CreateInstance(type, new object[]
{
modContentPack2
});
}
}
If FirstOrDefault is replaced with a method call, which will return the ModContentPack where type comes from a DLL with the highest AssemblyFileVersion, then it should be the newest DLL, which is loaded, regardless of load order.

Unlike most other vanilla issues, this is near impossible to fix for modders because the most obvious solution would require using Harmony before 0Harmony.dll is loaded.

3
Help / How do I open a Dialog_MessageBox in main menu
« on: December 01, 2017, 11:02:20 PM »
I'm have made a PatchOperation, which should open a Dialog_MessageBox if it fails. Right now it calls ModsConfig.RestartFromChangedMods() for testing purposes, but eventually I plan to make my own.

The problem seems to be that it requires Find.WindowStack before the GUI is started and it results in a black screen I have to force quit.

This leaves me with two problems:
  • How do I make a list, which can be written to during patch operations and read later?
  • How do I make a method, which will be called once the GUI is ready for opening windows?
While I planned to do this without Harmony, I can use it if I have to.

4
Ideas / Mod version tag in About.xml
« on: November 12, 2017, 09:38:35 AM »
Currently About.xml has targetVersion, but that's the required version of RimWorld. I would be nice to have the same for the mod itself as it will allow one mod to check the version of another mod being loaded.

The version is int.int.int, or major.minor.build. Looking at the vanilla code it looks like only minor is actually in use when using data from About.xml. This means it would be possible to use build as mod version number and while I have done so personally, it would be nice to get an official statement about this, in which case modders will know what to do with this number and not just write 0 or current RimWorld build version. Also an official statement would mean we know using build version for this will not break anything in the future.

5
Ideas / Mods working with multiple versions of RimWorld
« on: November 12, 2017, 09:31:06 AM »
Now that A18 unstable is out, mods are being released as A17 and A18. In some cases (like my own mod ModCheck), the difference is only About.xml. If that file could say both 17 and 18, mod releasing would be a whole lot simpler. While the ship has sailed for A17, adding this ability in A18 will solve the problem once A19 shows up.

6
Tools / [LIB] ModCheck: patch tests and error logging for mod loading
« on: October 31, 2017, 06:57:04 PM »
Toolkit for xml modders. Adds 13 new patch operations. Eliminate the need for patchmods. Patch according to presence, order or version of mods. Reorder xml elements, write patching results and test results to the log. Allow faster patching (shorter startup time) and measure how long each patch spends patching.

The primary task for ModCheck is to remove the need for patch mods. It's done by adding test operations, which can tell if another mod is loaded, if one mod is before another, is of at least a certain version, either in About or in ModSync. Each of those can be reversed (not loaded etc).

Adds logic operators, like sequence, AND, OR, If else conditions. This can be used together with the test operations or vanilla operations to make complex test conditions if needed.

Boost performance. Need to do multiple patching operations on the same building? Search, keep the result and run a list of patches without performing a full xpath search for each operation.

Added a bed and want it to appear in the build menu next to the vanilla beds? The Move operation allows you to alter where your modded building will appear.

Feature rich log writing. Get operations to write messages, warnings and errors if operations succeed or fail. Tell the user if a needed mod isn't loaded or load order is incorrect. Also allow writing conditionless with whatever message you might want to add.

Profile patches. Measure how much time is spend on each patch. ModCheck is aware of which mod owns which patch, meaning you can get a precise view of the startup time of your mod. You can name your patches if root operation is from ModCheck, like ModCheck.Sequence. This will allow performance printout with names rather than just patch 1,2,3....

Links:
Steam
GitHub (direct download, source code and wiki manual)

Quote from: Changelog
Updating ModCheck will not break existing xml files unless stated otherwise.

v1.8.1
- Fixed compatibility issue with updated Rimworld. Profiling works again.
- Updated version URL for Fluffy's Mod Manager

v1.8 (RW 1.0)
Update to about only. The DLL file will not even have to be updated.

- RimWorld 1.0 support
- Added support for Fluffy's Mod Manager

v1.8
This is a significant update from a coding point of view. B19 vanilla changed completely regarding patching.
While the code is significantly better, all Harmony calls from ModCheck had to be redesigned and rewritten.

- Added B19 support
- Added Search operation to speed up patching when the same object is patched multiple times in a row
- Added Move operation to control cases where order matters (like order of building buttons)
- Patch profiling now measures the time more accurately (less rounding errors)

XML BREAKING ALERT!!!
Removed FindFile operation (vanilla rewrite renders it both obsolete and impossible to implement)
Any xml file with FindFile will need updating.

v1.7
- Speedboost: cached mod indexes for massive speed boost of some ModCheck internals
- Rewritten the log writing system to give better control/more features to patch writers
- Rewritten error messages to make it easier to find the error
- Changed profiling output. Total on top, one entry for each mod
- All PatchOperation names can now be used starting with both upper and lower case (fixes naming inconsistency)
- Added new mode to LoadOrder. It can now use first and last strings instead of the old approach (which still works)
- Added Sequence operation, which does the same as the vanilla operation, but with ModCheck specific options
- Added logic operations AND, OR, IfElse, Loop and Once
- Added warning/error if outdated versions of ModCheck are being loaded (risk of new vs old conflicts)
- Added a preview logo (thanks to larSyn for drawing it)
- Added support for ModSync RW
- Fix: profiling now displays correct time if the hardware has a high precision timer
- Fix: profiling will no longer cut off the output if you have a lot of patches
- Removed the need to include yourMod and modName unless they are actually used

v1.6
- Added FindFile to greatly speed up patch files
- Added patch operation profiling (with verbose logging only)

v1.5
- Added ModSync.xml
- Added PatchOperation isModSyncVersion

v1.4
- Fixed issue where cache failed to update as needed

v1.3
- Fix: checks are now only executed once (massive performance boost)
- Fix: writing to the log will now always only write one line and never repeat the same many times

v1.2
- Added custom message support (like logging: My mod detected modX and will patch itself accordingly)
- Added ability to detect another mod by more than one name (like name v1.3 and name v1.4)
- Major code redesign to greatly reduce the risk of bugs when adding new features

Note: stopped releasing for A17 due to the release of stable B18.
If you want to use A17, copying the v1.2 DLL will likely work, but it's untested.

v1.1
- Converted to mod layout for steam release

v1.0
- Initial release


7
Bugs / butcherProducts prevents butchers from hauling meat to stockpile
« on: October 24, 2017, 09:06:04 AM »
When a butcher butches an animal, he/she will haul meat to the freezer and leave leather behind. However if xml contains anything in butcherProducts, the butcher will haul an item from butcherProducts and leave meat to rot. It's quite rare in vanilla, but it affects every single mod, which touches butcherProducts.

The problem appears to be with the method ButcherProducts. It's called in Verse.Corpse and it seems that the butcher will haul the first thing, which is returned with yield return. It returns the output from ButcherProducts in Verse.Pawn. Here it calls ButcherProducts in Verse.Thing, which in turn returns the contents of butcherProducts in xml. After that Verse.Pawn will move on to generating meat and return that.

This means a fix would be to make sure meat will be returned first and the simplest solution I can see is to go into Verse.Pawn and flip the first two pieces of code to generate yield returns, which would be:
Code: [Select]
foreach (Thing t in base.ButcherProducts(butcher, efficiency))
{
yield return t;
}
Code: [Select]
if (this.RaceProps.meatDef != null)
{
int meatCount = GenMath.RoundRandom(this.GetStatValue(StatDefOf.MeatAmount, true) * efficiency);
if (meatCount > 0)
{
Thing meat = ThingMaker.MakeThing(this.RaceProps.meatDef, null);
meat.stackCount = meatCount;
yield return meat;
}
}
I haven't directly tested this through modding and doing so would end up with mod incompatibility issues. However working on the bone mod, I removed bones from butcherProducts(xml), attached a butcherProducts(Verse.Pawn) Postfix function using Harmony and then used that one to add bones to the end of the list. Sure enough the butchers went from hauling bones to hauling meat and leaving bones on the floor.

It would be best to always haul the item, which can rot. This means if meat comes first, then butcherProducts(xml) and then the rest of the stuff because this will allow xml modders to pick what is hauled for corpses without meat.

Pages: [1]