[LIB] ModCheck: patch tests and error logging for mod loading

[larSyn]

I have a question about FindFile.  Obviously, it works well when changing something in another mod/core file, but should it be used if I need to change something in my mod if a certain mod is loaded?  Or would this still be a case where isModLoaded used?

[Nightinggale]

But, that's why editors/proofreaders exist, and I don't mind helping out in that respect.

Sounds good. However right now I put documentation on hold and is working on loading the newest version of ModCheck rather than the first in the load order. It looks like it will require everybody to update once it's out and then updating will not really be an issue unless the mod itself makes use of the new features. I better get this working before too many mods use ModCheck.

I checked the doc with Wordpad and it didn't lose any of the changes/formatting, so you should be good too.  Just an fyi, I use Libre Writer, it's a free open source version of Word.  Might be worth checking out, if you need a word processor.

It's good that nothing is lost. I do have LibreOffice installed. In fact I I used it back before it forked from OpenOffice. However that too can lose formatting when importing word documents.

I did a couple quick logos last night, and exported them as svg's as requested.  I'll attach them both to this comment.

The second one looks awesome. It's nothing like what I had in mind, but that doesn't matter because this one is better.

Now if there can be a version of it, which indicates an update is available, then it would be great. That way I can add each to git and the forum posts can point to one for a specific version. When I release a new one, I can update the old to indicate that it's out of date and it will automatically update for all the mods using it. This will then be a reminder to update the files the next time the mod is released anyway.

I have a question about FindFile.  Obviously, it works well when changing something in another mod/core file, but should it be used if I need to change something in my mod if a certain mod is loaded?  Or would this still be a case where isModLoaded used?

When patching, you make a sequence as follows:

• FindFile
• isModLoaded
• loadOrder
• patching

Remember that every single line needs to pass for the next one to be reached. This mean to reach patching, all the lines before it needs to pass.

FindFile is primarily for improving performance and it should go first. You should only skip it if you patch multiple files using the same code (that's rare). If you patch multiple files in the same mod, you use FindFile without file and it will try to patch all files in modName. Since it's first, the following will not even test if they are ok.

isModLoaded will pass if the mod is loaded (obviously). However it's the same for each call, meaning if the mod is loaded, it will pass for all files and not prevent attempts at patching the wrong files. isModLoaded can be skipped if modName is the same as in FindFile. It's only useful if they look for different mods.
Exception: if modName is required and isModLoaded will give an error if it fails, it has to go before FindFile if they use the same modName. Otherwise the error will never trigger.

loadOrder is useful in case the mod from the previous test(s) needs some other mod to be loaded first and it will cause errors if it's loaded later. Adding loadOrder will prevent those errors. Quite useful if it writes an error to the log since it changes nasty error messages into a single one, which is easy for the users to understand.

Any number of each test can be used, though it often makes little sense to use multiple FindFile. However you can test for modName only and then make nested sequences where each start with FindFile where it only checks for a file because you know at that point it's the correct mod. It's less readable, but if you have other checks in between, which might give error messages or similar, then it could be a good idea.

[larSyn]

Sounds good. However right now I put documentation on hold and is working on loading the newest version of ModCheck rather than the first in the load order. It looks like it will require everybody to update once it's out and then updating will not really be an issue unless the mod itself makes use of the new features. I better get this working before too many mods use ModCheck.

No problem.  I look forward to any updates you have planned.

The second one looks awesome. It's nothing like what I had in mind, but that doesn't matter because this one is better.

  Glad you like it!  I updated it to make the xml easier to use.  It's attached to this comment.

Now if there can be a version of it, which indicates an update is available, then it would be great. That way I can add each to git and the forum posts can point to one for a specific version. When I release a new one, I can update the old to indicate that it's out of date and it will automatically update for all the mods using it. This will then be a reminder to update the files the next time the mod is released anyway.

This page tells how to set it up on Github.  I tried it and it wasn't too hard.  If I knew how I could transfer the gist over to you I would, but I have no idea if I can or if it's possible...let me know if you know of a way. 

This is the code you will be looking for:

Code Select Expand


<g id="red_x5F_BG">
<g>
<g>
<path id="SVGID_1_" fill="#FFFFFF" d="M566.047,130.993c0,6.56-4.873,12.378-11.5,12.378h-514c-6.627,0-12.5-5.818-12.5-12.378
V57.747c0-6.56,5.873-11.375,12.5-11.375h514c6.627,0,11.5,4.815,11.5,11.375V130.993z"/>
</g>
<g>
<defs>
<path id="SVGID_2_" d="M566.047,130.993c0,6.56-4.873,12.378-11.5,12.378h-514c-6.627,0-12.5-5.818-12.5-12.378V57.747
c0-6.56,5.873-11.375,12.5-11.375h514c6.627,0,11.5,4.815,11.5,11.375V130.993z"/>
</defs>
<clipPath id="SVGID_3_">
<use xlink:href="#SVGID_2_"  overflow="visible"/>
</clipPath>

<rect x="511.547" y="45.871" clip-path="url(#SVGID_3_)" fill="#ff0000" stroke="#000000" stroke-width="5" stroke-miterlimit="10" width="55" height="97"/>
</g>
<g>
<defs>
<path id="SVGID_4_" d="M566.047,130.993c0,6.56-4.873,12.378-11.5,12.378h-514c-6.627,0-12.5-5.818-12.5-12.378V57.747
c0-6.56,5.873-11.375,12.5-11.375h514c6.627,0,11.5,4.815,11.5,11.375V130.993z"/>
</defs>
<clipPath id="SVGID_5_">
<use xlink:href="#SVGID_4_"  overflow="visible"/>
</clipPath>

<rect x="128.548" y="45.871" clip-path="url(#SVGID_5_)" fill="#ff0000" stroke="#000000" stroke-width="5" stroke-miterlimit="10" width="383" height="97"/>
</g>
<g>
<defs>
<path id="SVGID_6_" d="M566.047,130.993c0,6.56-4.873,12.378-11.5,12.378h-514c-6.627,0-12.5-5.818-12.5-12.378V57.747
c0-6.56,5.873-11.375,12.5-11.375h514c6.627,0,11.5,4.815,11.5,11.375V130.993z"/>
</defs>
<clipPath id="SVGID_7_">
<use xlink:href="#SVGID_6_"  overflow="visible"/>
</clipPath>

<rect x="28.548" y="45.871" clip-path="url(#SVGID_7_)" fill="#ff0000" stroke="#000000" stroke-width="5" stroke-miterlimit="10" width="100" height="97"/>
</g>
<g>
<defs>
<path id="SVGID_8_" d="M566.047,130.993c0,6.56-4.873,12.378-11.5,12.378h-514c-6.627,0-12.5-5.818-12.5-12.378V57.747
c0-6.56,5.873-11.375,12.5-11.375h514c6.627,0,11.5,4.815,11.5,11.375V130.993z"/>
</defs>
<clipPath id="SVGID_9_">
<use xlink:href="#SVGID_8_"  overflow="visible"/>
</clipPath>
<path clip-path="url(#SVGID_9_)" fill="none" d="M566.047,130.993c0,6.56-4.873,12.378-11.5,12.378h-514
c-6.627,0-12.5-5.818-12.5-12.378V57.747c0-6.56,5.873-11.375,12.5-11.375h514c6.627,0,11.5,4.815,11.5,11.375V130.993z"/>
</g>
</g>
</g>


Specifically these lines (there are 2, you will need to change both to change each red section to a different color):

Code Select Expand


<rect x="511.547" y="45.871" clip-path="url(#SVGID_3_)" fill="#ff0000" stroke="#000000" stroke-width="5" stroke-miterlimit="10" width="55" height="97"/>


You'll want to change the "ff0000" for the fill to alter the colors in the image.  The link I left in the earlier post has a few codes for colors, and I found this page, which has a lot more.

To change the version number, look for the "text" section.  It's the last line in that part.

When patching, you make a sequence as follows...

Thanks for the explanation.  I have a case where I add a RecipeDef if Vegetable Garden is loaded and since there is no file for it I assume that isModLoaded is the way to go.

I also updated my patches to 1.6, and I definitely noticed an improvement in load times.  Nice work!

[attachment deleted by admin: too old]

[Nightinggale]

Question for everybody
I need to place ModCheck.dll in the mod, but outside Assemblies or any subdir of Assemblies. Any proposals to where it could be?

The reason is that I have finally managed to get ModCheck to use the newest DLL instead of the first if multiple are present. This is by far the hardest to implement (Harmony Transpiler is piece of cake in comparison). After a lot of experimentation and research, it turns out that once RimWorld has loaded a dll, then we can't remove or replace it. If the same dll is loaded multiple times, the first will be used. If they aren't all of the same version, some nasty stuff can happen. Despite my efforts there is nothing I can do about that.

The solution is to add ModCheckLoader.dll. It has just one purpose, which is to loop all mods, look for ModCheck.dll and figure out which one has the newest AssemblyFileVersion. It then manually loads that one and calls the init code. No more and no less, meaning ModCheckLoader.dll should ideally never have to be replaced, which removes the risk of multiple versions of the same DLL.

WARNING: this will NOT work if the game can find any ModCheck.dll in Assemblies (it searches recursively), meaning starting from the next version, an error will be written to the log for each such file found. This mean anybody who released a mod with an included DLL will have to update it in the near future. The only alternative is to rename every single class used in the patch files, which I consider even worse. Remember the update is to prevent urgent update requirements in the future.

I also updated my patches to 1.6, and I definitely noticed an improvement in load times.  Nice work!

Turn on verbose logging and ModCheck will print how much time is spend on each patch. That way you can tell how much faster it is and not just "it feels faster". With just the patching mod loaded (it patches Core), using FindFile tend to make the patch around 30 times faster if xpath searches are already optimized. Obviously the gain is even greater if it's dead slow to start with.

I will look into the logo stuff when my head is not full of dll issues.

[Jaxxa]

I suggest a new Folder next to Assemblies.

Maybe named something along the lines of "AdditionalAssemblies" or "LibraryAssemblies".

In it should contain a .txt file explaining what you said in your post about why this additional folder is required.

If this is an issue with other library type mods, such as Harmony / HugsLib it might be a good idea to see if you can standardize the folder and the .dll that does the loading.

[frenchiveruti]

Something like "Modcheckdll"? like, straight on point.

[Nightinggale]

I suggest a new Folder next to Assemblies.

Maybe named something along the lines of "AdditionalAssemblies" or "LibraryAssemblies".

I was thinking of ManuallyLoadedAssemblies, but I decided to not mention it as it would disturb the feedback.

In it should contain a .txt file explaining what you said in your post about why this additional folder is required.

Good point.

If this is an issue with other library type mods, such as Harmony / HugsLib it might be a good idea to see if you can standardize the folder and the .dll that does the loading.

I have been wondering about this one. It sounds like the way to go, but it's not as easy as it would appear at first. Some mods have special requirements.

What I came up with is this (after many ideas, which ended up as flawed)

• Loop all mods and add all dll files to a list
• Sort alphabetically and delete duplicates
• For each, call the code I wrote now to load the newest
• Once loaded, make an instance of (dll name without extension).LoaderInit

It seems simple, but some consideration went into this. Sorting alphabetically allows 0Harmony to load first (not sure if Harmony needs this, but it better support it if needed). The create instance code is because currently it calls this:

Code Select Expand

assembly.CreateInstance("ModCheck.HarmonyStarter");
It creates an instance, which goes out of scope right away. However it's still useful because it calls the default constructor of that class inside the newly loaded dll file. ModCheck use this to start Harmony and then set up a singleton, which is used to keep track of a bunch of stuff while patching. The class can be renamed to meet a standard.

I would need to create a new LIB mod for this. Making the code part of ModCheck is not the way to go if it is to be used for other mods as well.

[sulusdacor]

maybe a bit of a noob question here: is this faster then using the short code?

meaning this:

Code Select Expand

internal class PatchOperationFindMod : PatchOperation
{
private string modName;

protected override bool ApplyWorker(XmlDocument xml)
{
return !GenText.NullOrEmpty(this.modName) && ModsConfig.ActiveModsInLoadOrder.Any((ModMetaData m) => m.Name == this.modName);
}
}
(thats what i currently use and it seems to works fine.)

in one post on the previous page you mention shorter load times, suggesting that using modcheck speeds it up quite a bit. if thats the case what would be the order/minimum to implement the modcheck patchoperations to benefit from that speed up. you mentioned:
    FindFile
    isModLoaded
    loadOrder
    patching
Or
like this what BrokenValkyrie did:

Code Select Expand


<Patch>
<!--Patch in projectile def if Framework detected.-->
<Operation Class="PatchOperationSequence">
<success>Always</success>
<operations>
<li Class="ModCheck.isModLoaded">
<modName>Combat Extended</modName>
<yourMod>Dragon Mod</yourMod>
<success>Invert</success>
</li>
<li Class="ModCheck.isModLoaded">
<modName>Range Animal Framework</modName>
<yourMod>Dragon Mod</yourMod>
<customMessageSuccess>Range Animal Framework: Adding Range attack to Dragon Mod.</customMessageSuccess>
</li>
<li Class="ModCheck.FindFile">
<modName>Dragon Mod</modName>
<file>Projectile_Dragon.xml</file>
</li>
<li Class="PatchOperationReplace">
<xpath>Defs/ThingDef[defName = "ARA_DragonFireBreathAlpha"]</xpath>
<value>
// do stuff
</value>
</li>
</operations>
</Operation>
</Patch>
(code taken from here)

so does findfile first or ismodloaded?

is the loadorder check needed? since i found recently that in b18 this did not matter at all when patching with the simple modcheck i posted at the start. so i'm a bit confused on that.

if i wanted to include the ismodsyncversion, at which position would that go in the sequence? or does that even matter?

just asking, couse the guide/github linked at the first page is a bit older and i'm slighty confused here^^

thx for putting the work. seems like a really big help for modders

[Nightinggale]

maybe a bit of a noob question here: is this faster then using the short code?

Despite being a simple question, the answer is not simple. You can't predict performance based on the number of C# lines. You have to consider how the CPU will handle the resulting code and while it is often possible to merge multiple lines into one, the resulting IL code might not change at all. In other words it's not possible to give a simple yes or no answer.

The best approach is to measure multiple solutions to see the result. Also it makes sense to measure the "before" code since even an 80% reduction is worthless if the method use 0.00001% of the combined CPU time. Luckily I added profiling to v1.6. Just enable verbose logging and it will tell how long it spends on each patch, even patches, which have nothing to do with ModCheck.

Code Select Expand

internal class PatchOperationFindMod : PatchOperation
{
private string modName;

protected override bool ApplyWorker(XmlDocument xml)
{
return !GenText.NullOrEmpty(this.modName) && ModsConfig.ActiveModsInLoadOrder.Any((ModMetaData m) => m.Name == this.modName);
}
}
(thats what i currently use and it seems to works fine.)

This code will do the same as isModLoaded. It's less code and less method call overhead, which will in theory speed up the test. However the difference is so minor that you can't tell the difference. 99%+ of the time is spend on the search in ModsConfig.ActiveModsInLoadOrder.Any(). This mean the time the CPU spend is a constant multiplied by the number of calls to that method. Your approach will search every time the PatchOperation is called, which might be once for each xml file in each mod, meaning it can easily be a 4 digit number of times (it's around 400 times for vanilla alone). That's how ModCheck started out and reports of adding 10 minutes to startup time made me rethink the design. Starting from v1.3, ModCheckBase makes the check once and stores the result. The following calls will return the cached result. This mean your approach is significantly slower despite using way less lines of code.

in one post on the previous page you mention shorter load times, suggesting that using modcheck speeds it up quite a bit. if thats the case what would be the order/minimum to implement the modcheck patchoperations to benefit from that speed up. you mentioned:
    FindFile
    isModLoaded
    loadOrder
    patching
Or
like this what BrokenValkyrie did:

Code Select Expand


<Patch>
<!--Patch in projectile def if Framework detected.-->
<Operation Class="PatchOperationSequence">
<success>Always</success>
<operations>
<li Class="ModCheck.isModLoaded">
<modName>Combat Extended</modName>
<yourMod>Dragon Mod</yourMod>
<success>Invert</success>
</li>
<li Class="ModCheck.isModLoaded">
<modName>Range Animal Framework</modName>
<yourMod>Dragon Mod</yourMod>
<customMessageSuccess>Range Animal Framework: Adding Range attack to Dragon Mod.</customMessageSuccess>
</li>
<li Class="ModCheck.FindFile">
<modName>Dragon Mod</modName>
<file>Projectile_Dragon.xml</file>
</li>
<li Class="PatchOperationReplace">
<xpath>Defs/ThingDef[defName = "ARA_DragonFireBreathAlpha"]</xpath>
<value>
// do stuff
</value>
</li>
</operations>
</Operation>
</Patch>
(code taken from here)

so does findfile first or ismodloaded?

is the loadorder check needed? since i found recently that in b18 this did not matter at all when patching with the simple modcheck i posted at the start. so i'm a bit confused on that.

if i wanted to include the ismodsyncversion, at which position would that go in the sequence? or does that even matter?

FindFile should always go first if used and if you can find a case where it shouldn't be used, then I would like to know about it. Admittedly isModLoaded can't be considered slow, but it's generally a good idea to have the guidelines aiming for the best result. It's purely a matter of performance.

Don't use FindFile and isModLoaded for the same mod since FindFile will fail if the mod isn't loaded (obviously). isModLoaded is useful like in the example where Dragon Mod is patched if CE and Range Animal Framework are loaded.

The reason is that in a sequence each PatchOperation is called until one fails. This mean it will make sense to sort the tests based on speed and likelihood of failure (fail as early as possible if failing). Tests such as isModLoaded will always be either true or false while FindFile will change result based on the current file and it will in most cases be false.

loadOrder is useful in one specific case. If you add a def file, which includes something with a parent where the parent is in another mod, then that other mod needs to be loaded first. A good example of this is alien races, which relies on alien framework. Often it's used to display an error if the requirement isn't met, though it supports conditional patching. If you don't what I wrote here, then you don't need to use it.

ismodsyncversion can be added in any order, though I would recommend after FindFile.

just asking, couse the guide/github linked at the first page is a bit older and i'm slighty confused here^^

I have a plan for a complete rewrite. It was sort of ok at first and then I just added each time I updated something and now it's an unplanned mess. I'm holding off updating it before v1.7 though because that one will split into two DLL files and it will once and for all solve the issue of the problems from mods not using the same version of ModCheck. Keeping ModCheck up to date will still be a good idea, but failing to do so will not break other mods.

thx for putting the work. seems like a really big help for modders

Thanks. It's supposed to be useful because I wrote it to cover my own needs. Since I instantly viewed it as useful for everybody and no such standalone mod existed at the time, I decided to make one. Ironically ModCheck has taken up all of my modding time and the planned mod it's made for is still a planned mod.

[ilikegoodfood]

Hi, I'm brand new to modding and have just started work on a monster mod, called Monster Mash.

I implemented my first creature, the Inferno Beetle, and am now working on compatibility with other mods, using ModCheck.
My vanilla patch to implement the Range Animal Framework (LINK) is working perfectly, however, it is using the vanilla patch even when I have Combat Extended (A18 pre-release) installed and the Combat Extended Patch set up.

Here is the code for the Vanilla patch that is supposed to disable it if CombatExtended is installed:

Code Select Expand

<Patch>
<Operation Class = "PatchOperationSequence">
<success>Always</success>
<operations>
<!--Continue if combat extended does not exist IE patch vanilla-->
<li Class="ModCheck.isModLoaded">
<modName>Combat Extended</modName>
<yourMod>Monster Mash</yourMod>
<success>Invert</success>
<customMessageSuccess>Vanilla combat systems: Patching ranged attacks...</customMessageSuccess>
</li>
<li Class="ModCheck.loadOrder">
<modName>Core</modName>
<yourMod>Monster Mash</yourMod>
<errorOnFail>true</errorOnFail>
</li>
<li Class="ModCheck.FindFile">
<modName>Monster Mash</modName>
<file>Races_Animal_Insect.xml</file>
</li>
<li Class="PatchOperationAdd">

And here is the code in the CombatExtended Patch taht should trigger it if the CombatExtended mod is installed:

Code Select Expand

<Patch>
<Operation Class="PatchOperationSequence">
<success>Always</success>
<operations>
<li Class="ModCheck.isModLoaded">
<modName>CombatExtended</modName>
<yourMod>Monster Mash</yourMod>
<customMessageSuccess>Combat Extended found: Modifying organisms for extreme combat realism...</customMessageSuccess>
</li>
<li Class="ModCheck.loadOrder">
<modName>CombatExtended</modName>
<yourMod>Monster Mash</yourMod>
<errorOnFail>true</errorOnFail>
</li>

I'm really not sure where my error is and I would be very grateful if someone could explain to me what is going wrong and how to fix it.

Thanks in advance.

EDIT: I have found the error that was preventing the patch from working.
Combat Extended has a space in the mod name, which I correctly implemented in the vanilla patch, but I somehow removed the spaces for the CE patch.
Now to bugfix the patch itself...

[Nightinggale]

I have good news and bad news. The good news is that I have finally made a DLL, which is able to pick which ModCheck DLL to load and it will indeed pick the newest one if there are multiple available. The bad news is that while I can load the DLL this way and call methods from it, I can't get it to work with the xml passing code, meaning it can't find ModCheck.FindFile even if that method is loaded. Even worse, despite my efforts, I can't really tell why, meaning it doesn't appear I will be able to fix it

Right now I can't really think of any other approach than to keep the current system and rely on people to update whenever ModCheck is updated. What I can do is to add code to ModCheck startup, which makes it scan for ModCheck in all loaded mods and print a warning for each outdated and error if it isn't the newest, which is in use. Not ideal, but it's the best solution I can think of.

[Nightinggale]

ModCheck 1.7 has been released.

v1.7 changelog
- Speedboost: cached mod indexes for massive speed boost of some ModCheck internals
- Rewritten the log writing system to give better control/more features to patch writers
- Rewritten error messages to make it easier to find the error
- Changed profiling output. Total on top, one entry for each mod
- All PatchOperation names can now be used starting with both upper and lower case (fixes naming inconsistency)
- Added new mode to LoadOrder. It can now use first and last strings instead of the old approach (which still works)
- Added Sequence operation, which does the same as the vanilla operation, but with ModCheck specific options
- Added logic operations AND, OR, IfElse, Loop and Once
- Added warning/error if outdated versions of ModCheck are being loaded (risk of new vs old conflicts)
- Added a preview logo (thanks to larSyn for drawing it)
- Added support for ModSync RW
- Fix: profiling now displays correct time if the hardware has a high precision timer
- Fix: profiling will no longer cut off the output if you have a lot of patches
- Removed the need to include yourMod and modName unless they are actually used

The best I can do regarding people running multiple versions of ModCheck is to make the DLL aware of it and print a warning if multiple versions are loaded and an error if the newest isn't first. On top of that all harmony code will only trigger in the newest version. In other words if the newest is loaded first, it should work, particularly when the time comes for 1.7 to be the oldest. However it's not great and I will try to avoid minor updates in the future and instead wait and collect for major releases.

The log writing system is completely rewritten. You can now use tags to tell when the string should be used and what kind of output: message(white), warning(yellow) and error(red). You can add verbose to make it only trigger if verbose logging is enabled. Examples: ErrorFail and VerboseMessageSuccess. Works on cached operations and the new ModCheck.LogWrite.

Log writing can now take arguments, like {0} and {1}, which are mod and file being patched respectively. Those two are likely the most useful because that's the strings needed for ModCheck.FindFile. There are 5 such arguments in total.

ModCheck.OR has been added. It's a sequence where it stops at the first operation, which passed and then it will pass itself and only fail if everything failed. You can use it to make a list of FindFile operations if you for some reason want to apply the same patch operation(s) to multiple files.

The changelog will have to do for the rest of the changes for the time being. I plan to rewrite the wiki from scratch. Since this release is fully backward compatible, updating shouldn't break anything and you can continue to use the 1.6 documentation until the 1.7 version is ready.

[Nightinggale]

I started writing the wiki for 1.7, but it would be nice to get feedback on the style of writing/layout before I go through all the operations. The pages so far:

• FindFile
• LogWrite
• Log writing

Is this something people can read and understand?
Obviously it will need some overview page at some point. This is pages for getting details on specific features.

[Nightinggale]

I have good news and bad news.

The good news is that I finished documenting each PatchOperation on the wiki. I have plans for other pages, like concepts and guides for how to use them together and stuff like that.

The bad news is that I just provided feedback to RimWorld 1.0 because it completely destroys ModCheck in such a way that it's not possible to fix. If ModCheck will make it to 1.0, RimWorld itself will have to change to become more open to ModCheck.

[SargBjornson]

Keep us informed! I hope Tynan can patch things up