[Tutorial] xml-Only Dynamic Definitions using ModCheck

[ilikegoodfood]

xml-Only Dynamic Definitions

What are dynamic definitions and why should I (the reader) care?
Currently, using ModCheck and the xpath operations that it provides and expands upon, it is possible to perform simple patches to a mod's definitions at run-time, in response to the presence or absence of certain other mods.

Elements of this conversation will be easier to explain using an example. This example will be persistent throughout this guide:
Let's say that you have made a mod that adds a new fruit to RimWorld and that you want it to be compatible with a wildly popular mod that adds Jam.
This hypothetical Jam mod adds a new workstation, a recipe to turn each fruit (there are only three in vanilla RimWorld) into its very own jam, and one additional recipe for mixed-fruit jam.

Using normal patching techniques, it is fairly easy to detect the Jam mod and add your fruit to the list of allowed ingredients for the mixed jam.

This guide is for an advanced and experimental mod-patching method. There are many good guides and decent examples of patches using ModCheck already available.

The limitations with standard patching don't start to emerge until you try to add a recipe for the jam made from your own fruit.
You might first encounter the issue that you can't patch in root definitions, such as a RecipeDef, from scratch. A targetable Def with an appropriate name is required first.
You might then try adding a the RecipeDef (to make the jam) and ThingDef (to have the jam item) to your own mod, that uses the Jam mod's baseJam definition.

This is all very well and good until you then run a test without the Jam mod installed and find that your new definitions are throwing up cross-referencing errors, such as: Cannot find ParentName.

This error occurs because the Parent is a file from another mod, one which isn't currently installed. Many modders may be tempted to copy its base definitions into their own mod (which is very bad practice and hasn't been required since a16), or to create their own version of the parent (which is also bad practice).
While this might solve the immediate problem, the game still won't be able to cross reference the workstation that is supposed to receive the recipeDef or find the graphic that is added by the Jam mod.

This is where "dynamic definitions" come in.
What I mean by a dynamic definition, is a definition that is selectively edited down to remove all cross-referencing issues with other mods and then patched at run-time to a working state only when the other mod is detected.

How to Make a Dynamic Definition
First off, a word of warning: Working with dynamic definitions is by no means a trivial task, requiring more work than creating a compatibility sub-mod. There are a number of unknown behaviors, mechanisms and errors at play, which are all solvable, that I will attempt to explain and justify as we work through this guide.

Your first Dynamic Definition
The first step is to create the definition, exactly as you normally would and assuming that the Jam mod (or other compatibility target) is installed.
It might look something like this:

Code Select Expand

<RecipeDef ParentName="makeJamBase">
<defName>makeBlackberryJam</defName>
<label>make blackberry jam</label>
<description>Make a delicious blackberry jam.</description>
<jobString>Making delicious blackberry jam.</jobString>
<ingredients>
<li>
<filter>
<thingDefs>
<li>blackberry</li>
</thingDefs>
</filter>
<count>25</count>
</li>
</ingredients>
<fixedIngredientFilter>
<thingDefs>
<li>blackberry</li>
</thingDefs>
</fixedIngredientFilter>
<products>
<BlackberryJam>1</BlackberryJam>
</products>
<skillRequirements>
<Cooking>5</Cooking>
</skillRequirements>
<workSkill>Cooking</workSkill>
</RecipeDef>

If you tried to run your mod at this stage, it would cause several errors. makeJamBase is from a mod that isn't necessarily installed and we haven't made the item BlackberryJam yet (We will ignore this error in this guide, since you should be able to comment it out for testing and create the ThingDef for BlackberryJam).

To prevent the recipe from causing errors, you will need to remove the ParentName attribute (don't forget what the parent should be! You will need it soon). This will lead to the RecipeDef being incomplete, since it is no longer inheriting properties from its parent.

The game already has a method for handling incomplete data, but it is primarily used for the base definitions themselves; Abstract = "True"

Abstract = "True"
Abstract definitions are a bit of a special case. These definitions are loaded while the game is loading, but they are not error-checked nearly as rigorously, they do not need to be complete and, most importantly of all, they are discarded when loading completes.

Once the game is loaded, that definition essentially doesn't exist within the game.
You won't get errors from it, unless the error is passed on to its children, you won't be able to use it and you won't find it in the developer-mode tools.

This is key to the implementation of Dynamic Definitions.
If the Jam Mod isn't installed, the BlackberryJam simply doesn't exist.

At this stage, your RecipeDef would look like this:

Code Select Expand

<RecipeDef Abstract="True">
<defName>makeBlackberryJam</defName>
<label>make blackberry jam</label>
<description>Make a delicious blackberry jam.</description>
<jobString>Making delicious blackberry jam.</jobString>
<ingredients>
<li>
<filter>
<thingDefs>
<li>blackberry</li>
</thingDefs>
</filter>
<count>25</count>
</li>
</ingredients>
<fixedIngredientFilter>
<thingDefs>
<li>blackberry</li>
</thingDefs>
</fixedIngredientFilter>
<!--<products>
<BlackberryJam>1</BlackberryJam>
</products>-->
<skillRequirements>
<Cooking>5</Cooking>
</skillRequirements>
<workSkill>Cooking</workSkill>
</RecipeDef>

The Patch
I'm assuming here that you have followed other people's guides and already know how to patch.
If this is not the case, there are several good resources, including A quick tutorial of xpathing and patching and [A17] A warning to modders: xpath performance, which are both pinned at the top of the Help sub-section of the modding forum, and Introduction to PatchOperation by Zhentar.

Any mod-dependent patch should contain a basic test to make sure that the target mod is installed and is above this one in the loading order. The opening section of a patch file usually looks something like this:

Code Select Expand

<?xml version="1.0" encoding="utf-8" ?>

<Patch>

<Operation Class="PatchOperationSequence">
<success>Always</success>
<operations>
<!-- Continue if Yummy Jams exists -->
<li Class="ModCheck.isModLoaded">
<modName>Yummy Jams</modName>
<yourMod>Blackberry Bonanza</yourMod>
<customMessageSuccess>Blackberry Bonanza :: Yummy Jams detected: Patching...</customMessageSuccess>
</li>

<li Class="ModCheck.loadOrder">
<modName>Yummy Jams</modName>
<yourMod>Blackberry Bonanza</yourMod>
<errorOnFail>true</errorOnFail>
</li>

There are several things here that need explaining:

<success>Always</success>

In a PatchOperationSequence, it will usually run until there is an error, report it, and then stop. That would be if <success>Normal</success> was used. However, when set to <success>Normal</success>, errors are logged that don't appear to do any harm or prevent the patch from working and the PatchOperationSequence is stopped unnecessarily.
This is why most mod patches unquestioningly use <success>Always</success>.

<customMessageSuccess>Blackberry Bonanza :: Yummy Jams detected: Patching...</customMessageSuccess>

ModCheck.isModLoaded is used to test if the required mod is installed, but it also allows you to send a custom message to the Debug Log.
It can be very useful to add duplicates, each with different custom messages, for testing purposes. If the message doesn't reach the Debug Log, then you know that there is a serious error between the last message that was sent and the one that wasn't sent.
Be aware though, that these messages do not appear in the Debug Log in the order that they are written in the patch. Check carefully.
There may be a solution for this (Combat Extended seems to have a strict message order), but I do not know what it is.

Next, we create two sub-sequences.

Wait, but can't it be done in a single sequence?
Well, the answer to that is "Sometimes".
In some patches and in some combinations it seems to work just fine, but at other times it will fail if there are different PatchOperations in the same sequence. The behavior appears highly inconsistent, so its best to make sure that each PatchOperationSequence only contains one type of PatchOperation.
The primary PatchOperationSequence, the one that spans the entire page, is therefore a sequence of PatchOperationSequence(s), and each sub-sequence handles a specific task.

So, back to our two sub-sequences.
The 1st sub-sequence is there to add the ParentName attribute back in to our definitions, and will look like this:

Code Select Expand

<li Class="PatchOperationSequence">
<success>Always</success>
<operations>
<li Class="PatchOperationAttributeAdd">
<xpath>Defs/RecipeDef[defName = "makeBlackberryJam"]</xpath>
<attribute>ParentName</attribute>
<value>makeJamBase</value>
</li>
</operations>
</li>

The 2nd sub-sequence then removes the Abstract="True" attribute from the definition, allowing it to be loaded into the game, and will look like this:

Code Select Expand

<li Class="PatchOperationSequence">
<success>Always</success>
<operations>
<li Class="PatchOperationAttributeRemove">
<xpath>Defs/RecipeDef[defName = "makeBlackberryJam"]</xpath>
<attribute>Abstract</attribute>
</li>
</operations>
</li>

We remove the Abstract attribute after defining the ParentName to avoid receiving errors from the error-check that is performed on a non-abstract definition as a result of the definition still being incomplete
When removing the Abstract attribute, it is important to select the definitions specifically. If you have created your own base definition in that file, it should remain Abstract, otherwise it will cause you a whole new set of errors.

Repetition, repetition, repetition...
From this point forward, you test, develop the remaining required definitions, the patches for them and re-test.
Make sure to un-comment any incomplete cross-references as you complete the definitions, since we still have BlackberryJam as the product commented-out in our example.

Closing Information and Notes
You will also, the exact reasons are unknown, but I believe that it is related to the order by which PatchOperations are performed, occasionally get a persistent cross-reference error with regards to one of your other dynamic definitions. In the case of our example, the ThingDef for BlackberryJam would be a likely error-point.
When that happens, remove the offending reference from the definition and patch them back into the definition using a final PatchOperation sub-sequence at the bottom of your Patch.
It might look like this:

Code Select Expand

<li Class="PatchOperationSequence">
<success>Always</success>
<operations>
<li Class="PatchOperationAdd">
<xpath>Defs/RecipeDefDef[defName = "makeBlackberryJam"]</xpath>
<value>
<products>
<BlackberryJam>1</BlackberryJam>
</products>
</value>
</li>
</operations>
</li>

Some additional unknowns are associated with the order of file-patching.
The custom messages sent to the Debug Log from different mods do not necessarily appear in the order that the mods are loaded (listed in the mod menu, top-bottom) and sometimes messages from other patch files within your own mod will appear in-between message that are half-way through an ongoing sequence.
While this could be a result of the message order only, it may also be that different patches from mods, different files or possibly even different sequences are performed out of order in some way.

To avoid this unknown operation order from impacting your dynamic definitions, they should all be patched inside of the Primary PathOperationSequence and all in the same file.

Further xpath resources available to you include online xpath checkers and xpath plugins for popular text editors such as Notepad++ and SublimeText.
You can also use my Monster Mash mod as an example framework (Steam Workshop) (Forum). It is the mod that I, ilikegoodfood, along with significant time and assistance from wwWraith, developed this patching system for and it uses the system to dynamically implement new artificial animal organs and prosthetics for A Dog Said... compatibility with my Land Kraken.
If you do you use this system, a reference would be highly appreciated, but it is not necessary.