[WIP][A13] Modpack helper

Started by natbur, June 03, 2016, 03:57:15 AM

Previous topic - Next topic

natbur

Unless I've missed something, there is no current way to update a Def without completely replacing it.

The goal of this Mod is to allow for easier creation of mod-packs, and hopefully allowing them to survive version changes in the underlying mods. 
Rather than a complete replacement of existing Defs, this allows a mod-pack creator to update properties on Defs already loaded.

I took a lot of inspiration from CCLs AdvancedResearchDef.  I had considered developing this as a patch to the CCL, but decided to get everything working solo first.  That said, I'd be happy to hear any criticisms of my technique from those who are more familiar with the inner workings of RimWorld's code. 

It's been working well in my testing and my big TODO remaining is to implement a priority loading system, similar to CCLs AdvancedResearchDef priorities.

Source: https://github.com/natbur/NB_ModPack_Core

Version 0.2.1 - Now with less code abuse
Version 0.2 based on Fluffy's Idea.  Only slightly abuses RW's Def loader, haven't seen any problems so far.

Adds one new Def: PartialDef, an example is attached

<Defs>
  <PartialDef type="ThingDef" priority="0">
    <targetDefs>
      <!-- Always target bed, target FancySingleBed if it exists -->
      <li>Bed</li>
      <li ifExists="TableStonecutter">TableStonecutter</li>
      <li optional="">FancySingleBed</li>
    </targetDefs>
    <!-- Add any fields compatible with @type specified above -->
    <properties>
      <!-- Replace existing cost list -->
      <costList>
        <!-- Only add WovenCloth if it exists, otherwise add regular cloth -->
        <WovenCloth optional="">12</WovenCloth>
        <Cloth ifNExists="WovenCloth">12</Cloth>
      </costList>
      <!-- Append new researches to any already loaded -->
      <researchPrerequisites mode="append">
        <!-- Three ways of doing the same thing -->
        <li ifModExists="NB_ClothWeaving">ClothMaking</li>
        <li ifExists="WovenCloth">ClothMaking</li>
        <li optional="">ClothMaking</li>
      </researchPrerequisites>
    </properties>
  </PartialDef>
</Defs>


Specification for the attribute tags:

    <!-- Only one category of Defs per partial (Possibly allow for base type i.e. BuildableDef -->
    <PartialDef type="TARGET_DEFTYPE" priority="INT"/>

    <!-- If a Def exists, process tag -->
    <TAG ifExists="DEFNAME"/>

    <!-- If a Def doesn't exist, process tag -->
    <TAG ifNExists="DEFNAME"/>

    <!-- Try to process tag, ignore errors -->
    <TAG optional=""/>

    <!-- If a mod exists, process tag -->
    <TAG ifModExists="MODNAME"/>

    <!-- If a mod doesn't exist, process tag -->
    <TAG ifModNExists="MODNAME"/>

    <!-- Append or replace items in list (non-unique always replace), default replace -->
    <LISTTAG mode="replace|append"/>

Fluffy (l2032)

Hi there! I've been thinking about something like this in the past, and I think it'd be a great resource to have, and certainly something that we'd be happy to put in CCL.

That said, I'm not sure I like your implementation with specific update defs per type. In my view, the ideal solution would be to have a generic 'partialDef' type, with a deftype and name tag to specify the def to override (possibly even a list of defnames to mass override). The contents of the def would then be parsed for tags matching fields in the targetted def, and if found they would be overriden. The benefit of such an approach is that it's much less hassle to implement, and it should extend to any def type. The drawback is that the backing code would be quite a bit more complicated, possibly to the point of overriding the vanilla def loading code to allow such dynamic def targetting (I believe vanilla would error/discard unknown fields otherwise?).

I'd be more then happy to help with implementing such a system, I can be your rubber duck - or I can help with the code if need be.

1000101

#2
CCL has a partial implementations of something like this already.  The ModHelperDef has a TickerSwitcher (override), ThingDefAvailability (override), ThingComps (ThingComp injector), Facilities (CompFacility & CompAffectedByFacilities linker) and TraderKinds (StockGenerator injector).  These could be expanded on and refactored in the future for a more flexible approach using something like fluffy suggested.  In the case of ThingDefAvailability, all the fields are strings or lists and type-checked at run-time for valid values.  If they are null, they are ignored.

By all means, continue your work and look at how CCL handles these things (/DLL_Project/Defs/ModHelperDef/).  We accept all code submissions which fit into the context of CCL and the team can always be expanded by people who are willing to work on and help maintain the library.  :)

(2*b)||!(2*b) - That is the question.
There are 10 kinds of people in this world - those that understand binary and those that don't.

Powered By

natbur

I'm really glad to hear that there's support for this idea!
Quote from: 1000101 on June 03, 2016, 06:27:38 AM
The ModHelperDef has a TickerSwitcher (override), ThingDefAvailability (override), ThingComps (ThingComp injector), Facilities (CompFacility & CompAffectedByFacilities linker) and TraderKinds (StockGenerator injector).

I'll have a poke around some more in the CCL ModHelperDef code next week and bounce some ideas off of you or Fluffy about possible refactoring.

Quote from: Fluffy (l2032) on June 03, 2016, 04:49:44 AMThat said, I'm not sure I like your implementation with specific update defs per type. In my view, the ideal solution would be to have a generic 'partialDef' type

I like that idea, I'd initially implemented my code as a generic UpdateDef but if I remember correctly, I ran into some issues with Unity trying to parse the Def with a generic param as the list type (List<T> targetDefs).  That's why I have the slightly upside-down property def and where my current inheritance model came from. 

Quote from: Fluffy (l2032) on June 03, 2016, 04:49:44 AMI can be your rubber duck
My very own Fluffy rubber duck!  ;D

A few quick thoughts, some contradictory:


1.  Is there an easy way (preferably without resorting to reflection) to hook into RimWorlds existing XML loaders?  I'm thinking about a structure something like this:

<partialDef>
    <defType>ThingDef</defType>
    <targetDefs><li>Bed</li></targetDefs>
    <properties>
        <!--Anything Valid in a ThingDef-->
        <costList>
          <Cloth>12</Cloth>
        </costList>
    </properties>
</partialDef>


If it were possible to run the properties node through RimWorlds XML loader, ignoring errors, it might make implementation easier.



2.  Playing devil's advocate to myself, implementing a custom XML loader would allow for XML attributes to be taken into account making things more flexible (and complicated), but possibly allowing for more advance interoperability.


<!-- Only one category of Defs per partial (Possibly allow for base type i.e. BuildableDef -->
<partialDef type="DEFTYPE" priority="INT"/>

<!-- If a Def exists, process tag -->
<TAG ifExists=""/>

<!-- If a Def doesn't exist, process tag -->
<TAG ifNExists=""/>

<!-- Try to process tag, ignore errors -->
<TAG optional/>

<!-- If a mod exists, process tag -->
<TAG ifModExists=""/>

<!-- If a mod doesn't exist, process tag -->
<TAG ifModNExists=""/>

<!-- Append or replace items in list (non-unique always replace), default replace -->
<LISTTAG mode="replace|append"/>



An example:

<partialDef type="ThingDef" priority="0">
    <targetDefs>
        <!-- Always target bed, target FancySingleBed if it exists -->
        <li>Bed</li>
        <li optional>FancySingleBed</li>
    </targetDefs>
    <!-- Replace existing cost list -->
    <costList>
        <!-- Only add WovenCloth if it exists, otherwise add regular cloth -->
        <WovenCloth optional>12</WovenCloth>
        <Cloth ifNExists="WovenCloth">12</Cloth>
    </costList>
    <!-- Append new researches to any already loaded -->
    <researchPrerequisites mode="append">
        <!-- Three ways of doing the same thing -->
        <li ifModExists="NB_ClothWeaving">ClothMaking</li>
        <li ifExists="WovenCloth">ClothMaking</li>
        <li optional>ClothMaking</li>
    </researchPrerequisites>
</partialDef>

<partialDef type="ThingDef" priority="9" ifModNExists="FluffysBetterMod">
...
</partialDef>




3.  Another thought/concern I've been toying with is when to replace or merge lists.  This is probably only relevant for research, but I can see an argument for allowing a researchPrerequisite to be appended to the existing list rather than replacing it.  Going route 2 would make this easier, but even without it, maybe just a property on the Def for add/replace research.



4.  More of a technical issue that I encountered and was interested in clarification on.  I had to do the Updates during the ResolveReferences phase, rather than PostLoad.  Whenever I tried to search the DefDatabase during PostLoad, it was always empty.  I didn't notice any side-effects from handling updates in ResolveReferences, it just feels like the wrong phase to hook into.

1000101

Unfortunately, the xml is loaded before any user code is run.  This means that by the time we can detour the xml loader, it's too late.  This is why ThingDefAvailability works with strings and lists of strings/defNames.  We have no ability to pre-empt and parse it ourselves.  Trust me, if we could, I would have a long time ago.  ;)
(2*b)||!(2*b) - That is the question.
There are 10 kinds of people in this world - those that understand binary and those that don't.

Powered By

natbur

#5
Quote from: 1000101 on June 03, 2016, 08:11:53 PM
Unfortunately, the xml is loaded before any user code is run.  This means that by the time we can detour the xml loader, it's too late.  This is why ThingDefAvailability works with strings and lists of strings/defNames.  We have no ability to pre-empt and parse it ourselves.  Trust me, if we could, I would have a long time ago.  ;)

In the context of a single Def this isn't exactly true, RW checks for 'LoadDataFromXmlCustom' on the Def class, and runs that if it exists instead of it's own XML loading.  Just updated my mod using this idea.

1000101

Yes, but it's still not running user code at that point.  CCL hooks in as soon as it can through it's custom defs (in this case ModHelperDef) but it's still after the XML is loaded.
(2*b)||!(2*b) - That is the question.
There are 10 kinds of people in this world - those that understand binary and those that don't.

Powered By

natbur

#7
Quote from: 1000101 on June 09, 2016, 11:55:27 AM
Yes, but it's still not running user code at that point.  CCL hooks in as soon as it can through it's custom defs (in this case ModHelperDef) but it's still after the XML is loaded.
Of course, you're right.  After spending more time with the code, it makes perfect sense why that's the case.  Unless someone invents some form of non-temporal programming, having a class loaded by the xml loader detour methods that are already finished would be some black magic indeed.

I still find it useful to be able to short the xml parser after loading, and move the parsing to a later phase (LongEventHelper.ExecuteWhenFinished() in this case).  This way I don't have to worry about RW throwing errors on the arbitrary structure of the <properties> node and can change which class ObjectToXml builds based on the @type attribute.

1000101

Well, I'm interesting in hearing about the results of your experiments.  Keep us posted.  :)
(2*b)||!(2*b) - That is the question.
There are 10 kinds of people in this world - those that understand binary and those that don't.

Powered By

Deimos Rast

so Module Manager (from KSP)? That would be lovely.
#StopPluginAbuse