[LIB] Harmony v1.2.0.1

Started by Brrainz, January 13, 2017, 04:59:21 PM

Previous topic - Next topic

RawCode

ever 48 years of experience will not shield you from undocumented runtime implementation features\bugs.

sceptic or not both method Prelink and PrepareMethod do nothing in your case, to be precise in case of mono runtime.

IL converted into native by jitter when "native constructor" is called on dynamic method, calling "finalize" or "build" is not suffice, getting function pointer before jitter is complited result in undefined behavior.


Brrainz

If mono can IL -> ASM then there is a way to call into mono to trigger it manually. I just have to find it.

Brrainz

After a whole weekend to get to the truth, this is what I know:

- detouring to dynamic methods works different

- you need to jit dynamic methods - simplest way is to have a "first time? return" statement at the beginning and then pre-run them

- detouring works for one method if replacement is dynamic method

- detouring does not work for more than one method regardless of what you do. It possibly corrupts the generic trampoline mechanism

- I need to understand trampolines way better before I continue work on Harmony

And no, RawCode, it is not a simple error. I rewrote the whole solution in different ways and simplified and reduced any chance for error and tested in a 100+ different ways. Say result every time. I can now get the whole patching work 100% reliable for a single patch but as soon as I write a small mod that does two non related patches all hell breaks loose (those two each alone work fine).

/Andreas

RawCode

sad but true, copy native code into native memory and rebase relative opcodes faster, more simple and feature no overhead compared to work with dynamic methods.

notfood

Trying to run it on Linux. Latest release from CameraPlus with the latest 0harmony.dll from Harmony.

Crashes with the following after trying to generate the map.
http://pastebin.com/raw/Ewx4MCeZ

RawCode

Well, to avoid direct spoilers, i created little thread with basics (my own experience) explained (somewhat).
https://ludeon.com/forums/index.php?topic=29861.0

Brrainz

YAY!!!

Well, I did it. I just finished rewriting the core of Harmony. Better, faster, no crashes and you can even manipulate the original method directly. Tested and with almost no assembler low level stuff (just two tiny mono methods via dllimport.

Guys, I am so happy that I finally got this working. Tomorrow, I merge my test project with Harmony and update the documentation. Stay tuned!

/Andreas 8)

PS: here is what you can do as an example:


using Verse;
using System;
using RimWorld;
using UnityEngine;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;

namespace Test
{
[StaticConstructorOnStartup]
public static class TestMain
{
static TestMain()
{
FileLog.Reset();
FileLog.Log("DynamicPatcher test started at " + DateTime.Now);

var p1 = new DynamicPatcher(CompileTools.Method(typeof(SelectionDrawer), "Notify_Selected"));
p1.AddPrefix(CompileTools.Method(typeof(Patches), "Notify_Selected_Prefix"));
p1.Patch();

var p2 = new DynamicPatcher(CompileTools.Method(typeof(SelectionDrawerUtility), "CalculateSelectionBracketPositionsWorld", null, new Type[] { typeof(object) }));
p2.AddPostfix(CompileTools.Method(typeof(Patches), "CalculateSelectionBracketPositionsWorld_Postfix"));
p2.Patch();

var p3 = new DynamicPatcher(CompileTools.Method(typeof(FloatMenuMakerMap), "AddDraftedOrders"));
p3.AddPostfix(CompileTools.Method(typeof(Patches), "AddDraftedOrders_Postfix"));
p3.Patch();

var p4 = new DynamicPatcher(CompileTools.Method(typeof(FloatMenuMakerMap), "AddUndraftedOrders"));
p4.AddPostfix(CompileTools.Method(typeof(Patches), "AddUndraftedOrders_Postfix"));
p4.Patch();

var p5 = new DynamicPatcher(CompileTools.Method(typeof(FloatMenuMakerMap), "AddHumanlikeOrders"));
p5.AddPostfix(CompileTools.Method(typeof(Patches), "AddHumanlikeOrders_Postfix"));
p5.Patch();

var innerType = typeof(RCellFinder).GetNestedTypes(CompileTools.all).First(t => t.Name.Contains("BestOrderedGotoDestNear"));
var p6 = new DynamicPatcher(CompileTools.Method(innerType, "<>m__60C"));
var p6_m1 = CompileTools.Method(typeof(PawnDestinationManager), "DestinationIsReserved", new Type[] { typeof(IntVec3), typeof(Pawn) });
var p6_p1 = CompileTools.Method(typeof(Patches), "DestinationIsReserved_Never");
var p6_m2 = CompileTools.Method(typeof(GenGrid), "Standable");
var p6_p2 = CompileTools.Method(typeof(Patches), "Standable_Always");
p6.AddModifier(new ILCode(p6_m1), new ILCode(p6_p1));
p6.AddModifier(new ILCode(p6_m2), new ILCode(p6_p2));
p6.Patch();

var p7 = new DynamicPatcher(CompileTools.Method(typeof(MemoryThoughtHandler), "TryGainMemoryThought", new Type[] { typeof(Thought_Memory), typeof(Pawn) }));
p7.AddPostfix(CompileTools.Method(typeof(Patches), "TryGainMemoryThought_Postfix"));
p7.Patch();

var p8 = new DynamicPatcher(CompileTools.Method(typeof(Pawn_FilthTracker), "Notify_EnteredNewCell"));
p8.AddPrefix(CompileTools.Method(typeof(Patches), "Notify_EnteredNewCell_Prefix"));
p8.AddPostfix(CompileTools.Method(typeof(Patches), "Notify_EnteredNewCell_Postfix"));
p8.Patch();

FileLog.Log("Done");
FileLog.Log("");
}
}

public static class Patches
{
// Demo 1: cast letters for all selected pawns
//
public static void Notify_Selected_Prefix(object t)
{
var pawn = t as Pawn;
if (pawn != null && pawn.NameStringShort != null && pawn.NameStringShort.Length > 0)
Find.LetterStack.ReceiveLetter(new Letter(pawn.NameStringShort, "You selected this pawn", LetterType.Good));
}

// Demo 2: expand selection brackets to show them apart from the selected object
//
public static void CalculateSelectionBracketPositionsWorld_Postfix(Vector3[] bracketLocs)
{
bracketLocs[0] = bracketLocs[0] + new Vector3(-.5f, 0, -.5f);
bracketLocs[1] = bracketLocs[1] + new Vector3(+.5f, 0, -.5f);
bracketLocs[2] = bracketLocs[2] + new Vector3(+.5f, 0, +.5f);
bracketLocs[3] = bracketLocs[3] + new Vector3(-.5f, 0, +.5f);
}

// Demo 3-5: add extra menu choice into context menu
//
public static void AddDraftedOrders_Postfix(List<FloatMenuOption> opts)
{
opts.Add(new FloatMenuOption("DRAFTED", null));
}
public static void AddUndraftedOrders_Postfix(List<FloatMenuOption> opts)
{
opts.Add(new FloatMenuOption("UNDRAFTED", null));
}
public static void AddHumanlikeOrders_Postfix(List<FloatMenuOption> opts)
{
opts.Add(new FloatMenuOption("HUMANLIKE", null));
}

// Demo 6: Modifying the Predicate inside BestOrderedGotoDestNear() so a destination is never reserved and everything
//         is standable
public static bool DestinationIsReserved_Never(PawnDestinationManager p1, IntVec3 p2, Pawn p3)
{
return false;
}
public static bool Standable_Always(IntVec3 p1, Map p2)
{
return true;
}

// Demo 7: Log all old and new thoughts to error log
//
public static void TryGainMemoryThought_Postfix(MemoryThoughtHandler __instance, Thought_Memory newThought)
{
if (__instance.Memories.LastOrDefault() == newThought)
Log.Error("New thought from " + newThought.pawn.NameStringShort + ": " + newThought.LabelCap + " [" + newThought.MoodOffset() + "]");
}

// Demo 8: track changes to the filth a pawn carries
// (here, we use __state to compare before/after call to Notify_EnteredNewCell)
//
public static void Notify_EnteredNewCell_Prefix(ref object __state, Pawn_FilthTracker __instance)
{
__state = ((List<Filth>)typeof(Pawn_FilthTracker)
.GetField("carriedFilth", BindingFlags.NonPublic | BindingFlags.Instance)
.GetValue(__instance)).ToArray();
}
//
public static void Notify_EnteredNewCell_Postfix(object __state, Pawn_FilthTracker __instance)
{
var oldFilth = ((Filth[])__state).ToList();
var oldFilthSum = 0;
oldFilth.ForEach(f => oldFilthSum += f.thickness);

var newFilth = (List<Filth>)typeof(Pawn_FilthTracker)
.GetField("carriedFilth", BindingFlags.NonPublic | BindingFlags.Instance)
.GetValue(__instance);
var newFilthSum = 0;
newFilth.ForEach(f => newFilthSum += f.thickness);

if (oldFilthSum == newFilthSum) return;
var diff = "" + (newFilthSum - oldFilthSum);
if (newFilthSum - oldFilthSum > 0) diff = "+" + diff;

var pawn = (Pawn)typeof(Pawn_FilthTracker).GetField("pawn", CompileTools.all).GetValue(__instance);
Log.Warning(pawn.NameStringShort + " filth " + diff + " at " + pawn.Position.x + "," + pawn.Position.z);
}

}
}

RawCode

i hope you will explain what was wrong and how things are fixed?

Brrainz

Quote from: RawCode on February 01, 2017, 02:20:01 AM
i hope you will explain what was wrong and how things are fixed?
Absolutely. Just polishing the code a bit

Brrainz

Quote from: RawCode on February 01, 2017, 02:20:01 AM
i hope you will explain what was wrong and how things are fixed?
Hi RawCode. I released a fixed version (1.0.6) of Harmony. The mistake that I made was so stupid that I am too embarrassed to tell. Maybe you find it out (hint: has NOTHING to do with low level stuff at all).

Brrainz

HARMONY 1.0.6 - the future of patching code at runtime

So this is it. I have release a public preview of Harmony on GitHub: https://github.com/pardeike/Harmony/releases/tag/v1.0.6

Everyone is welcome to try it out. It most certainly will find its way in the popular community libraries as well (HugsLib). However, the design is done in a way so modders can use it in their own mods without any conflict with the Harmony version that is used by other mods or by a common library like HugsLib.

The github repository has a ready made version of the lib in dll form and you can either add it to your Assemblies folder (add reference and keep "Copy" on) or merge it with a tool like ILMerge into your own dll. I strongly recommend to test things before going wild and releasing stuff to all users.

Documentation exists in the form of a wiki on github: https://github.com/pardeike/Harmony/wiki or you can download two mods I made for demonstration purpose: SameSpot (https://github.com/pardeike/SameSpot) or Camera+ (https://github.com/pardeike/CameraPlus). SameSpot has some advanced technique to modify the original method so I recommend Camera+ to begin with.

Thanks for your patience and support. Please try out this release and keep the feedback coming.

Cheers,
Andreas Pardeike

RawCode

Quote from: pardeike on February 04, 2017, 06:41:29 AM
Quote from: RawCode on February 01, 2017, 02:20:01 AM
i hope you will explain what was wrong and how things are fixed?
Hi RawCode. I released a fixed version (1.0.6) of Harmony. The mistake that I made was so stupid that I am too embarrassed to tell. Maybe you find it out (hint: has NOTHING to do with low level stuff at all).

you hooked into "Invoke" method instead of actual code of dynamic methods?

good job anyway.

MaximStale

#57
You are best!


scuba156

I haven't had the time to try it out yet, but congratulations on getting it working. Looking forward to using it in the future.