New code injection method

[Micktu]

It's finished!

Tested on Windows, OSX, and Ubuntu x64 (and I assume x86 would work too). I also believe it's stable and reliable for most methods. Still can't hook into IEnumerable<> generators though.

It can also stack overrides on the same function multiple times (if multiple mods need to hook to the same thing), and correctly handles CCL-detoured methods, so you can hook to these too.

You need these files (these are direct links): HookInjector.cs Platform.cs AsmHelper.cs - they will be moved into master branch some time later.

Define your override:

Code Select Expand

internal static void PreLoadUtility_CheckVersionAndLoad(string path, ScribeMetaHeaderUtility.ScribeHeaderMode mode, Action loadAct)
{
    Log.Message("!!! CheckVersionAndLoad");
    PreLoadUtility.CheckVersionAndLoad(path, mode, loadAct);
}

And then hook it up:

Code Select Expand

HookPatcher = new HookInjector();
HookPatcher.Inject(typeof(PreLoadUtility), "CheckVersionAndLoad", typeof(VerseExtensions));
As you can see, if you don't specify the target method name, it defaults to YourClass.SourceClassName_SourceMethodName().

You don't need to create multiple injectors (although it won't break if you do), one is enough.

[1000101]

Interesting code.

It could use some polish and needs some work to match the coding style and structure before being added to CCL though.

I'm eager to see a complete working example. 

[Micktu]

>It could use some polish and needs some work to match the coding style and structure before being added to CCL though.
Nice joke, the polish and coding style of college students, right. Actually, you know what, don't touch the source code, I'll make it an assembly and you can use that if you wish.

>I'm eager to see a complete working example
So make it. Test it, mr. part of the community.

[RawCode]

doh...

This is not about making blackbox assembly and giving it to other, it's about making code and explaining how it works, to allow other individuals to learn how it works.

In longrun, such activity will allow others to improve base work, just like it happened with my initial commit, improved by community to handle x64.
After researching changes, i managed to greatly improve base work, result is ABS hooks posted recently.

"Coding style" do matter, if base code implement "cryptic" features like indirect invocation, magic numbers, offsets instead of names, "var", "using" or homebrew lambdas.
Reason behind is simple - such code harder to understand and follow for others.

I have private builds that use "stuff" including "var" and 3 *random* letters as varname, but, i never post "raw" code on forum, because it looks like after obfuscation and impossible to follow.

[Micktu]

Oh no no no, var is a production-quality blessing. I scold people for not using that, because BaseContainerInfoHandler baseContainerInfoHandler = new BaseContainerInfoHandler() is obviously fun and improves readability immensely. That's exactly why Java is unusable, as well as C++ before C++11.

I agree on everything else though, no magic number, no ambiguity. That thing was rewritten about 4 times, I'm sure every line is in its place besides a couple of line breaks and brackets. I went ways to actually make it clean and readable, and I'm satisfied with the result.

This is also the last time I argue about style.

Back on topic, what I overlooked is that you have to be able to Invoke() from inside of your stuff to call private base methods, but now you obviously end up in a stack overflow and crash. I'll make it do an extra check later.

[Micktu]

After a little though I believe checking for Invoke wouldn't work, because there are other places that might want to Invoke your method if it's a delegate.

We need figure out a way to call private methods directly if possible, maybe through IL preprocessing. RawCode please try to think about it.

Or I'll make an alternative check that does not involve call stack. Someeting dumb like setting a flag when you enter your method and clearing it when you leave it. Not thread-safe of course.

[Micktu]

So when I found out that on x64 stack alloc sometimes is a single instruction and can't be reliably replaced with a jump, I weighted different solutions and tried copying the whole method as a rescue. I assumed it's rare, and it was only small methods that can be this way and that it will work in most cases.

Turned out I was wrong, it's very common:



And the methods can be huge:

But the good news is that it works perfectly well. It is even is able to resolve its trampolines successfully in its new location (thanks mono programmers).

Unfortunately this won't work on x86 because of relative addressing that has to be handled, and that increases complexity too much to be nice. But then again, it's not a problem on x86 because I'm yet to see a stack alloc pattern that's shorter than 5 bytes.

By the way I'm still patching with relative jumps in x64 because the chances that memory pages will end up 2Gb apart are maybe one in a thousand years.

[RawCode]

I never hit any issues with recursive calling, i need some kind of example to implement a fix.

Also stack allocation for methods is not issue at all, just like methods below 5 bytes.

You can change stack allocation for hooked methods, it does not matter as long as you restore stack pointer correctly before return, in other case runtime crash due to jump into invalid memory offset.
(return opcode is actually pop and jmp, call is push and jmp, i abused this feature to implement indirect invocation - last method of ABS hook)

Methods below 5 bytes ALWAYS inlined (only exception is noinline flag), you can't hook them  anyway, ever if you manage to hook them, this won't have any effect with exception to reflection based invocations.

And yes, methods can be very large, especially methods with exception handling (also they have return opcode in unexpected locations)

[Micktu]

>I never hit any issues with recursive calling, i need some kind of example to implement a fix.
It's obvious: I check for return address. If you Invoke, the return address is not the function you're checking against, but mono_runtime_invoke's address. Therefore your function gets called again and you crash with stack overflow.

>Also stack allocation for methods is not issue at all, just like methods below 5 bytes.
It's not, reliable detection of what can be copied and replaced is. It's not trivial unless you write your own disassembler thing to analyze the instructions. Which is a good solution, but it increases complexity manyfold.

>And yes, methods can be very large, especially methods with exception handling (also they have return opcode in unexpected locations)
What I meant is that I assumed that methods that do not make use of stack variables are usually small, and this turns out to not be true.

[Micktu]

Anyway, the current working example is now up on workshop, imma cross my fingers and hope it works for everyone. The source is at https://github.com/micktu/RimWorld-BuildProductive if you want to take a look at usage.

Another problem I stumbled upon is that sometimes hooking into certain methods ends up in segfault on stack allocation for no apparent reason in random places within execution chain. E.g you can't hook to Designator.DesignateThing(), it's called from a Command's delegate, and I think it happens when the hooked method is called from within a delegate. Hope I arrive at a better explanation later. Ah, and it only happens with x64 compiler. I'll try some things like adding prolog to such functions, or maybe try to wrap them in a delegate of my own and see if it works.

[RawCode]

It's obvious: I check for return address. If you Invoke, the return address is not the function you're checking against, but mono_runtime_invoke's address. Therefore your function gets called again and you crash with stack overflow.

its your fault, same will happen if you manage to hook constructor or allocator, if pointer does not belong to managed method, you should not modify it.

[Micktu]

So there have been dozens of new installs, and nobody complained yet, so I assume the method is working an stable. Feel free to use it people, it's the Injection folder in the repo.

I'll make it handle invokes properly later.