Ludeon Forums

RimWorld => Mods => Help => Topic started by: RawCode on December 24, 2015, 05:38:45 AM

Title: Mono x86-32 JIT machine code hooks
Post by: RawCode on December 24, 2015, 05:38:45 AM

unsafe static public int HookMethodFromTo
(
Type source_type, string source_method_name,
Type destination_type, string destination_method_name,
bool usejump)
{
MethodInfo Source_Method = source_type.GetMethod (source_method_name, (BindingFlags)(60));
MethodInfo Destination_Method = destination_type.GetMethod (destination_method_name, (BindingFlags)(60));

if (Source_Method == null || Destination_Method == null)
return 404;

int Source_Base = Source_Method .MethodHandle.GetFunctionPointer ().ToInt32();
int Destination_Base = Destination_Method.MethodHandle.GetFunctionPointer ().ToInt32();

int offset_raw = Destination_Base - Source_Base;
uint* Pointer_Raw_Source = (uint*)Source_Base;

if (usejump)
{

*(Pointer_Raw_Source+0) = 0xE9909090;
*(Pointer_Raw_Source+1)= (uint)(offset_raw-8);
}
else
{
*(Pointer_Raw_Source+0) = 0x83ec8b55;
*(Pointer_Raw_Source+1) = 0xe89008ec;
*(Pointer_Raw_Source+2) = (uint)(offset_raw-12);
*(Pointer_Raw_Source+3) = 0x9090c3c9;
}

return 0;
}


nuff said.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on December 24, 2015, 05:40:59 AM
Difference "usejump" is:

Entry:Main
   at System.Environment.get_StackTrace() in C:\cygwin\sources\mono\mcs\class\corlib\System\Environment.cs:line 261
   at RWX.MainClass.dummy_return_B()
   at RWX.MainClass.dummy_return_A()
   at RWX.MainClass.Main(System.String[] args)
method B internal
METHOD A RETURN 64


vs

Entry:Main
   at System.Environment.get_StackTrace() in C:\cygwin\sources\mono\mcs\class\corlib\System\Environment.cs:line 261
   at RWX.MainClass.dummy_return_B()
   at RWX.MainClass.Main(System.String[] args)
method B internal
METHOD A RETURN 64


for call

   HookMethodFromTo(typeof(MainClass),"dummy_return_A",typeof(MainClass),"dummy_return_B",false\true);
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on December 24, 2015, 06:02:59 PM
omgomgomg if this is what I think it is I frigging love you. I'm too drunk/tired right now to try it out, but my family may not see much of me tomorrow if this actually works :P
Title: Re: Mono x86-32 JIT machine code hooks
Post by: 1000101 on December 25, 2015, 01:33:58 AM
The jump method would be more efficient too as there is less stack pushing/popping which has the secondary advantage of keeping more stack space available to methods called within the new method call chain.

Anyway, great job, I had tried to do similar things months ago but made no headway.  If you're open it, I'd like to incorporate a simple API into CCL to do method replacements.  Full credit for getting the code to work will go to you, of course.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on December 25, 2015, 06:36:20 AM
You can use code in CCL without any restrictions.
Suggested to omit call section and use jump only.
Call implemented for comparison.

Will need some testing with nonwindows platforms.

I will contribute to CCL when got some time by providing "standard hooks" and simple event handling.

Without event handling system similar to bukkit\forge only one mod will be allowed to hook same method.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on December 25, 2015, 07:02:38 AM
I haven't had time to experiment yet, but I wonder, will this handle methods with parameters at all? I ask because you're finding methods by name, so overloads are always going to be ambiguous?

Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on December 25, 2015, 09:43:17 AM
addition of single "s" will allow to return array of methods including all overloads...

this is not complete ready to use API, this is proof of concept.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on December 25, 2015, 02:00:51 PM
Fair enough, I get that. I was just afraid it might not work with parameters at all, and sadly because of this christmas family stuff I haven't had much time to play with it yet ;)
Title: Re: Mono x86-32 JIT machine code hooks
Post by: 1000101 on December 25, 2015, 04:33:02 PM
As far as parameters are concerned, they would have to match the original function prototype.  Since the function being replaced is simply jumping to the replacement (as opposed to using the call method which would push a new return address), there would be nothing new on the stack to invalidate stack pointers.  The first parameter will still be at esp+04, the second at esp+08, etc.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on December 25, 2015, 05:11:33 PM
Mornin' fokes... been messing around with the function and now I got dis:
        unsafe static public int HookMethodFromTo (string source_method_name, string destination_method_name)
        {
            Type source_type;
            Type destination_type;

            //Drop the brackets...
            source_method_name      = source_method_name.Remove     (source_method_name.Length      - 2);
            destination_method_name = destination_method_name.Remove(destination_method_name.Length - 2);

            //Split MethodName into its parts...
            string[] source_method_name_parts      = source_method_name     .Split('.');
            string[] destination_method_name_parts = destination_method_name.Split('.');

            //Get the name of the function
            string source_method_name_end      = source_method_name_parts     [source_method_name_parts.Length - 1];
            string destination_method_name_end = destination_method_name_parts[source_method_name_parts.Length - 1];

            //Console.WriteLine(source_method_name_end + " AND " + destination_method_name_end);

            //Get the class name...
            source_method_name       = source_method_name    .Remove(source_method_name     .Length - source_method_name_end     .Length - 1);
            destination_method_name = destination_method_name.Remove(destination_method_name.Length - destination_method_name_end.Length - 1);

            //Console.WriteLine(source_method_name + " AND " + destination_method_name);

            //Convert the string into a class type
            source_type       = Type.GetType(source_method_name);
            destination_type = Type.GetType(destination_method_name);

            //Check for errors...
            if (source_type == null || destination_type == null)
            {
                Console.WriteLine("NO Type - 404 ERROR");

                return 404;
            }

            //Grab the methodinfo...
            MethodInfo Source_Method        = source_type      .GetMethod(source_method_name_end,       (BindingFlags)(60));
            MethodInfo Destination_Method = destination_type.GetMethod(destination_method_name_end, (BindingFlags)(60));

            if (Source_Method == null || Destination_Method == null)
            {
                Console.WriteLine("NO MethodInfo - 404 ERROR");

                return 404;
            }

            //Get where the function is...
            int Source_Base       = Source_Method.MethodHandle       .GetFunctionPointer().ToInt32();
            int Destination_Base = Destination_Method.MethodHandle.GetFunctionPointer().ToInt32();

            //Calculate the diffrence between the 2 function's locations
            int offset_raw   = Destination_Base - Source_Base;

            uint* Pointer_Raw_Source  = (uint*) Source_Base;

            // [WEIRD POINTER MATH] //
            //From RawCode
            *(Pointer_Raw_Source + 0) = 0xE9909090;
            *(Pointer_Raw_Source + 1) = (uint)(offset_raw - 8);
            // [/WEIRD POINTER MATH] //

            return 0;
        }

The function can be called like this:
HookMethodFromTo("Full.Path.To.Function()",
                 "Another.Full.Path.To.Another.Function.Thingy()");


Example code:
using System;
using System.Reflection;

namespace Function_Far_Far_Away
{
    class RimAPIFunctions
    {
        static internal void AAPI()
        {
            Console.WriteLine("A API");
        }
        static public void BAPI()
        {
            Console.WriteLine("B API");
        }
    }
}

namespace Function_Jumping
{
    class Program
    {
        static void Main(string[] args)
        {

            Function_Far_Far_Away.RimAPIFunctions.AAPI();
            Function_Far_Far_Away.RimAPIFunctions.BAPI();
            C();

            Console.WriteLine("HIT ENTER!!! \n\r \n\r");
            Console.ReadLine();

            //Hook Method
            FunctionJumper.HookMethodFromTo("Function_Far_Far_Away.RimAPIFunctions.AAPI()",
                                            "Function_Jumping.RimFunctions.A()");

            FunctionJumper.HookMethodFromTo("Function_Far_Far_Away.RimAPIFunctions.BAPI()",
                                           "Function_Jumping.RimFunctions.B()");

            FunctionJumper.HookMethodFromTo("Function_Jumping.Program.C()",
                                            "Function_Jumping.RimFunctions.C()");

            Function_Far_Far_Away.RimAPIFunctions.AAPI();
            Function_Far_Far_Away.RimAPIFunctions.BAPI();
            C();

            Console.WriteLine("HIT ENTER!!! \n\r \n\r");
            Console.ReadLine();
        }

        static private void C()
        {
            Console.WriteLine("C Override");
        }
    }

    class RimFunctions
    {
        static public void A()
        {
            Console.WriteLine("A NORM");
        }
        static protected void B()
        {
            Console.WriteLine("B NORM");
        }
        static private void C()
        {
            Console.WriteLine("C NORM");
        }
    }

    class FunctionJumper
    {
        unsafe static public int HookMethodFromTo (string source_method_name, string destination_method_name)
        {
            Type source_type;
            Type destination_type;

            //Drop the brackets...
            source_method_name      = source_method_name.Remove     (source_method_name.Length      - 2);
            destination_method_name = destination_method_name.Remove(destination_method_name.Length - 2);

            //Split MethodName into its parts...
            string[] source_method_name_parts      = source_method_name     .Split('.');
            string[] destination_method_name_parts = destination_method_name.Split('.');

            //Get the name of the function
            string source_method_name_end      = source_method_name_parts     [source_method_name_parts.Length - 1];
            string destination_method_name_end = destination_method_name_parts[source_method_name_parts.Length - 1];

            //Console.WriteLine(source_method_name_end + " AND " + destination_method_name_end);

            //Get the class name...
            source_method_name      = source_method_name     .Remove(source_method_name     .Length - source_method_name_end     .Length - 1);
            destination_method_name = destination_method_name.Remove(destination_method_name.Length - destination_method_name_end.Length - 1);

            //Console.WriteLine(source_method_name + " AND " + destination_method_name);

            //Convert the string into a class type
            source_type      = Type.GetType(source_method_name);
            destination_type = Type.GetType(destination_method_name);

            //Check for errors...
            if (source_type == null || destination_type == null)
            {
                Console.WriteLine("NO Type - 404 ERROR");

                return 404;
            }

            //Grab the methodinfo...
            MethodInfo Source_Method      = source_type     .GetMethod(source_method_name_end,      (BindingFlags)(60));
            MethodInfo Destination_Method = destination_type.GetMethod(destination_method_name_end, (BindingFlags)(60));

            if (Source_Method == null || Destination_Method == null)
            {
                Console.WriteLine("NO MethodInfo - 404 ERROR");

                return 404;
            }

            //Get where the function is...
            int Source_Base      = Source_Method.MethodHandle     .GetFunctionPointer().ToInt32();
            int Destination_Base = Destination_Method.MethodHandle.GetFunctionPointer().ToInt32();

            //Calculate the diffrence between the 2 function's locations
            int offset_raw = Destination_Base - Source_Base;

            uint* Pointer_Raw_Source  = (uint*) Source_Base;

            // [WEIRD POINTER MATH] //
            //From RawCode
            *(Pointer_Raw_Source + 0) = 0xE9909090;
            *(Pointer_Raw_Source + 1) = (uint)(offset_raw - 8);
            // [/WEIRD POINTER MATH] //

            return 0;
        }
    }
}


Output:

A API
B API
C Override
HIT ENTER!!!



A NORM
B NORM
C NORM
HIT ENTER!!!


Full credit to RawCode for idea...
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on December 25, 2015, 10:29:19 PM
Jump work with parameters and return just fine from the box.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on December 26, 2015, 02:30:09 AM
I've been interested in what your doing, so I got the code to print out what your inserting using the pointers, and the next 8 value and the 8 values before it (cause context is everything).

The output was:
A API
B API
C Override
HIT ENTER!!!



2567422952
24176
19270860
0
2567418856
24176
19270988
0
are the 8 aresses before it...
3918565520
64
Were inserted at:
23727360
and
2210793
17063680
2567410664
548464
19271392
0
0
0
are the next 8 adresses...

19270860
0
2567418856
24176
19270988
0
3918565520
64
are the 8 aresses before it...
3918565520
64
Were inserted at:
23727368
and
2567410664
548464
19271392
0
0
0
2239465
16801536
are the next 8 adresses...

1683298488
3924658433
7236
1683264696
3924658433
1889078296
2194409
33578752
are the 8 aresses before it...
3918565520
288
Were inserted at:
23727152
and
2567465960
548464
19269720
0
0
0
2567459816
24176
are the next 8 adresses...

A NORM
B NORM
C NORM
HIT ENTER!!!


So then I arranged the address and converted to Hex (a very painful procces):

Atmp 1: At 16A0D00
9907C3E8
5E70
1260CCC
0
9907B3E8
5E70
1260D4C
0
E9909090
40
21BBE9
1045F00
990793E8
85E70
1260EE0
0
0
0

Atmp 2: At 16A0D08
1260CCC
0
9907B3E8
5E70
1260D4C
0
E9909090
40
E9909090
40
990793E8
85E70
1260EE0
0
0
0
222BE9
1005F00

Atmp 3: At 16A0C30
645518B8
E9ED8901
1C44
645494B8
E9ED8901
70990C18
217BE9
2005F00
E9909090
120
99086BE8
85E70
1260858
0
0
0
990853E8
5E70


Well... that's utter gibberish to me... so I got an disassembler and did it on all 3 parts... i got dis:
https://www.onlinedisassembler.com/odaweb/f1x3G0GS/0
https://www.onlinedisassembler.com/odaweb/xmmXha51/0
https://www.onlinedisassembler.com/odaweb/9pFiKhmn/0 (Odd, E9909090 wasn't split into 'jmp 99090' then 'NOP', I assume its faulty disassembling)

The common thing between the 3 is that you jump to 9090 (where either that is), then do nothing (NOP literately means do nothing), then add a random function (the difference between the 2 pointers).

Of course, my question is that if you jump to 9090 then why insert something else? NOP does nothing, and inserting a random function is sure to crash the program. I'm I missing something?
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on December 26, 2015, 06:37:29 AM
1) Endianness
2)

int TMP = 0;
int I = 0;
bool resflag = false;
byte* POINTER_1x = (byte*)POINTER_1;
for (;;)
{
TMP = *(POINTER_1x + I);
I++;
Console.WriteLine (TMP.ToString ("X2"));

if (TMP == 0xC3)
if (resflag)
break;
resflag = false;

if (TMP == 0xC9 && !resflag)
{
resflag = true;
continue;
}
}


3) You need assembly and c knowledge to play with mono internals (basic level will be fine, but you need it)

4) E9909090 is

DO NOTHING (0x90)
DO NOTHING (0x90)
DO NOTHING (0x90)
JUMP NEAR (0xE9)

5) Why nops? answer is alignment.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on January 02, 2016, 01:24:28 AM
Hohoho, todays Christmas New years present is EVENTS...

GET IT HERE:
https://files.slack.com/files-pri/T07L8SKEY-F0HJM0NNN/download/code_example_followed_by_function_itself___.txt

You'll need to copy the whole class ONCE FOR EVERY FUNCTION in RW... enjoy the fun...
Please note that functionToOverrideName needs to be the name of the function you'll override BEFORE ya run the Init() function... after that your too late too change it.

Since you, RawCode, are the master of convoluted (no offence :) ) code I'd be joyful if you could find some sort of convoluted (no offense once again) method to insure that we some unlucky poor sod doesn't have to copy it 8bil ^ 9bil times...

Ty in advanced...
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 02, 2016, 07:40:59 AM
slack force registration.

use pastebin instead.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on January 02, 2016, 02:45:25 PM
Opps, sorry mate...

You can get it here:
https://gitlab.com/RimGentzCo/FunctionJumping/tree/master

And yes, its set to public so ya don't have to login...
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 03, 2016, 02:53:59 AM
there is no reason to copy classes since you can use generics or emit code at runtime

oncall injections are useless sine they do not allow to change outcome of invokation without special implementation.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on January 03, 2016, 01:40:03 PM
Quote from: RawCode on January 03, 2016, 02:53:59 AM
there is no reason to copy classes since you can use generics or emit code at runtime

How would generics solve the problem? How would generics change OnCalled's function signature, or OnCalledGate's function signature?

        public static void OnCalledGate()   
        {
            if (OnCalled != null)
                OnCalled();  //How do generics pass parameters forward?
        }



Why would you emit code at runtime? How would you emit code at runtime? What is 'emitting code at runtime'?

Srsly tho, I'm not the pro assembly coder (I don't even know assembly). In all my yeas of coding C++, for the most part I've avoided lambas, pointers, C, Inline Assembly, Generics and Overriding Operators... I'd always pass them over to some poor sod to do for me. Saying "emit code at runtime to solve problem" has no meaning to me.

Quote from: RawCode on January 03, 2016, 02:53:59 AM
oncall injections are useless sine they do not allow to change outcome of invokation without special implementation.

What? How do they NOT change the outcome?

I clearly showed calling the same 3 functions gave different results every time using my events system, does that NOT change outcome?

The output was:
A API
B API
C OVERRIDE
81
HIT ENTER!!!



A NORM
B NORM
C NORM
9
HIT ENTER!!!



0
HIT ENTER!!!



A NORM
B NORM
A NORM
B NORM
C NORM
9
HIT ENTER!!!

Is that not different results from calling the same 3 functions?
Did you even compile the program?


I asked for CONVOLUTED answers, not OBSCURE answers.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 03, 2016, 11:28:40 PM

function void blabla(some arguments)
<injection point>
calculate damage, not invocation, just some fancy math
do damage to pawn(local)
do damage to environment(arg)
increment some counter()
corrupt soul of developer()
<injection point>
return 42


Plan A:
Please remove soul corruption effect with your "framework".

Plan B:
Please alter pawn damage, special note, it is not just passed as argument, it is calculated inside function.

note: outcome != returned value.

Actually both "plans" are perfectly possible, but in both cases there is absolutely no need for delegates and additional classes, just event bus provided by api and nothing more
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on January 04, 2016, 10:22:30 AM
Quote from: RawCode on January 03, 2016, 11:28:40 PM

function void blabla(some arguments)
<injection point>
calculate damage, not invocation, just some fancy math
do damage to pawn(local)
do damage to environment(arg)
increment some counter()
corrupt soul of developer()
<injection point>
return 42


Plan A:
Please remove soul corruption effect with your "framework".
>>1- I never said its a framework.
2- Please remove 'soul corruption effect' with your detours. Inlighten me how?
RawCode's detours:
    1-Copy past 'blabla' into 'blabla2'
    2-Remove 'soul corruption effect' from 'blabla2'
    3-Detour from 'blabla' to blabla2'.
Now here is how ya do it in mine:
    1-Copy past 'blabla' into 'blabla2'
    2-Remove 'soul corruption effect' from 'blabla2'
    3-Init event
    4-Suscribe to event


Plan B:
Please alter pawn damage, special note, it is not just passed as argument, it is calculated inside function.
>>Just Override the WHOLE GODDAM function. Now was that hard?

note: outcome != returned value.
>>FALSE!

Since you only seam to understand code, let me give you this:

public static bool IsRawcodeInCorrect(){
if  (NumOfOutcomes == 1){Console.WriteLine("Returned value is the outcome!"); return true;}
bool outcomesEqual = true; Outcome firstOutcome = outcomes[0]; outcomes.Remove(0);
foreach (Outcome _outcome in outcomes[]){if (_outcome != firstOutcome) {outcomesEqual = false; break;}}
If (outcomesEqual){Console.WriteLine("All outcomes are equal so the last outcome is the return value!"); return true;}else{Console.WriteLine("Gosh, RawCode is correct... :("); return false;}}


Output: True. In both examples only ONE function was a subscriber.

IF more than 1 delegates subscribe THEN we will only return the LAST outcome, as we can't return all of them.

Actually both "plans" are perfectly possible, but in both cases there is absolutely no need for delegates and additional classes, just event bus provided by api and nothing more

Implement an event buss then. You have done nothing but criticize my work with false information.

PS: C# Events NEED delegates to work, so go on implementing your custom event system.




RawCode + not reading code + criticizing my hard work = Pain for RawCode
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 05, 2016, 12:03:37 AM
QuoteImplement an event buss then. You have done nothing but criticize my work with false information.

done nothing...
kay.

Quotenote: outcome != returned value.
>>FALSE!

you just wasting my time.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on January 06, 2016, 07:04:02 AM
wow gent, cool down a bit. RawCode's done a significant chunk of research and work to get the whole detour thing to be possible in the first place - that's not exactly nothing.

As for your 'event system', by your own admission it's currently pretty much useless because you have to create a gazillion extra classes. So what did you expect? A pat on the head?

As for C# needing delegates for events, that may or may not be true, but C# also doesn't normally allow detouring/hooking function calls, yet with some black magic RawCode made that possible. I don't know if a proper simple to use event system is possible/feasible, but I know yours isn't it, and that if anyone can make it happen it's probably RawCode.

P.S. I also had no idea what 'emiting code' could possibly mean. However, we both clearly have internet access, a quick google search for C# emit code gives a few interesting results. Instead of criticizing the man for giving answers you don't understand, maybe try a little bit harder to understand first.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 08, 2016, 05:33:03 AM
Issue is simple, each *event* MUST have "human made" body that specially designed to handle possible changes from mods using API.

There is no way to perform this automatically for most methods for "reasons":


set field
call method that include field
perform calculations that include previous set and method invocations
call do_pawn_damage(calculated value)


Since field set and method invocation are both "outcome changing" such things are not allowed to happen before handlers process "event" and decide, is such event allowed to happen or not and how it should happen.

Sadly, there is no way to pass information about event to bus, since such information not yet exists.
Lines before actual damage used to calculate damage may include damage to armor, damage to shields, various counter and stuff.

Just moving damage part to event will result in ugly thing: pawn "blocked damage by jedi trick" but armor and shield still got damage and bleeding wound is here...

Only viable way to handle such situation - to provide different implementation that can handle all stuff properly, calculate everything in "non outcome changing" manner.

There is no need in custom class per each event, "this" is argument #1 and can be processed inside static method just fine.
There is no need in delegates, as probably nobody aware - delegates are syntax sugar and will be compiled into ugly stuff that include hidden interface and other ugly things.

As for private methods and fields - i will provide magical "stuff" that will allow to both compile and execute code that violate all protection rules soonish.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: akisute on January 17, 2016, 04:06:06 AM
Well, as a Mac player, I sometimes want you guys to remember other platforms :'(

https://github.com/RimWorldCCLTeam/CommunityCoreLibrary/issues/54

The current implementation in CCL inspired by code here doesn't work in Mac (most likely Linux too). This is not a matter of your code but CLL, but...

I'm not a hardcore C# developer but I still understand you have to do some real black magic to detour (or hook) the function calls without touching the prebuilt code of Core Rimworld. If you could happen to find out the black magic that works in all platforms that would be awesome...
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 18, 2016, 05:25:30 AM
It will work on all platforms, issue is endianess, not x64 or different version of mono (mono shipped with game and have fixed version).

Single check and 2 lines to handle endianess and you will have magic on mac\linux.

Actually support of all platforms and all possible cases is reason why i avoid going native, all this much more simple to implement in pure C.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: 1000101 on January 19, 2016, 02:20:43 PM
Modern Macs use x86 so endianness isn't an issue.

Some functions (methods/properties/whatever you want to call them) can't be detoured for "reasons."  Some of these reasons include but are not limited to, tighter memory restrictions (code sections may not be writeable), the original function may be inlined by the JIT (small, simple functions such as property getters), the original function may too small (not enough code to over-write with the jump).

Nobody in the CCL team has a Mac to test with and the detouring code should be considered experimental at this point.  Also, this is a fairly advanced topic so a much greater understanding of what the compiler, what the JIT, how the hardware and platform works is required.  This isn't a trivial topic.

While I have been a programmer for decades (C, C++, ASM, Pascal, various flavours of BASIC, some scripting languages) and understand x86 hardware and the Windows platform, C# specifics and it's compiler are relatively new to me.

All these things need to be taken into consideration when trying to do unsafe things such a code detours (aka "hot patching").

RawCode has done something I tried about a year ago although through a different means (I tried replacing the JIT's vtable table entry for the function) without success.  It's a learning process for us all and it needs to be undertaken with an open mind and respecting each others input.  Sometimes it's the seemingly simplest questions which have the most obvious answers which lead to what you're looking for.

And don't forget your rubber duck (https://en.wikipedia.org/wiki/Rubber_duck_debugging).
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 20, 2016, 04:11:53 AM
QuoteSome of these reasons include but are not limited to, tighter memory restrictions (code sections may not be writeable), the original function may be inlined by the JIT (small, simple functions such as property getters), the original function may too small (not enough code to over-write with the jump).

0) applicable only for mono shipped with unity v2c

1) It will throw CPU level exception (crash runtime) if you try to modify native function, but, you can modify MPF before doing this and this will work just fine.

JIT generated code marked RWX on allocation and W permission never revoked.

2) Inline, generics, virtual, native and interface calls are special cases that will require special handling.

3) functions that do something or return something are at least 32 bytes, function that do completely nothing - well, they do nothing, there is no reason to hook them.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: 1000101 on January 20, 2016, 05:25:45 PM
Quote3) functions that do something or return something are at least 32 bytes, function that do completely nothing - well, they do nothing, there is no reason to hook them.

I don't disagree with anything else you said except this.  I've written many functions which have compiled to less than 32 bytes.  The smallest function you can write is 1 byte (retf) although this is generally useless, it serves the point.  There is no guarantee of a "minimum size" for a function.

That being said, you would generally have "padding" for alignment to the next function, but the amount of padding depends on the machine word size targeted.  Most modern compilers use paragraph (16-byte) alignment but there, again, is no guarantee of the padding used unless (a) the compiler allows you to specify the alignment or (b) you are writing code in assembler where you can explicitly control such things.

/nitpick
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on January 21, 2016, 03:14:51 AM
i already explained all this and don't see reasons to repeat it.

if method cannot be altered due to low size or some special handling from engine, all methods that call it will be changed instead.

i already have somewhat working prototype that replaces calls to method A inside method B instead of injecting jumps.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: isistoy on February 03, 2016, 07:43:08 PM
Hey!
There was that call to some ticking method, that was detecting factory comp classes for stats calculations, but wouldn't consider inheritance...
You mean this could allow me to inject a rewritten method and have custom inheritance to work?
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on February 04, 2016, 01:51:41 AM
This code shows how to modify code of methods, there are no limits on actions you can perform.

This could allow to inject custom vtable, but it is not designed for this and wont do it without specific modifications.

Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on February 04, 2016, 03:02:34 AM
@Isistoy, yep, pretty much.

Also, the version here doesn't work on mac os x. E made some adaptations to allow a 64 bit method ( os x only runs 64bit code ), and some small tweaks. It's included in CCL 0.12.7, but you can find the updated code here;

https://github.com/RimWorldCCLTeam/CommunityCoreLibrary/blob/master/DLL_Project/StaticClasses/Detours.cs
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Wivex on February 21, 2016, 05:36:19 AM
How can i detour the override\virtual method?
Title: Re: Mono x86-32 JIT machine code hooks
Post by: isistoy on February 21, 2016, 09:01:35 AM
+1
Will we ever get access to the context of the object containing the method that is detoured?
Not really hopeful here...

Also, can it be made so that the method to detour is called with ours? (+= new delegate() sthg?)
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
Not really in the current way of doing things. The problem is the first few bytes of the 'old' method are overwritten with a jump to the new method, pretty much destroying what was there.

In theory I'm sure it's possible to copy the old method to a new one, see for example here;
http://stackoverflow.com/questions/24350769/copy-method-from-another-appdomains-assembly-and-execute-it-from-currentdomain
But thats going squarely into the domain of self-altering code, which is horrible to debug in the best of circumstances, and virtually impossible with the limited options we have in RimWorld.

So I'm afraid it's still the good old copy/paste from core. The good news is that you only have to copy paste the one method - not everything that calls it as well.

That said, a truly dynamic way to basically turn any method whatsoever into an event that can be hooked into would be awesome, so if anyone feels like spending a couple of months figuring this out, please know I'd send you cookies.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on February 21, 2016, 11:27:45 AM
Quote from: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
That said, a truly dynamic way to basically turn any method whatsoever into an event that can be hooked into would be awesome, so if anyone feels like spending a couple of months figuring this out, please know I'd send you cookies.
I did this months ago and suggested it into CCL. I got totally shot down.

I called the C# compiler and had it compile an function overrider (the same one I got you guys a while ago) that detoured to an event with the correct signature, then made a wrapper for it so you can pass delegates and get (un)subscribed to the event.

Currently thier is no undo function, but a possibility is to save the first few bites in the function and replace them with what they used to be when your done.

As for copy pasting functions, also possible but don't expect me to do it, I've moved on to new projects.

LINK:
https://github.com/TheGentzy/FunctionJumping
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on February 21, 2016, 12:30:15 PM
IIRC your approach required making a copy of the detouring class for every method/class in the vanilla code, which isn't exactly what I'd call truly dynamic.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on February 21, 2016, 02:00:55 PM
Quote from: Fluffy (l2032) on February 21, 2016, 12:30:15 PM
IIRC your approach required making a copy of the detouring class for every method/class in the vanilla code, which isn't exactly what I'd call truly dynamic.

Which is why I compile one! Every time you call the function I compile one just for you!
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on February 21, 2016, 02:08:19 PM
well, if that actually works, and you've still got it lying around, I'd love to give it a try.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on February 21, 2016, 02:17:34 PM
I gave ya the link! Lazy people!  ;D

Anyway here is the C# code:
https://github.com/TheGentzy/FunctionJumping/blob/master/Function%20Jumping/Program.cs

main() shows how to use it, its pretty crude I must say.

EDIT:
The actual functions are here:
https://github.com/TheGentzy/FunctionJumping/blob/master/FunctionJumper/ReturnCodes.cs
https://github.com/TheGentzy/FunctionJumping/blob/master/FunctionJumper/EventMaster.cs
https://github.com/TheGentzy/FunctionJumping/blob/master/FunctionJumper/EventInterface.cs

EDIT2:
For the lazy who can't bother to use compilers:
Results from overriding 4 different functions (NOTE: 0's are not the function outputs)

A API
B API
C OVERRIDE
81
D API
-29
HIT ENTER!!!



0
0
0
0

0
0
0
0

A NORM
B NORM
C NORM
9
D NORM
-44
HIT ENTER!!!

0
0
0
0

A NORM
A NORM
B NORM
B NORM
C NORM
C NORM
9
D NORM
D NORM
-44
HIT ENTER!!!

0
0
0
0

A NORM
B NORM
C NORM
9
D NORM
-44
HIT ENTER!!!
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Fluffy (l2032) on February 21, 2016, 05:47:05 PM
make it work without requiring users to install a C# assembler, and for instance methods, then write a simple proof of concept mod that implements it, and I will send you those cookies.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: TheGentlmen on February 21, 2016, 09:28:42 PM
Quote from: Fluffy (l2032) on February 21, 2016, 05:47:05 PM
make it work without requiring users to install a C# assembler,

Mabey the C# assembler comes with the runtimes! Have you checked?

Quote from: Fluffy (l2032) on February 21, 2016, 05:47:05 PM
and for instance methods,

Can't be bothered. That requires passing instances and stuff.

Quote from: Fluffy (l2032) on February 21, 2016, 05:47:05 PM
then write a simple proof of concept mod that implements it
working on it.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: isistoy on February 22, 2016, 03:42:56 AM
Quote from: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
Not really in the current way of doing things. The problem is the first few bytes of the 'old' method are overwritten with a jump to the new method, pretty much destroying what was there.
Virtual base calls and object context (class instance members) are show stoppers for me, right now..
Doesn't mean it's not useful, but it seems obviously limited to some uses.

Quote from: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
In theory I'm sure it's possible to copy the old method to a new one, see for example here;
http://stackoverflow.com/questions/24350769/copy-method-from-another-appdomains-assembly-and-execute-it-from-currentdomain
But thats going squarely into the domain of self-altering code, which is horrible to debug in the best of circumstances, and virtually impossible with the limited options we have in RimWorld.
Agreed. Plus, honestly, I am in no way close to being able to decypher use of unsafe blocks using pointers manip. and asm instructions. Meaning my brain can't deal with any weird pb that could happen combining it with .net marshaling or basically anything else :D

Quote from: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
So I'm afraid it's still the good old copy/paste from core. The good news is that you only have to copy paste the one method - not everything that calls it as well.
As always, if your use for sthg like this doesn't fit, reconsider why you would use it in the first place. I should do that more often ;)

Quote from: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
That said, a truly dynamic way to basically turn any method whatsoever into an event that can be hooked into would be awesome, so if anyone feels like spending a couple of months figuring this out, please know I'd send you cookies.
If that means propagate all class instance members (through a context object of some sort) and have pre/post triggers to plug in to (jump), I'd say how high!
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on February 22, 2016, 04:16:35 AM
there is no stoppers since mono source code available and you always can go native if something really bad happens.

proper way of doing "hooks" is patching call instructions only, without doing damage to methods.

if method A calls method B, instead of damaging method B with jump opcode, we should patch method A.

this also will allow to call method B from methods C and D without any interference.

i will provide some time later methods to resolve jit stubs into actual methods and methods to parse raw assembly into instructions for easy and safe processing, basically, this is only things that required to implement efficient and safe hooks.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: isistoy on February 22, 2016, 04:52:59 AM
Quote from: RawCode on February 22, 2016, 04:16:35 AM
there is no stoppers since mono source code available and you always can go native if something really bad happens.
They are for me, with my current set of abilities. I should have made that clear, sry.
Can't wrap my head around the idea jumping into the context of another class instance method and potentially missing access to that particular instance private object members or class hierarchy.

EDIT: now, we are talking! Context is there, apparently unchanged and perfectly responding.
Stack call to base.Method() and other members defined/cloned from original class look available.
Had to also dummy implement some abstracts to pass compilation, but so far so good!

Quote
proper way of doing "hooks" is patching call instructions only, without doing damage to methods.

if method A calls method B, instead of damaging method B with jump opcode, we should patch method A.

this also will allow to call method B from methods C and D without any interference.

i will provide some time later methods to resolve jit stubs into actual methods and methods to parse raw assembly into instructions for easy and safe processing, basically, this is only things that required to implement efficient and safe hooks.
Being able to hook a particular call with a block of your own looks promising. :)
Do you mean replacing the initial or also, maybe just complementing it?
Title: Re: Mono x86-32 JIT machine code hooks
Post by: ppumkin on September 15, 2016, 05:33:34 PM
Hi Guys
I have just started to do mods first time ever since I saw you can do it with C# pretty easily, for most things. Obviously I ran into a problem where I want to change one little static method in Rimworld just to do something slightly different. I had no knowledge of CCL or this thread before.

I started searching for answers on StackOverflow obviously, and found this one http://stackoverflow.com/a/22014532/706363 talking about CLR injection, basically injection IL code. It was a bit over my head and I tried it but nothing happened. I think I was not doing it correctly but I eventually landed up on this page to see it looks very similar to what that post mentioned, both from rawcode and gentlemen.

Thanks for good work guys.

This is gonna be very noobish though. I admit I don't fully grasp Unity and the full spectrum of the Rimworld dll but I understand C# well. I am just wondering, where do I put this code, either version of it? I understand it should be loaded before the game fires up, I think?

Not had time to try any of these solution here, I am a bit tired after searching and searching but can I use this jumping/swapping/detouring code like when I first create a think, on like PostMake() or do I need to create some class for earlier binding somewhere. Sorry if that sounds a bit stupid, I am a bit confused how to use this.


Title: Re: Mono x86-32 JIT machine code hooks
Post by: ppumkin on September 16, 2016, 05:56:44 AM
Ahhh. It was a stupid question. Never mind. I was so tired I wasn't reading my code properly any more. It was a typo in the methodName I wanted. [Copy and Paste + Tiredness syndrome]

It works now. I did it using
[StaticConstructorOnStartup]

And inject my method there. Tested in game and works. Bloody amazing rawcode.

Thanks allot for this, seriously rawcode,
Thank you for sharing!

(sharing is caring)
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Rock5 on November 07, 2016, 07:11:53 AM
Quote from: Fluffy (l2032) on February 21, 2016, 10:47:48 AM
Not really in the current way of doing things. The problem is the first few bytes of the 'old' method are overwritten with a jump to the new method, pretty much destroying what was there.

In theory I'm sure it's possible to copy the old method to a new one, see for example here;
http://stackoverflow.com/questions/24350769/copy-method-from-another-appdomains-assembly-and-execute-it-from-currentdomain (http://stackoverflow.com/questions/24350769/copy-method-from-another-appdomains-assembly-and-execute-it-from-currentdomain)
But thats going squarely into the domain of self-altering code, which is horrible to debug in the best of circumstances, and virtually impossible with the limited options we have in RimWorld.

So I'm afraid it's still the good old copy/paste from core. The good news is that you only have to copy paste the one method - not everything that calls it as well.

That said, a truly dynamic way to basically turn any method whatsoever into an event that can be hooked into would be awesome, so if anyone feels like spending a couple of months figuring this out, please know I'd send you cookies.
So do I understand correctly that the beginning of the original function gets overwritten with an instruction to jump to the new function thereby making the original function unusable?

Does that mean that if 2 mods try to jump the same method then the second to run will overwrite the changes of the first?

Couldn't you copy the bit at the beginning before overwriting it then use that to jump back and run the original function?

If this is nonsense please forgive my ignorance.

Hm... I feel like I might have read something like that somewhere. My memory is not the best.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on November 07, 2016, 10:49:52 AM
https://ludeon.com/forums/index.php?topic=21015.msg230283#msg230283
Title: Re: Mono x86-32 JIT machine code hooks
Post by: Rock5 on November 07, 2016, 11:38:44 AM
Thanks. So I take it the copied data can't be used to somehow call the original function? That's what I understood from the topic.

I could nearly envision it working. Say you had Afunction(args) and you copy the beginning bit to another provided function, lets call it _CallOriginalAfunction(args). Then follow it with a jump back into the original Afunction() to continue after the beginning bit. Then you do what you do now and overwrite the beginning bit with a jump to the new function called say _Afunction(args). Then in _Afunction(args) you could call the original function if needed. If this could work then multiple modders could modify the same function as long as they call the original function.

Oh well. I suspect it doesn't work because you can't expect code that is split arbitrarily, and joined with a jump instruction, to work.
Title: Re: Mono x86-32 JIT machine code hooks
Post by: RawCode on November 08, 2016, 04:29:43 AM
you can copy original function and call it if needed...

please read book about C and ASM for more info, trying to explain how x86 works is waste of time.