xChatMC-style mod - websocket woes

[bluevulpine]

(Apologies if this double-posts; it logged me out in the middle of writing it, then said I've already posted it, but I didn't see it appear...)

Hi all. I've spent a couple days trying to get past this brick wall, and decided it was high time to consult the experts.

This is my first foray into modding for any game; my day job is a software engineer, although I'm usually wading through 20-year-old C code that was directly ported from 30-year-old COBOL code. I don't have a great deal of C# experience and zero Unity experience, but I'm learning very painfully as I go.

My end goal is to have something like xChatMC, where a streamer's viewer-chat will appear in game so he or she doesn't have to check another screen to respond to viewers. As there's no actual chat mechanism in RimWorld, the intention was to make this a read-only affair. This is what it looks like so far:



I borrowed very heavily from the Verse Messages class in order to create the UI and message handling queue (there's still some oddities though, background fill overlaps itself if I shrink the rect to a smaller height than what's in the screenshot), and dug through an ILSpy output from EdB's colonist bar to grok how to get a UI-only mod to even load in the first place. I am now able to fire test messages onto the screen all day and have them expire gracefully. I figured getting this stuff on screen would be the harder part. I have been proven somewhat wrong.

I'm hoping to also do Twitch chat, but I wanted to start with Beam.pro. They use WebSockets for their chat connection. They do also have a WebAPI, but I was going more for event driven message pushes versus constant webAPI polling/scraping. I do have to hit the web API to get the initial channel ID data and chat endpoints based on the streamer's username, and I have that working. Beyond that I've not been able to find a WebSocket class that will load properly into RimWorld once I add it to my mod's classes. I wind up getting reflection errors, which as far as I've been able to discover means the DLL is probably built against too new a framework versus what Rimworld is utilizing. (First ran into this when trying to use JSON.NET, and wound up changing to JsonFX)

Example:

ReflectionTypeLoadException getting types in assembly RimChat: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
  at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)
  at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0
  at Verse.ModAssemblyHandler.AssemblyIsUsable (System.Reflection.Assembly asm) [0x00000] in <filename unknown>:0

Loader exceptions:
   => System.TypeLoadException: Could not load type 'RimChat.ComponentRimChat' from assembly 'RimChat, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.


As soon as I remove the WS object's field from the class, the error is gone and things load, but then I can't exactly use it. I have so far tried WebSocketSharp and Verso, but have had the above reflection errors. Other DLLs I've tried to pull in from NuGet won't even finish installing under .NET 3.5.

TL;DR: Anyone know of a WebSocket class that would work with the Unity engine driving RimWorld? Is there one built-in that I've overlooked?

[attachment deleted by admin - too old]

[RawCode]

it's lupus!

best answer i can give without any code provided.

[Fluffy (l2032)]

You might have more luck for this one over on the unity forums, I've never seen a RimWorld mod that had any kind of streaming/API action going on - so there likely is limited experience at best.

You're definitely right that any modules you end up using do have to be .NET 3.5

[bluevulpine]

it's lupus!

best answer i can give without any code provided.

It's never lupus!

Apologies... I'd been staring at it for 6 hours at the time and it skipped my mind. Oops. :/

The overall project probably isn't super well organized out, but the class that is attempting to pull in the websocket is here: https://github.com/bluevulpine/RimChat/blob/master/Beam/WSSChat.cs

Code Select Expand


using System;
using System.Collections.Generic;
using Verso;
using Verso.Protocols.Websockets;

using UnityEngine;
using Verse;
using RimWorld;
using JsonFx.Json;

namespace RimChat.Beam
{
    class WSSChat
    {
        private string[] endpoints;
        private bool connected = false;
        private bool joined = false;
        private int uniqueMessageID = 0;

        // This breaks
        WebsocketClient socketClient;
        ...


Both Verso and WebSocketSharp claim to be 3.5 compatible, but I've also seen some references saying there's a .NET 2.0 API compatibility that needs to be accounted for as well.

[RawCode]

Very stupid question:

did you dropped both API and native parts into assembly folder?

[Fluffy (l2032)]

Also, you are building to 3.5, right?

[bluevulpine]

Yep - in the RimWorld/Mods/RimChat/Assemblies folder, I have the RimChat.dll, along with JsonFx.dll, and at the moment both of Verso.dll and websocket-sharp.dll. (Also, I've pushed the versions I've tried using into the repo.)

The Project's target framework is set to 3.5, and both Verso and websocket-Sharp list version 2.0 of mscorlib in their ILSpy references which appears to be appropriate.

[RawCode]

check what you drop, json stuff called Newtonsoft.Json.dll not jsonfx

[bluevulpine]

Newtonsoft Json library gave me the same reflection problem, so I explicitly switched to jsonfx's library instead.

[RawCode]

You must drop exactly same library as you referenced from IDE.

Code below works without exceptions, modloader correctly handle xdll references, atleast as long as references are actually valid.

Code Select Expand

    [StaticConstructorOnStartup]
    public class li
    {
        static li()
        {
            Log.Warning("DEBUG TEST OUT");
            Type t = typeof(JsonConvert);
            Log.Warning(t.ToString());
            Log.Warning(JsonConvert.Undefined);

        }
    }
}

[bluevulpine]

I cannot replicate your newtonsoft result. The moment newtonsoft's library is loaded by rimworld (only Core as the other thing in the Mods directory), I get a completely different reflection error than the Websockets libraries gave me when the game loads. I literally cannot show any code for this because there is nothing other than Newtonsoft.json.dll in the Assemblies folder. See attached screenshot.

This is the log:

Code Select Expand

ReflectionTypeLoadException getting types in assembly Newtonsoft.Json: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
  at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)
  at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0
  at Verse.ModAssemblyHandler.AssemblyIsUsable (System.Reflection.Assembly asm) [0x00000] in <filename unknown>:0

Loader exceptions:
   => System.TypeLoadException: Could not load type 'Newtonsoft.Json.Utilities.EnumUtils+<>c' from assembly 'Newtonsoft.Json, Version=8.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'.

My only thought is that we are using different versions of the newtonsoft library, and that the 8.0.3 version I got from Nuget is not compatible. I'll start grabbing older versions and see which one I may be able to use. What version of the newtonsoft library are you using that Rimworld loads without error?

---------------------

EDIT: Had to back up to a version from 2011 to get Rimworld to load successfully. Nuget indicates this is Newtonsoft.Json version 3.5.8.

Going to start re-wiring in the json parsing, verify that works, then see what happens with the websocket libraries when I add one of them. Thanks for your help so far!


[attachment deleted by admin - too old]

[RawCode]

1)
https://github.com/Verso-Services/net-api

2)
https://github.com/Verso-Services/net-api/archive/master.zip

3)
net2 versions of libs

4)
mono runtime embedded into unity is mono2 for net 2.0

5)
sources are available, you can pick only methods you actually need and copypaste them into your dll.

[bluevulpine]

https://github.com/Verso-Services/net-api

Unfortunately unrelated to the verso websockets project. The above repo is meant to connect to a supply chain product returns system (see: http://verso-logistics.com/).

However, I have made progress. The websocket-sharp library definitely works with RimWorld as long as it isn't a field in a class. Using websocket-sharp, I can do the following:

Code Select Expand


namespace RimChat.Beam
{
    class WSSChat
    {
        public WSSChat()
        {
            string endpoint = "ws://echo.websocket.org";

            WebSocket websocket = new WebSocket(endpoint);

            websocket.OnMessage += HandleSocketMessage;
            websocket.OnOpen += HandleOpen;

            websocket.Connect();
            websocket.Send("Hello?");
        }
        private void HandleOpen(object sender, EventArgs e) //websocket4net
        {
            Log.Message("Websocket connection opened!");
        }
        private void HandleSocketMessage(object sender, WebSocketSharp.MessageEventArgs e)
        {
            Log.Message("Received from socket: " + e.Data);
        }


It connects, sends, and receives an echo back. WOO! That's huge progress. But the connection needs to stick around, so I tried to move it up into the class definition:

Code Select Expand

namespace RimChat.Beam
{
    class WSSChat
    {
        private WebSocket websocket;

        public WSSChat()
        {
            string endpoint = "ws://echo.websocket.org";

            websocket = new WebSocket(endpoint);

This breaks the loader with the regular "System.TypeLoadException: Could not load type 'RimChat.Beam.WSSChat'" error. There's something about making it a part of the class that makes everything go haywire.

[bluevulpine]

It's the DLL loading order.

I renamed 'websocket-sharp.dll' to 'Awebsocket-sharp.dll' to make it come alphabetically before the RimChat.dll file, and the error goes away and things work. Wheee! Also works to rename my DLL to come after the websocket-sharp one. Weird... Shouldn't the references make it load things as they're needed?

EDIT: Also, this is the most beautiful thing I've seen all day, and I went to a wedding this afternoon:

Code Select Expand

Received from socket: {"type":"event","event":"Error","data":{"type":"ENOTJSON"}}


YESSSSS. I may actually get this done in the morning. Thanks so much!

[RawCode]

yes can agree with you, vanilla should invoke asm.GetTypes() after all dlls are already loaded, not right after load.