Ludeon Forums

Ludeon Forums

  • January 17, 2020, 08:12:23 AM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: [B18] (WIP) ModSync  (Read 1157 times)

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
[B18] (WIP) ModSync
« on: May 29, 2018, 12:14:32 AM »

The mod is now posted in releases: https://ludeon.com/forums/index.php?topic=41634.msg410175#msg410175


It's time to make a decentralized mod synchronization tool

That's the goal of ModSyncRW. It's the spiritual successor of ModSync.ninja

Goal:
  • Create a decentralized way to manage mods
  • Support multiple different repositories / download sites
  • Open Source - this will be hosted on Github for people to add support to other repo/file hosts... Repo: https://github.com/KiameV/rimworld-modsync-rw

My first goal is to have support for Github for hosted mods.

In game a mod author will, from their mod's page where they'd upload the mod to steam, have a button (just like modsync.ninja) Create ModSyncRW file (only shown if one has not already been created.

The window would then have:
  • Where the mod is hosted (initially just Github)
  • The owner's name
  • The project's name
  • Where to download new updates from (Clone or Releases)
  • Branch (initially filled in as master)

Note: Everything under host selection would be dynamic for the host selected

A file would then be created in the mod's About folder:
ModSyncRW.xml
Code: [Select]
<?xml version="1.0" encoding="utf-8"?>
<ModSyncRW>
<host name="github">
<owner>KiameV</owner>
<project>rimworld-changedresser</project>'
<last-update></last-update>
<download-from>Releases</download-from>
<branch>master</branch>
</host>
</ModSyncRW>
(i wish this game used json, XML is sooo 90's ;D )

The only thing remaining is figuring out how to keep track of the last time the end user has updated the mod. I don't think the local file system is the best route. My current thought is it would be the end-users job to select the mod/s they've update and select [Is Up To Date] at which point their version of ModSyncRW.xml would get a new field entered last update time. That can then be used to compare to the host's last update time.

The end-user would have a screen very similar to ModSync.ninja's where they can see all their non-steam mods and which ones possibly need to be updated.


As for adding support for new hosts, my hope is to have a simple Interface with just a few methods to implement with hopes that as other devs want to host their stuff elsewhere it can be supported.


Please feel free to comment on this idea and share any ideas you may have.

Thanks!
« Last Edit: June 01, 2018, 12:27:03 AM by Kiame »
Logged

Nightinggale

  • Colonist
  • ***
  • Posts: 378
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #1 on: May 29, 2018, 01:23:05 AM »

I don't get how it's supposed to figure out if the mod is outdated or not. Sure it can detect the newest version on GitHub, but how will it figure out which version is on the HD? If we add a version tag then it will clash with mod owners not having to do anything after initially adding the mod.

Also about version tag. ModCheck can display errors if a mod requires at least a certain version of another mod. To do this, it reads the version tag from ModSync and it assumes it to be readable by Version, meaning it's x.x or x.x.x or x.x.x.x where x is an int. It would be nice if ModSync enforces this version system as well, though if the version number is out of reach for ModCheck, then it doesn't matter anyway.
Logged
ModCheck - boost your patch loading times and include patchmods in your main mod.

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #2 on: May 29, 2018, 02:24:47 AM »

The version on the HD is set by the user after they've d/l the mod. It's a physical action on the end-users part.
When the game first loads, the mod could also go through all mods and for any mod that doesn't have the date field set it sets it to Now. Yes this could cause confusion at times if the end-user downloaded an older version or some such. By and large I don't think this would happen (or i am just naive)

The one problem could arise is if the mod author distributes the file with a date already set. Though as long as that date gets set by the author it's averted.

I can retract the idea of 'Modder doesnt have to update version' though really 'last update date' IMO should suffice for the vast majority of (if not all) cases.
« Last Edit: May 29, 2018, 02:41:30 AM by Kiame »
Logged

Canute

  • Transcendent
  • *****
  • Posts: 8000
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #3 on: May 29, 2018, 03:40:16 AM »

Quote
The version on the HD is set by the user after they've d/l the mod. It's a physical action on the end-users part.
Don't await to much from the end-user side part.
Allways think that any end-user is a DAU primary.
Logged

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #4 on: May 29, 2018, 03:47:24 AM »

Well i think i'm convinced to stay the status quo with using versions and not date, not due to users, but because it'll allow other mods to continue working and it will feel like a more seamless transition.

Originally i was thinking i'd create a new .xml file in About but now thinking the dialog will hijack the existing ModSync.xml file and reformat it to the new standard.
« Last Edit: May 29, 2018, 03:49:55 AM by Kiame »
Logged

Nightinggale

  • Colonist
  • ***
  • Posts: 378
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #5 on: May 29, 2018, 09:00:14 PM »

Originally i was thinking i'd create a new .xml file in About but now thinking the dialog will hijack the existing ModSync.xml file and reformat it to the new standard.
How about just adding the needed tags to the existing file? Right now ModCheck reads <ModSyncNinjaData><Version> in About/ModSync.xml. If it's moved, then ModCheck would need to be updated to look at the new location, but it should likely look at the old one as well to keep support of mods not having updated yet.

Releasing a new version of ModCheck is a bit problematic because it requires mod owners to update their mods with a new included DLL. In response to this problem I plan to add everything from the TODO list in the next release. If ModSync requires a new approach to version reading, then I will need to include that as well. The problem is that next release is fast approaching and it could be a matter of days and after that I have no idea if/when the next version will be released.

In other words it would be a good idea to decide on a way to store and read the version tag now and then stick to it.
Logged
ModCheck - boost your patch loading times and include patchmods in your main mod.

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #6 on: May 29, 2018, 10:11:40 PM »

Yeah once i realized that i'm keeping everything in place. Do you know if MoCheck uses anything else other than version?

I was thinking the file would turn into:
Code: [Select]
<?xml version="1.0" encoding="utf-8"?>
<ModSyncNinjaData>
<ID>477e8a45-5513-4d39-9d56-f9125b736f8f</ID>
<ModName>Change Dresser</ModName>
<Version>0.18.4.0</Version>
<SaveBreaking>false</SaveBreaking>
<Host name="github">
<Owner>KiameV</Owner>
<Project>rimworld-changedresser</Project>
<DownloadFrom>Releases</DownloadFrom>
<Branch>master</Branch>
</Host>
</ModSyncNinjaData>
« Last Edit: May 29, 2018, 10:14:58 PM by Kiame »
Logged

Nightinggale

  • Colonist
  • ***
  • Posts: 378
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #7 on: May 29, 2018, 10:31:15 PM »

Do you know if ModCheck uses anything else other than version?
It doesn't and I'm 100% sure because I wrote the whole thing.... well apart from the ModSync version reading code, which is copied with permission  ;)

I was thinking the file would turn into:
Looks good. I don't have to change anything. It will just work even with the current version. That is if the file is still named ModSync.xml, though it would be silly to change that.
Logged
ModCheck - boost your patch loading times and include patchmods in your main mod.

historic_os

  • Colonist
  • ***
  • Posts: 105
  • Rollin'
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #8 on: May 29, 2018, 11:46:41 PM »

hope this one works! and please don't forget us none github users.

as per version synchronization, there are multiple approaches for that. the local file change will not work (as you already mentioned) since opening the file will cause modification and Linux changes modify date on extraction (if I remember correctly). plus timezones, etc..

from what I understand, your approach is to move the synchronization queries to the client side (modsync was server obviously).
with only github in question, it would be easy using github API.
use case:
1. user acquired mod
2. user opened the game (your best hook) - locally save all modsync mods installed(only ones you see for the first time!), attempt to query each to github to check repo commit #.
3. user opens modsync window - query each modsync mod, compare locally stored commit # to one in github using API.


I'd encourage you to force modders to add a mod version to About file or your sync file. it's time we have a proper version variable to all mods.

as per permissions, I didn't read the whole thread but came across permissions to use modsync code - obviously, as part of modsync ninja, Kiame has permissions to use any / all of the project code.

good luck with this one! and talk to me if you need anything

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #9 on: May 30, 2018, 12:33:03 AM »

Thank historic_os. I was going to PM you to verify permissions - really i'm exploring this idea and i think i'm close to having it running for github. As for what pieces I'm re-using, it's mainly just the dialog DrawWindow code to keep the same look/feel.
I will attribute those to you for the originator of the idea and the dialog windows.

Initially it will be just github with hooks for anyone to add anything in the future. Currently the things that will need to be added to support new hosts:
1. A new class which implements and interface with:
"DownloadPageUri" - page where the mod can be downloaded from
"ModSyncXmlUri" - page where the ModSync Xml is located in a raw format

2. Add a branch within a factory class to create the new Host

3. Add a branch within the Create/Update Mod Sync File Dialog to support whatever fields need to be entered.
« Last Edit: May 30, 2018, 12:39:15 AM by Kiame »
Logged

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #10 on: May 30, 2018, 02:13:33 AM »

Code is written and compiling. Lot of in-game errors though and nothing works  ;D

Time for some sleep!
Logged

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #11 on: May 31, 2018, 01:54:40 AM »

Quick status update

The window where mods can be checked is complete. The look/feel should be very much like ModSync.ninja.

In this case the Update button will open a web browser here: https://github.com/KiameV/rimworld-changedresser/releases/

When creating/updating the ModSync file - for github at least - it can be specified whether to go to the main page where the repo can be cloned/downloaded or the releases page. The former basically doesn't have /releases appended to the url
« Last Edit: May 31, 2018, 01:57:37 AM by Kiame »
Logged

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #12 on: May 31, 2018, 02:25:28 AM »

ModSync file creator is available from the Mods window. Selecting a mod in Developer mode will make the button appear to either Create or Edit a ModSync file.

Git Repo is up: https://github.com/KiameV/rimworld-modsync-rw
I hope to have the initial release out this weekend.
« Last Edit: May 31, 2018, 02:43:27 AM by Kiame »
Logged

Kiame

  • Planetologist
  • ****
  • Posts: 1263
  • Refugee
    • View Profile
Re: [B18] (WIP) ModSync
« Reply #13 on: June 01, 2018, 12:27:35 AM »

Logged