[Mod Request] C-RAN (CKAN for Rimworld) {A Mod Management System}

[Vas]

https://github.com/KSP-CKAN/CKAN
The system source code already exists right there.  It just needs some tweaking to work with Rimworld, with a portability system so that it doesn't need to lock it's self to a specific directory.  Its already quite the success with KSP even and I know for a fact it works great and is easy to use.

It allows the creation of mod packs too, where you check box all the mods you want to install, then create a "pack" file which is a file that contains the list of things to download and you give that file to someone else, they load it in their CKAN and it downloads all the same mods from the official source.

[hawkinator]

As a veteran Kerbaler, I, for one, wholeheartedly support this idea. The CKAN system is one of the best pieces of modding support software I have ever had the pleasure of using, and I would absolutely LOVE to see it work it's way out of the KSP zone and into other games. My favorite part, I think, is the way modpacks are done. Rather than shoving all the mods into a single big download (which the KSP modders are not very fond of), it allows someone to create a "modpack" that automatically downloads all the mods from their original authors.

Basically, the way CKAN works, for those of you that don't know, is that a bunch of metadata files are uploaded onto a central location. These metadata files contain:

• The name of the mod.
• A description of the mod.
• What version of the game the mod is designed to run on.
• Where to download the mod (the direct download link, can be pretty much anywhere on the interwebs that will allow direct downloads)
• Some information about what folders the modfiles need to go in
• And, possibly the most importantly, a list of other mods that the mod either (a.) Depends on {these mods will always be installed whenever the user selects to install this mod} (b.) Recommends {these mods will be installed unless the user chooses otherwise} (c.) Suggests {the user will be asked if he/she would like to install these mods, but they won't be installed by default} or, (d.) Conflicts {if one of these mods is on the system, this mod won't be installed}

The CKAN client fetches these metadata file from the central dump, and then allows the user to select which ones they want, then it automatically downloads and installs them all.

In order to create a modpack, all one needs to do is create a mod that depends, recommends, or suggests a bunch of mods.

It should, of course, be noted that there are differences in the way KSP loads mods versus the way Rimworld does it, for example KSP doesn't need to load mods in a specific order, ModuleManager exists for KSP to allow patches to config files without actually replacing them whole sale (I suspect Rimworld has some of this functionality built in, but ModuleManager is extremely flexible, and CKAN was built around it to a certain extent), and KSP mods are typically built for absurd levels of cross-mod compatibility already (there are very few confilcts overall)

[Toggle]

Thank you Hawkinator, that explained what it does extremely well for someone who had no idea what it was.

[mrofa]

you mean ?
https://ludeon.com/forums/index.php?topic=13198.0
and
https://ludeon.com/forums/index.php?topic=8524.0

[hawkinator]

you mean ?
https://ludeon.com/forums/index.php?topic=13198.0
and
https://ludeon.com/forums/index.php?topic=8524.0

Those both look like very promising systems. Are they cross platform compatible? One of the advantages of the above mentioned CKAN is that it supports Windows,  Mac, and Linux using only one executable by targeting the .NET framework for Windows / Mono framework for Linux and Mac. Personally, I am of the opinion that a mod manager ought to work on all the platforms that the game it is designed for runs on.

Also, here is the link to the CKAN topic on the KSP forum, if you would like some more information: http://forum.kerbalspaceprogram.com/index.php?/topic/90246-the-comprehensive-kerbal-archive-network-ckan-package-manager-v1160-13-dec-2015/

[isistoy]

I personally don't now about ksp nor ckan, thus why I wanted to push some more info for those in my case.
The idea has apparently been discussed or tried out in the past here, thanks to mrofa to point that out.
Looked popular, no joke... Who, as a gamer wouldn't want sthg like this for your beloved Rimworld changing experiences?

Missing knowledge of ksp modding capabilities, game's structure and community, I personally can't compare these 2 games.

Is it that we only miss the good will and a central mod repository?
Or are there intricacies we face in RW, that don't exist in KSP?
If so, what effort could be put into managing things differently compared to today?

From that kerbal ckan thread (interesting thoughts, thanks hawkinator) :

Taking lessons from decades of package installation and management experience in the Debian and Perl projects, each of which successfully manage tens of thousands of distinct packages, I am very glad to present to you...

The CKAN is designed to be friendly to mod authors. There's no requirement for authors to include metadata in their files, or change their release process in any way. Where possible, new releases are indexed automatically from systems which provide us with release APIs

The CKAN has been an amazing team effort, with dozens of contributors providing code, metadata, advice, and support. However, there are a few I would like to personally thank:
...

Thread is 102 pages, started in Nov 2014.

[TheGentlmen]

I will only hint at the the fact I've been attempting this for the last week. Currently I can call git.exe and download ANY git respiratory to ANYWHERE!

I will provide no more info. .

PS: Current ETA is 1-2 months.

[hawkinator]

Missing knowledge of ksp modding capabilities, game's structure and community, I personally can't compare these 2 games.

Is it that we only miss the good will and a central mod repository?
Or are there intricacies we face in RW, that don't exist in KSP?
If so, what effort could be put into managing things differently compared to today?

I shall indulge in some backstory, if I may: Much like Rimworld draws inspiration from Dwarf Fortress, Kerbal Space Program draws its inspiration from a game/simulator called Orbiter. Orbiter provided a remarkably nice simulation of orbital mechanics, and a robust game engine, but tended to be lacking in content in the core game (to the point that by default it didn't even have sound). It did, however, have a vast array of mods available for it, mods that added everything from new ships and planets to sound. It was said of Orbiter, by some, that it was less of a game in itself than a really great platform for mods. Inspired by Orbiter, KSP lead developer Felipe Falanghe (AKA HarvesteR) decided to pitch this crazy idea for a video game to the company he worked for, Squad (then an advertising agency), for a video game about building and flying rockets in space, with cartoony-looking little green men for pilots. And the rest is, as they say, history. /backstory

Nowadays, while containing plenty of content on its own, KSP still shows its Orbiter heritage by allowing nearly any piece of the game to be modded, from rocket parts to planets and even core systems such as the aerodynamics simulation. As for comparing it to Rimworld, it's a little bit like comparing apples to tomatoes, they do very different things in an occasionally similar way. Both games are built using the Unity game engine, and both games place modfiles in a central folder alongside core gamefiles. In RW, this folder is called Mods, and the developer-produced content is in a folder called Core; In KSP the equivalent of the Mods folder is GameData, and the Core folder is Squad.

Really, the only thing I can see being different about a modmanager for RW versus KSP is the need to write a ModsConfig.xml file (and it would be possible to do without that feature, albeit with more work for the user)

I will only hint at the the fact I've been attempting this for the last week. Currently I can call git.exe and download ANY git respiratory to ANYWHERE!

I will provide no more info. .

PS: Current ETA is 1-2 months.

Nice to see a fellow Kerbaler, and looking forward to the results

[isistoy]

Thanks, helpful to get a bit of history.

Two fronts then

• 
  ckan for RW website referential repositories for mods:
  you would request URIs and content from them, in order to download mods or check latest versions.
  I remember reading some discussions around it with ccl modders, 6 months ago, on slack. People usually have an opinion on this, be it good or bad, but certainly modders are aware of the involvement it would represent. Kinda tough call to ask anyone to create a "willing" referential website for mod deposit and push others to "discipline" (even gently) into it. Possibly, any official announcement made towards sthg can also turn into back stab (quick news on that below), but won't hopefully dissuade Ludeon or anyone else from participating to that discussion.
  


• 
  adapting the ckan tool for rw:
  there are several executables, namely a GUI, a command line tool and some other Net stuff. Mostly, I would say it's having the right sets of rules programmed in there to check for conflicts and define packaging, that would be of a big focus. It's a versioning database's job, using relationships and sometimes, constraints. rw's own set of rules and logic. There is possibly also some differences in the mechanics that expose rw objects to modding, compared to ksp.
  

News - talks about some recent ksp mod repository situation: http://www.shacknews.com/article/93320/3rd-party-kerbal-space-program-mod-site-kerbalstuff-shuts-down-squad-issues-response.

[hawkinator]

News - talks about some recent ksp mod repository situation: http://www.shacknews.com/article/93320/3rd-party-kerbal-space-program-mod-site-kerbalstuff-shuts-down-squad-issues-response.

Some hopefully useful information about this topic: KerbalStuff was a website that was hosted by a fan of the game / member of the community (SirCmpwn), which hosted modfiles for KSP. While KerbalStuff was well liked, and had more mods hosted there than any other platform, CKAN was never directly tied to it, the only connection between the two was that mod authors uploading their mods to KerbalStuff had the option of having their mods automatically added to the CKAN (as opposed to doing this themselves, or letting someone else do it for them). As I mentioned earlier, CKAN has the ability to download the files from any site that allows machine downloads (Dropbox and Github were and still are popular alternatives to KerbalStuff {and its replacement, SpaceDock}, and the official modsite on Curse could be used as well).

What this means for adapting CKAN to RW is that mod authors would be able to have their files hosted wherever they like, and if an official or community driven mod hosting solution arises, then that would be supported too.

(the CKAN metadata repository is hosted on Github, IIRC)

[Vas]

I figured, instead of writing your own mod manager from scratch, you can start with the source code of one that already exists and is very well done.  That's why I wanted to link it and get some of the mod manager makers here to take a look into something that already works great.

If you were to include the ability to write the ModsConfig.xml file, you could basically let us rearrange mods in a specific order and have any mod that marks one as a dependency always load below that mod.  Some other such things like that and all.

If you wanted to get real fancy, you could even add special things that allow authors to mark certain of their mods as a core which will load at the top just under "Core" (core game).  But anyway, just some other ideas I wanted to throw in.

Hawkinator, KSP shut down that site because they are douches.  :/  They wanted people to use crappy curse for hosting mods because I believe Curse gets revenue for everyone who uses it and may have complained or something.  So Squad trademarked the name "Kerbal" and forbids anyone from using it in their domain.  Would be like if Tynan trademarked "Rimworld" and prevented anyone from making RimworldInformation.com or RimworldModding.com.  Luckily, KerbalStuff is back up under a new site and new ownership.  http://spacedock.info
And actually, CKAN was very tied to it.  Many of my mods won't update because they are still linking to KerbalStuff and everyone is in the process of moving their things over to SpaceDock.

[isistoy]

What this means for adapting CKAN to RW is that mod authors would be able to have their files hosted wherever they like, and if an official or community driven mod hosting solution arises, then that would be supported too.
(the CKAN metadata repository is hosted on Github, IIRC)

Ok, yes. That thing looks well managed and indeed has that supporting feature you describe https://github.com/KSP-CKAN/CKAN-meta
Really interesting.

I figured, instead of writing your own mod manager from scratch, you can start with the source code of one that already exists and is very well done.  That's why I wanted to link it and get some of the mod manager makers here to take a look into something that already works great.

That is sometimes the best starting point, even if only to get an example of feature implementation.

If you were to include the ability to write the ModsConfig.xml file, you could basically let us rearrange mods in a specific order and have any mod that marks one as a dependency always load below that mod.  Some other such things like that and all.
If you wanted to get real fancy, you could even add special things that allow authors to mark certain of their mods as a core which will load at the top just under "Core" (core game).  But anyway, just some other ideas I wanted to throw in.

My guess is that it would already be a change note to deal with, when considering both game's mod mgt behavior. There is still a mod ordering process to deal with, in rw.

By the way, below is an example of the format hawkinator told us about. It's used for mods declaration and ckan puts on its metadata store, explaining why the netkan project:

Code Select Expand

{
    "spec_version": "vx.x",
    "identifier": "modId",
    "name": "Mod Name",
    "abstract": ?,
    "author": "x",
    "license": "WTFPL",
    "resources": {
        "homepage": "uri",
        "kerbalstuff": "uri",
        "x_screenshot": "uri"
    },
    "version": "xx.xx",
    "ksp_version": "x.x.x",
    "depends": [
        {
            "name": "x"
        },
    ],
    "recommends": [
        {
            "name": "x"
        },       
    ],
    "install": [
        {
            "find": "modId",
            "install_to": "GameData/MyModFolder",
            "filter": [
                "Thumbs.db"
            ]
        }
    ],
    "download": "zip archive url",
    "download_size": 6101666,
    "x_generated_by": "netkan"
}

Question: possibly, only .zip files supported?

...

I don't want to comment on who did what, but rather say it's a possibility that sort of things happens in the community. If it could be avoided through open/honest discussions here, then all for the best...

[hawkinator]

By the way, below is an example of the format hawkinator told us about. It's used for mods declaration and ckan puts on its metadata store, explaining why the netkan project:

Code Select Expand

{
    "spec_version": "vx.x",
    "identifier": "modId",
    "name": "Mod Name",
    "abstract": ?,
    "author": "x",
    "license": "WTFPL",
    "resources": {
        "homepage": "uri",
        "kerbalstuff": "uri",
        "x_screenshot": "uri"
    },
    "version": "xx.xx",
    "ksp_version": "x.x.x",
    "depends": [
        {
            "name": "x"
        },
    ],
    "recommends": [
        {
            "name": "x"
        },       
    ],
    "install": [
        {
            "find": "modId",
            "install_to": "GameData/MyModFolder",
            "filter": [
                "Thumbs.db"
            ]
        }
    ],
    "download": "zip archive url",
    "download_size": 6101666,
    "x_generated_by": "netkan"
}

Question: possibly, only .zip files supported?

Here is another example of a metadata file, showing how it works with an actual mod:

Code Select Expand

{
    "spec_version"   : 1,
    "name"           : "Advanced Jet Engine (AJE)",
    "abstract"       : "Realistic jet engines for KSP",
    "identifier"     : "AJE",
    "download"       : "https://github.com/camlost2/AJE/archive/1.6.zip",
    "license"        : "LGPL-2.1",
    "version"        : "1.6",
    "release_status" : "stable",
    "ksp_version"    : "0.25",
    "resources" : {
        "homepage"     : "http://forum.kerbalspaceprogram.com/threads/70008",
        "repository"   : "https://github.com/camlost2/AJE"
    },
    "install" : [
        {
            "file"       : "AJE-1.6",
            "install_to" : "GameData"
        }
    ],
    "depends" : [
        { "name" : "FerramAerospaceResearch" },
        { "name" : "ModuleManager", "min_version" : "2.3.5" }
    ],
    "recommends" : [
        { "name" : "RealFuels" },
        { "name" : "HotRockets" }
    ]
}

IIRC, .7z and .rar archives are supported as well, possibly others.

Here is the page describing what each line in the metadata file actually does: https://github.com/KSP-CKAN/CKAN/blob/master/Spec.md

[TheGentlmen]

What this means for adapting CKAN to RW is that mod authors would be able to have their files hosted wherever they like, and if an official or community driven mod hosting solution arises, then that would be supported too.

You will be capable of hosting it on any Git Respiratory other than the Metadata Respiratory.

Every modder should be already using git, it make life a 1000 times easier. I will be capable of helping people set it up when I'm done.

I figured, instead of writing your own mod manager from scratch, you can start with the source code of one that already exists and is very well done.  That's why I wanted to link it and get some of the mod manager makers here to take a look into something that already works great.

Its a project I have just for the funs. I don't really care what would be practical.

[bvierra]

The actual setup for CKAN is a really good starting point. I took the website for kerbalstuff and started to play with it (before life kicked my ass), there are a number of issues with how the website is done however.

One think I would recommend doing is allowing the mods to be hosted on something like github so that the dl bandwidth isnt on your tab.