Rayman 1 Level Editor/Viewer (Ray1Map)

I've merged your pull request now, seems to work just fine on Windows

Thanks! Good to know it works! I'll probably take a break from this now, at least until after Record Day, but after that I might come back and see if I can do anything about other emulators (BizHawk or otherwise).

EDIT (18 August): Btw I got the re-compressor working! It turns out that what I thought was an off-by-one was actually an off-by-4096 (or off-by-8192, etc.) because the C code I was copying from was written with the underlying assumption of a 4-kB buffer, whereas I was using arbitrary-sized buffers. Therefore I was silently chopping off the MSBs of some string offsets.

RayCarrot said:

Edit: I just had another look in the Designer exe and while I found where it sets the same max memory values it seems it's a lot more complicated this time around. They are no longer just constant values, instead it gets them from somewhere - not sure where yet :/.

Click to expand...

Just double-posting to say that I've finally gotten around to figuring out (with the help of the Dosbox Debugger) that it gets them from the "VERSION" section of COMMON.DAT – suddenly all the groundwork I've done to make a distributable EXE-patcher feels like a bit of a shaggy dog story. Worth noting that the size values are in kiB rather than bytes this time…

RayCarrot said:

Edit 2: Unrelated, but you might find it interesting: the RayKit exe actually has a ton of unreferenced functions leftover from when Ubisoft developed Rayman 1. There are functions to compile all sorts of files, like the .wld files etc. as well as some graphics editors or converters. Not sure how to enable any of it, but it's cool it's all still there! Also, in the list of power-ups it mentions Super-Seed, which is unused and crashes the game if used.

Click to expand...

I found one such function, which seems to be called "AnalyseFichierVersion", that can read a text-file equivalent of the "VERSION" section mentioned above, containing the memory sizes written out in kiB as numbers, and seemingly separated by spaces.

Oh, that explains a lot! Each version has an array of 32-bit integer values I never knew what they were... Setting one to 0 causes the game to say it can't allocate 0 bytes, but for some reason I never made the connection it was for this data. Nice!
Let me know if you need help patching it. I should also mention that I did some major refactoring for the editor code (in order to better support R3 GBA and for the web version), and I haven't yet re-implemented everything, so saving level changes might not fully work yet.
Oh also btw, something you might be interested in: I recently parsed the wldmap file in SPECIAL.DAT which contains data for every level in the worldmap, how maps are chained together, loading screens etc. You can see this by logging the archive files with the logging enabled. So in theory this could be used to create all new games :p Sadly in R1 this is hard-coded in the exe, but for edu and the kit games it isn't (also for Kit it has more evidence there was a fifth level for every world).

Edit: I've gone ahead and names it in ray1map now. This is how it looks for KIT now:


Then it hard-codes 0x100000 for the .mlt files.

It would be kind of fun to use this to have every .wld file contain DES and ETA from every world Of course the palettes would sometimes be wrong, but in theory it should work. Would be a pain adding them all to the .mlt files though... Maybe that could be done using some script (the comments in the .mlt files would be filtered out then though). But since the .mlt mem block is hard-coded it might not work...

RayCarrot said:

Oh also btw, something you might be interested in: I recently parsed the wldmap file in SPECIAL.DAT which contains data for every level in the worldmap, how maps are chained together, loading screens etc. You can see this by logging the archive files with the logging enabled. So in theory this could be used to create all new games :p Sadly in R1 this is hard-coded in the exe, but for edu and the kit games it isn't (also for Kit it has more evidence there was a fifth level for every world).

Click to expand...

That sounds really promising! When I first got Rayman Designer, I imagined being able to place my own levels on a world map, and was disappointed when it turned out to be much more limited than that. (Well, in actual fact, the tools are so counter-intuitive that it took me years to figure out how to make anything playable, when I found instructions online!)

RayCarrot said:

Edit: I've gone ahead and names it in ray1map now. This is how it looks for KIT now:

Code:

TailleMainMemTmp: 133120 bytes
TailleMainMemFix: 286720 bytes
TailleMainMemWorld: 829440 bytes
TailleMainMemLevel: 2560000 bytes
TailleMainMemSprite: 730112 bytes
TailleMainMemSamplesTable: 394240 bytes
TailleMainMemEdit: 2560000 bytes
TailleMainMemSaveEvent: 524288 bytes

Then it hard-codes 0x100000 for the .mlt files.

It would be kind of fun to use this to have every .wld file contain DES and ETA from every world Of course the palettes would sometimes be wrong, but in theory it should work. Would be a pain adding them all to the .mlt files though... Maybe that could be done using some script (the comments in the .mlt files would be filtered out then though). But since the .mlt mem block is hard-coded it might not work...

Click to expand...

Here's what I'm thinking: maybe the spreadsheet could include made-up DES names for events that are only in EDU and/or R1, and at runtime Ray1Map would only show up events with DESs that actually exist. Then there could be a button to import DESs from EDU and/or R1 (if the user has them installed), using the corresponding numbers in the spreadsheet to figure out what's what, and setting the limit appropriately when it's finished. Not sure how much UI programming that would take… Copying them between worlds sounds cool too – it could probably be implemented similarly, and then the DesKit and EtaKit columns could each have only one name instead of six! Then the MLT could be exported from Ray1Map based on what's in the spreadsheet? Of course, it might indeed be too big…

That would be really nice yeah. There's a lot I would ideally want to do with ray1map in terms of allowing for modifications. Something like an animation editor/creator would be really neat where you can drag around sprites and such for each frame. But yeah, then it's the problem with the UI. There also aren't as many people interested in Rayman 1 modding nowadays I feel, especially with Rayman Redemption soon getting its own editor. It's at least nice to finally understand how everything works in Rayman 1 I find these sort of things very interesting.

RayCarrot said:

Code:

TailleMainMemTmp: 133120 bytes
TailleMainMemFix: 286720 bytes
TailleMainMemWorld: 829440 bytes
TailleMainMemLevel: 2560000 bytes
TailleMainMemSprite: 730112 bytes
TailleMainMemSamplesTable: 394240 bytes
TailleMainMemEdit: 2560000 bytes
TailleMainMemSaveEvent: 524288 bytes

Then it hard-codes 0x100000 for the .mlt files.

Click to expand...

I just actually tried it in Ray1Map last night, and it only then hit me that the values are repeated for each language, even though they're obviously the same in all cases! Wonder what they were thinking there. Also, the devs misspelled "DEUTSCH" in the header. I also decided to investigate that byte referred to as "Unk1", and it turns out that it's a padding byte that gets overwritten at runtime with the index of the user's language choice, so that's a (very minor) mystery solved.

As for automating the transfer of ETAs/DESs from game to game, maybe it's something that could happen at startup, with a checkbox in the general options to enable/disable it, so it wouldn't necessarily require a lot of UI work. I might see if I can get something going along those lines…

Ah nice! I named the property now

The reason they're duplicated is because in the edu game they wanted each volume to be configurable separately from the rest as they had their own .wld files and such, which could be of varying sizes (although they rarely were...).

Regarding the automatic transfer, sure I'll look into getting saving support working again the next few days as I haven't reimplemented all functions yet after the refactoring we did.

Btw, something else interesting is we have hitboxes displayed now.

Sadly it's not perfect. A lot of hitboxes are missing (like the Hunter's detection zone) and some are wrong (like Bzzit always shows the collision for the fruit he carries in the chase level, even in other levels like the boss). There's nothing we can really do about these are the game hard-codes the majority of this. It's actually a complete mess the way the game handles it

On the other hand they seemed to have learned from their mistakes, cause R2 uses a unified system which means we can show collision for every object, including gendoors (shown in blue)!

Is there somewhere that I can download an installer or executable package for the level editor? I am not interested in designing levels. I would like to extract some Rayman files. Particularly, I would like to extract the sound effects from both the original Rayman game and Amazing Learning Games With Rayman. I would also like to extract the dancing white flowers from the Dream Forest (the beginning of Anguish Lagoon contains one). It would also be nice to extract the rainbow ting from Rayman Redemption; but, I do not know if Ray1Map supports that game (although, given that Rayman Redemption is a “refresh” of Rayman 1, reading its information might work already). A while back, I had seen somewhere on this forum where the sound effects can be downloaded in zip format. But, adjusting the frequency for each to match the frequency used in the game is difficult and tedious. Using Ray1Map, I might not be able to extract the sounds without needing to adjust the frequencies. If there is already somewhere that I can download the animations, then a link to that page (or file) would be much appreciated.

I sent you a link to all of the assets in a pm No release is currently available for Ray1Map as it's still in its early stages when it comes to actual editing and UI. However there is a web version coming soon (without the editing features though).

RayCarrot, I attempted to rely to your private message, but the option was not available. Upon attempting to compose a new message, I was presented the following words: We are sorry, but you are not authorised to use this feature. You may have just registered here and may need to participate more in discussions to be able to use this feature.
I am guessing that replying to users’ private messages is an option; but, it becomes available only after so many posts have been approved or so many tings have been collected (however that happens). Remember the magician’s rule of thumb: “10 tings, please!” That reminder is best heard when in combination with Rayman’s reminder to everyone that… he’s waiting! Thank you for the link to the files and the information about Rayman Redemption. I was not aware that Rayman Redemption was built using a separate engine from the original game. Given that, it might be quite a while before Ray1Map supports that game, if it ever does.

Hey Wodd, PMs will be available automatically once you've made a few posts here, it's a security measure to protect the place from spam and other nuisances. Sorry for the inconvenience, and welcome to Pirate-Community, by the way!

Thank you for the information and warm welcome.

Welcome Wodd!

RayCarrot said:

The reason they're duplicated is because in the edu game they wanted each volume to be configurable separately from the rest as they had their own .wld files and such, which could be of varying sizes (although they rarely were...).

Click to expand...

Ah, of course, that makes sense.

RayCarrot said:

Regarding the automatic transfer, sure I'll look into getting saving support working again the next few days as I haven't reimplemented all functions yet after the refactoring we did.

Click to expand...

I went ahead and put something together, which you can see here. I used a button on the Settings form, rather than having it happen automatically, and it's just as well because it's extremely slow! I think there are probably efficiencies to be gained, but I'm happy enough that I got it working. I don't think there's a way to modify the contents of archive files at the moment, so the game's memory allocation isn't actually being changed by this right now. Are you planning on adding write functions for archive files after the refactoring? (Or did I just not find what's there? )

RayCarrot said:

Btw, something else interesting is we have hitboxes displayed now.
Unity_2020-09-24_08-38-43.png
Sadly it's not perfect. A lot of hitboxes are missing (like the Hunter's detection zone) and some are wrong (like Bzzit always shows the collision for the fruit he carries in the chase level, even in other levels like the boss). There's nothing we can really do about these are the game hard-codes the majority of this. It's actually a complete mess the way the game handles it

On the other hand they seemed to have learned from their mistakes, cause R2 uses a unified system which means we can show collision for every object, including gendoors (shown in blue)!

Click to expand...

I had noticed that about the hitboxes, including the Bzzit one! Well, it's good to know that they did learn from their mistakes here!

Welcome to the forum, Wodd!

PluMGMK said:

I went ahead and put something together, which you can see here. I used a button on the Settings form, rather than having it happen automatically, and it's just as well because it's extremely slow! I think there are probably efficiencies to be gained, but I'm happy enough that I got it working. I don't think there's a way to modify the contents of archive files at the moment, so the game's memory allocation isn't actually being changed by this right now. Are you planning on adding write functions for archive files after the refactoring? (Or did I just not find what's there? )

Click to expand...

Nice!
If you want the edit the event manifest I recommend doing so on the online sheet or else I might accidentally overwrite it. If we're to add DES/ETA names for R1 I think it might be best making a new column for it. This could potentially help with PS1 editing too as I could make some mapping file where I list all pointers for each DES/ETA depending on the level.
Regarding writing the archive, I made a method called "RepackArchive" you can use You pass in the files you have modified in a dictionary and it'll update those when repacking. It'll also remove the xor encryption by setting it to 0 to avoid having to deal with that (checksum it still there though, but it gets automatically calculated).

Yeah, I just stuck those edits into the CSV as test-cases. I'll edit the spreadsheet itself when I do a more significant list.
Separate columns for R1 DES/ETA files sounds promising all right, especially if it'll help with PS1. At that point, would we still put them in the Kit column too like I've done, so that the editor knows they're available in Kit when the files are ported over? Or should the Kit editor look at both columns, so that it's clearer from looking at the spreadsheet which events are in stock RD and which can be added?

Thanks for the new method! Setting the XOR key to 0 makes sense all right – why have that complexity when it's so easy to remove it for these files!

I think an all the column separate from Kit would be the best. It becomes clearer then which ones were manually added. As you said though that will mean we'll have to check both columns, but I don't think that should be that hard to change. Perhaps we don't even need to have a separate des/eta per world now if we'll have the editor automatically check which ones are currently available in each .wld file... The problem would be the ones like the stalactites, spiky balls etc. which I think have several des, but we could just duplicate them then.

That sounds like it would bring an efficiency gain all right, and allow for copying everything across WLDs like you were saying before!

I might work on that later on, meanwhile this is what I've got: https://github.com/PluMGMK/Ray1Map/commit/3536b0fbadfd07c8f52ee1cc07c7a710fbe959ca
There are even more edits to Events.csv now, since I was trying to cram as much into each WLD as possible and verify that increasing the memory limit works, but I will go back and do it properly afterwards…

Anyway, it works! And Skops' palette is messed up in Designer – who'd have thought!

If you look at line 382 of R1_Kit_Manager.cs, you'll see I had to use Goto to get the serialization pointer to the right position. I didn't want to touch R1_PC_VersionFile.SerializeImpl in case anything else is relying on the pointer doing what it does by default. That function looks like it could be rewritten though, with all those hardcoded offsets, what do you think? I think I can do that when I'm working on adding the extra columns and such.

That's really nice that you got it to work! Does it even show his hitboxes if you enable the debug mode in Designer?

Regarding the hard-coded pointer, the reason I hard-coded them in R1_PC_VersionFile.SerializeImpl is because the game doesn't really parse the file. It just allocates it all to memory and then relies on hard-coded offsets. I suppose we could still parse it in a linear way and just skip any potential padding bytes. Another option is to use GoTo instead of DoAt to avoid it resetting the current position.