Rayman 1 Expert - 82 New Levels

[RayCarrot]

That happened to me once as well when testing some level edits I could keep typing on my keyboard and it'd keep writing those characters on the screen. Here it also starts by saying "Stack overflow", so I'm assuming that's what causes this.

[PluMGMK]

Well, what happened there was that the game was trying to write to a file and somehow had a Stack Overflow, and Watcom's fatal error handler exited cleanly but without resetting the video mode. So you were in a DOS prompt but the characters on the screen didn't look right. I'm pretty sure there are a number of ways to end up in that situation though!

Edit: the cause in Chacanger's case must have been different because he wasn't on a loading screen so the game wouldn't have been attempting to write to a file, I guess...

[Chacanger]

I'm about 71% of the way to finishing this project now and I'm debating on releasing a preview version with a few levels.

The first release will be for the DOS version of Rayman 1 as it covers those with the original PC version and those with Rayman Gold. However I plan on doing a PS1 translation afterwards.

Anyhow I noticed whilst doing this the other day, that the demo's that play in the main menu are now completely wrong seeing that the levels have been changed (see below video). I wish I knew how to change this but have no idea where the game stores the key presses...

[Hunchman801]

I have no idea either; in fact, I never asked myself how those demos worked until now, but saving key presses certainly sounds like the smartest way of doing it.

[RayCarrot]

They're stored in the exe. Which version are you using? I can find the offsets if you'd like to edit it. Updating the inputs will probably be quite hard though, so maybe best to disable the demos? I should also note the exe needs to be decompressed in order to edit it.

[Chacanger]

They're stored in the exe. Which version are you using? I can find the offsets if you'd like to edit it. Updating the inputs will probably be quite hard though, so maybe best to disable the demos? I should also note the exe needs to be decompressed in order to edit it.

It's DOS Rayman (US) v1.21.

I know that there are 6 demo's...

• The Swamps of Forgetfullness 1 (RAY9.LEV)

• Allegro Presto 1 (RAY7.LEV)

• Mr Stone's Peaks 2 (RAY7.LEV)

• Eraser Plains 1 (RAY1.LEV)

• Eat at Joe's 2 (RAY4.LEV)

• Mr Skops' Stalactites 2 (RAY10.LEV)

No idea what method of decompression is required, also would changing those commands mean changing a load of pointers?

[RayCarrot]

I'll send you a tool to decompress the exe with in a pm. The game runs just fine with it being decompressed, but you can optionally compress it again afterwards if you prefer.
As for the demo data, this is what there is:

Code: Select all

0xB7E1C (DemoRecordMap): 07 00 07 00 01 00 04 00 09 00 0A 00
0xB7E10 (DemoRecordWolrd): 02 00 03 00 04 00 05 00 01 00 05 00
0xB7E04 (DemoRecordSize): 9A 00 C0 00 C8 00 AC 00 C8 00 78 00
0xB7828 (DemoRecord): ...

DemoRecordMap indicates the levels and DemoRecordWolrd (yeah, the devs spelt this wrong ) the world. DemoRecordSize has the sizes of each demo. All of these are 16-bit values. The actual demo data is stored in the large array at DemoRecord. This is all a bit of a mess... In the 2D Rayman 2 engine this was cleaned up a lot and the demo data is stored as actual game data, but that doesn't help much here.
The absolute easiest way to deal with this is to set the size values to 0. Then the demos will finish immediately. Alternatively you could probably edit the code which loads the demo and prevent that from running. That'd be a bit more complicated to find though... The game's menu code is quite messy.

[Chacanger]

I'll send you a tool to decompress the exe with in a pm.

Thanks that'll be very useful for a few things, I'll probably look at that after I edit the game text.

On the topic of uncompressing and decompressing the game... I guess that creates an issue around me distributing this mod, originally it would have only been the WLD, LEV and DAT files; which would have meant I wasn't giving out a physical copy of the game out to anyone. If I supply an updated version of the EXE that would probably create some piracy issues wouldn't it?

The plan was going to be that I supply an installer where I could move files over etc. But I'll probably have to include my own implementation of a re/de-compressor to add that data in so that I'm not supplying the game itself just editing the users version.

[PluMGMK]

Yep, things get a lot more complicated where editing the EXE gets involved, unfortunately

I made some tools that do that for various tweaks, written in Rust. The same framework could be used to do this, or I could try to rewrite the codec in C# for inclusion in Ray1Map, as I alluded to here…

[RayCarrot]

On the topic of uncompressing and decompressing the game... I guess that creates an issue around me distributing this mod, originally it would have only been the WLD, LEV and DAT files; which would have meant I wasn't giving out a physical copy of the game out to anyone. If I supply an updated version of the EXE that would probably create some piracy issues wouldn't it?

The plan was going to be that I supply an installer where I could move files over etc. But I'll probably have to include my own implementation of a re/de-compressor to add that data in so that I'm not supplying the game itself just editing the users version.

You're still distributing modified game files (the .wld and .dat files) which include a lot of data and assets from the original game either way, so I don't think having the exe included will make much of a difference in that regard. If you don't want to worry about piracy issues at all you'd probably have to just include the deltas for the files you've changed and patch those on top of what the user has. That can be done with existing patching tools, although most of those are made for single rom files I believe, so might not work easily for an entire game. But I wouldn't worry much about including the exe. You're not including the entire game as there are still several other files required for it to run.

If you'd like you could besides your installer also make a patch for your game in the Rayman Control Panel. That'll make it quite easy to apply and revert by users. I could then make this a built-in downloadable patch (see the Rayman 2 patches for examples) if you'd like. Note that I've only implemented file replacing in the patching system though, so it's not as advanced as being able to handle deltas as I mentioned before. But from my understanding that's how your installer was going to work anyway, so it should be the same then.

[Chacanger]

I'll send you a tool to decompress the exe with in a pm. The game runs just fine with it being decompressed, but you can optionally compress it again afterwards if you prefer.
As for the demo data, this is what there is:

Code: Select all

0xB7E1C (DemoRecordMap): 07 00 07 00 01 00 04 00 09 00 0A 00
0xB7E10 (DemoRecordWolrd): 02 00 03 00 04 00 05 00 01 00 05 00
0xB7E04 (DemoRecordSize): 9A 00 C0 00 C8 00 AC 00 C8 00 78 00
0xB7828 (DemoRecord): ...

DemoRecordMap indicates the levels and DemoRecordWolrd (yeah, the devs spelt this wrong ) the world. DemoRecordSize has the sizes of each demo. All of these are 16-bit values. The actual demo data is stored in the large array at DemoRecord. This is all a bit of a mess... In the 2D Rayman 2 engine this was cleaned up a lot and the demo data is stored as actual game data, but that doesn't help much here.
The absolute easiest way to deal with this is to set the size values to 0. Then the demos will finish immediately. Alternatively you could probably edit the code which loads the demo and prevent that from running. That'd be a bit more complicated to find though... The game's menu code is quite messy.

Thanks for sending me those tools, I started looking at the demo stuff and got really motivated to look at it.

It's actually pretty easy to understand, not sure if you'd already documented this somewhere but what I found was that the DemoRecord uses 2 bytes per command and each demo seems to have a 250 byte requirement (125 commands)...

• The first is to determine what commands are used at the same time, it's made up of the following flags ...

  Code: Select all

  Cmd	Bin		Dec	Hex
  ------------------------------------
  NONE	00000000	0	0x0
  FIST	00000001	1	0x1
  ACTION	00000010	2	0x2
  JUMP1	00000100	4	0x4
  JUMP2	00001000	8	0x8
  LEFT	00010000	16	0x10
  RIGHT	00100000	32	0x20
  UP	01000000	64	0x40
  DOWN	10000000	128	0x80

• The second byte is how long the command runs for, I don't think this is seconds, 0x0 ending immediately and 0xFF going on for a very long time.

With this knowledge I'm bulding a C# library to edit this, and for fun I'm going to try and make a converter that utilises Spy++ command output files and converts them into this format so that you can record a demo rather than manually edit it.

[RayCarrot]

Yeah, the commands are essentially RLE-compressed. The repeat is in number of frames. Since the game is hard-coded to 60 fps, it means that a value of 60 is 1 second.
The reason you have two jump flags is because the game used to have 4 actions where the fourth one was for the hand-stand powerup (triangle on a PS controller). But they removed that after the last JP PS1 demo.

Btw, the game actually has the code leftover for recording the demos. You could probably use cheat engine or something similar to toggle it on (it's a boolean for if the game should record your inputs) and then extract the bytes from the buffer. Only annoying part would be needing to allocate where the buffer should be written to as it will probably be null by default. But you could probably just point it to an existing demo-record buffer and have it overwrite that in memory. That'd be fairly easy to do.
The game has a struct like this in memory (taken from the PS1 version, but should be the same on PC):

Code: Select all

4 bytes: frame offset (0, 2, 4, 6 etc.)
4 bytes: repeat length
4 bytes: repeat frame (number of times it has repeated out of the current length)
4 bytes: frames length
pointer: buffer (this is also where it writes to, so it has to be a valid pointer from which the data can be extracted)
1 byte: 1 if it's recording, otherwise 0
1 byte: 1 if it's playing, otherwise 0
1 byte: 1 if it's finished, otherwise 0 (this was used by the devs to stop recording a demo, but it will otherwise stop once it reaches 9999 frames)
1 byte: padding

If you want to try this I could probably figure out where it is stored in memory for the PC version. For the PS1 version it's at 801e55f0. But maybe your idea of making a separate tool is better. Only trouble would probably be getting the timing to match the game's frames. Good luck!

[Chacanger]

Yeah, the commands are essentially RLE-compressed. The repeat is in number of frames. Since the game is hard-coded to 60 fps, it means that a value of 60 is 1 second.
The reason you have two jump flags is because the game used to have 4 actions where the fourth one was for the hand-stand powerup (triangle on a PS controller). But they removed that after the last JP PS1 demo.

Ah that makes sense, I wasn't sure if it was to assist in double tapping jump tap to do something like helicopter quickly. Would the handstand have been jump 1 or jump 2?

Do you know if there's a pointer that states where the demo data starts? (aka shows 0xB7828)?

Also has anyone worked out if there's a value set somewhere which states what the total number of demos can be?

[RayCarrot]

Would the handstand have been jump 1 or jump 2?

The second one. But it doesn't seem to matter for the demos - they bound both to the same key there for some reason (which would explain why both trigger the jump, cause normally the handstand function is empty and wouldn't do anything).

Do you know if there's a pointer that states where the demo data starts? (aka shows 0xB7828)?

Also has anyone worked out if there's a value set somewhere which states what the total number of demos can be?

Both of these are hard-coded In the InitDemoJeu function it starts by doing this:

Code: Select all

if (NumDemo == 6)
{
  NumDemo = 0;
  ...

Even then it wouldn't be very easy adding more since you'd need to repoint the data tables then to resize them. You could easily decrease this though by editing the assembly.
As for pointers to the data tables, same there, it's all in the function, so the assembly has to be edited there too.
This makes me sad they didn't spend more time on the PC version of the game... Later on in Designer and the R2 2D prototype they made really good progress on structuring this type of data better (some of the ways which would later be used in the Rayman 2 CPA engine).

[Chacanger]

But maybe your idea of making a separate tool is better. Only trouble would probably be getting the timing to match the game's frames. Good luck!

Managed to create a tool and it works "mostly"; as you say, the framerate can make the odd 1 or 2 commands go out of sync, but I was able to record some complex demos and with a small tweak they play like a charm.

A very crude winform application but it does the job for me.

It also demonstrated that the game accepts RAY0.LEV as a valid level, though it treats it as a bonus level and rayman dosen't appear possibly due to te timer being set to zero.

Now I need to go back to the games text (LNG data), I've managed to decrypt and encrypt fine with the xor keys and encoding but I''m unsure what to do about these bits...

• Unsure where the checksum I need to amend is stored in the uncompressed EXE for version 1.21 or any other version for that matter?

• I'm aware that the English, French and German languages work fine with the encoding code page value 850, but I'm noot sure what values are required for the japanese and chinese languages.

[RayCarrot]

Now I need to go back to the games text (LNG data), I've managed to decrypt and encrypt fine with the xor keys and encoding but I''m unsure what to do about these bits...

• Unsure where the checksum I need to amend is stored in the uncompressed EXE for version 1.21 or any other version for that matter?

• I'm aware that the English, French and German languages work fine with the encoding code page value 850, but I'm noot sure what values are required for the japanese and chinese languages.

In v1.21 it is stored at 0xB44E0 and looks like this:

Code: Select all

00 00 00 00 8A 10 00 00 30 B4 00 00
8A 10 00 00 69 12 00 00 82 A1 00 00 
F3 22 00 00 27 13 00 00 CF 5C 00 00 
1A 36 00 00 CF 09 00 00 D0 C0 00 00 
E9 3F 00 00 3E 09 00 00 95 14 00 00

Structure is:

Code: Select all

4 bytes: file offset
4 bytes: file size
1 byte: xor key
1 byte: checksum
2 bytes: padding

The Japanese and Chinese ones work a bit differently though by referencing sprites instead of being encoded as text. So each line of text is usually a single sprite.

I don't know the exe offset for the other versions, but you can easily search for it by checking the offsets and xor keys here. This is what Ray1Map uses. I hard-coded it in a bit of a lazy way, but it was the easiest for just reading the data. But if you're editing your levels on v1.21 then that should be enough, no? Transferring level files between versions doesn't work very well for various reasons (such as the vignette data being packed in a different order). And even so there's not one single v.1.12 release out there - but rather multiple for different languages which each have the exe compiled differently. Supporting all versions would be near impossible.

I'm aware that the English, French and German languages work fine with the encoding code page value 850

The encoding they use is 437, but others might work too. I also wrote a reader/writer for Rayman 1's text files here (mostly re-implemented from how the game does it), but I'm sure it's overkill for the language files since they only contain strings. It was more useful for the data in Rayman Designer.

[Chacanger]

The Japanese and Chinese ones work a bit differently though by referencing sprites instead of being encoded as text. So each line of text is usually a single sprite.

I don't think I'll bother with the Chinese or Japanese translations then, that sounds like a huge amount of extra work changing the sprites and getting the correct translations.

I managed to change most of the translations and it works fine in game, haven't checked the French or German ones yet though. I also need to update the credits (it will include me yourself and if you can think of any others that were involved in reverse engineering Rayman 1) but I couldn't see any of the names in the LNG file, so I assume they are held elsewhere in the EXE? Edit: I've found them in the EXE from address E1EAC.

I was contemplating on whether or not to change the world map medallion locations. I saw some stuff that was documented both here...

https://github.com/BinarySerializer/Bin ... rldInfo.cs

...and here...

https://github.com/BinarySerializer/Bin ... orldMap.cs

...but I got a little lost; do know what the pointers and structures are for this sort of stuff?

[RayCarrot]

I managed to change most of the translations and it works fine in game, haven't checked the French or German ones yet though. I also need to update the credits (it will include me yourself and if you can think of any others that were involved in reverse engineering Rayman 1) but I couldn't see any of the names in the LNG file, so I assume they are held elsewhere in the EXE? Edit: I've found them in the EXE from address E1EAC.

Yep, they're in the exe. There's a table of structs with pointers to them starting around 0xB71D4 in the exe (I haven't looked into the exact structure here). Might be hard adding new entries though. Maybe easiest just having the credits in like the loading screen or something like that?

I was contemplating on whether or not to change the world map medallion locations. I saw some stuff that was documented both here...

https://github.com/BinarySerializer/Bin ... rldInfo.cs

...and here...

https://github.com/BinarySerializer/Bin ... orldMap.cs

...but I got a little lost; do know what the pointers and structures are for this sort of stuff?

The first link is what you want yeah! It starts at 0xB6D5C in the exe and has 20 bytes per entry with there being 24 of them in the array (last 6 ones are the save points!). Some of the data gets filled in during runtime, but you can change their x and y positions, the levels they lead to and how they're linked on the map. The structure is:

Code: Select all

2 bytes: x position
2 bytes: y position
1 byte: up index (index of the entry to lead to when the user moves up from this entry)
1 byte: down index
1 byte: left index
1 byte: right index
2 bytes: runtime data - ignore this
1 byte: world (1-6, except save points which are set to 7 as a sort of Jaguar leftover where world 7 is the menus)
1 byte: level (1-30)
4 bytes: unknown
4 bytes: level name pointer (I think this gets set during runtime in the PC version)

[Chacanger]

Here is a new promo video for the mod...

https://rumble.com/v3z8md4-rayman-expert-promo.html

[Squiz]

Here is a new promo video for the mod...

https://rumble.com/v3z8md4-rayman-expert-promo.html

Saddly Rumble is now blocked in France so we can't see your video here