[Download] Ripping a soundtrack from an another UbiSoft game: Disney's PK: Out Of The Shadows

[A1020nt]

Any news ?

[The Jonster]

I find it curious this thread exists.

[PluMGMK]

Why so? It's not directly Rayman related, but there's no inherent reason that it wouldn't be of interest to some people here…

[The Jonster]

I didn't say it was a bad thing, I was just curious to see a non-Rayman related thread.

[A1020nt]

@Drolpiraat

Hello, is there a way, please, to modify your script "ra_ps2.bms" to be able to extract the segments in order, without giving them event names?
Thank you in advance.

[A1020nt]

Sorry to bump this. I know Droolie is busy, but is there any progress on this? I really can't work on this soundtrack if it stays with the event names in output, because the segments have numbers but they're not representative of the real order. There are a lot of segments and it makes the joining process extremely complicated and time consuming, considering I also reconstruct the music manually, without an automatic joiner or script. So I would be pleased if the script "ra_ps2.bms" could extract the segments without names (but with offset instead, since filenames aren't available).

Thanks in advance,

[Droolie]

Sorry to bump this. I know Droolie is busy, but is there any progress on this? I really can't work on this soundtrack if it stays with the event names in output, because the segments have numbers but they're not representative of the real order. There are a lot of segments and it makes the joining process extremely complicated and time consuming, considering I also reconstruct the music manually, without an automatic joiner or script. So I would be pleased if the script "ra_ps2.bms" could extract the segments without names (but with offset instead, since filenames aren't available).

Thanks in advance,

I just looked at this and the segment names indeed seem to be missing in PK PS2, or in different files.
Doesn't the bms script do what you want though? The script itself extracts the mib files with CUUIDs for names. Those files are later renamed with RayAudio, so if you remove the call to that from the bat file (the second line), it should keep the CUUID names.

If you prefer offsets, remove this portion of the script:

Code: Select all

string UNNAME += "_"
string UNNAME += KEY
String UNNAME += ".mib"

and modify this portion:

Code: Select all

	if EXT == 1
            get EXTSIZE long 0
            get EXTOFF long 0
            open FDSE EXTF 1
            log UNNAME EXTOFF EXTSIZE 1
		else
            log UNNAME FOFFSET FSIZE 0
        endif

so that the offset is added into the name here:

Code: Select all

	if EXT == 1
            get EXTSIZE long 0
            get EXTOFF long 0
            open FDSE EXTF 1
            string OFFSET_NAME P "%UNNAME%_external_%EXTOFF|4h%.mib"
            log OFFSET_NAME EXTOFF EXTSIZE 1
		else
            string OFFSET_NAME P "%UNNAME%_embedded_%FOFFSET|4h%.mib"
            log OFFSET_NAME FOFFSET FSIZE 0
        endif

Note: I haven't tested this new code, but it should work. You might need to replace the version of quickbms with the new version.

[A1020nt]

Thank you so much!! Though I thought it would correctly order the segments, but apparently the music isn't ordered by offsets in the DATA.HST. Do you still think there's another way to do it? it's approximately ordered with event names but sometimes it's really random (not your fault because they have bad filenames (no filenames)).

[DeeDyne]

Hello there ^^
After so many years of silence, the project is finished!
Real-life took over and I also focused on many other unrelated projects.
I almost completely forgot about it and left it in the work-in-progress pile for so long.

Now I'm finally back!
Everything has been properly exported in lossless FLAC with little to no altering.

If you wish to have download links or ask me anything in particular, feel free to contact me.
For now here's a YouTube link:
"Disney's Donald Duck: PK OST (Full In-Game Soundtrack)"
https://www.youtube.com/watch?v=gzdh8oR-WSU

Great thanks to Droolie and the entire community for all the help and tools you have provided!
Cheers!

[tudexo]

Hello,
Great work!
How did you manage to unscramble the music bits in the ps2 version?
I would like a download link if you can post it here or if you can DM me. I can't DM people as my account is limited for some reason.

[DeeDyne]

Thank you! ^^
Here's a temporary link that expires in about 7 days:
https://gofile.io/d/As0Szu

I have included the final release files in FLAC format and another ZIP which contains all of the organized and decoded chunks from the PAL version alongside M3U playlist files that memorize the orders I unscrambled.
I also checked the NTSC version, including the prototype, and I found no differences as the music chunks have all the same checksums even though many have different namings, but still with the same problem.

The answer to the sorting/unscrambling is complicated.

The short answer is:
Lots of different techniques combined together alongside lots of manual work, very attentive listening and analysis from different angles.

The long answer is:
I couldn't find any method/algorithm that could be automated by a script or a piece of code and that is because the 'scrambled' chunks don't have names of their own. That piece of information is missing.
Each time the script finds a chunk that has no name, it incrementally adds an ASCII underscore character to whatever the name of the previous chunk was, which is not indicative of where the chunk is actually supposed to be positioned in the original order.
For those chunks the original ordering information is lost, so it needs to be reconstructed.

Here's what I used to achieve that result:
Program/software list: foobar2000 with lots of various different plugins (including vgmstream), Adobe Audition, HxD and PSPad for hex editing and analysis, RayAudio for extraction, ffmpeg for regular audio conversions, some custom basic command-line utilities I coded myself in collaboration with other people (mostly regarding the manipulation of Sony 4-bit ADPCM chunks for the PS1 and PS2), HashTab and various other MD5/SHA-1 checksum utilities and more...

The reconstruction of the order of those unused chunks (even some of the used ones had their numbering wrong or in the reverse descending order) was possible also thanks to lots of reference material that has been uploaded on the internet over the years. The composer Daniel Masson himself posted MP3s of the soundtrack here and there on the internet, but those don't contain everything.

Another source of reference material was the GameCube version of the soundtrack funnily enough, because that version has the most consistent ordering with well-behaved numbering, even though it contains way fewer sequences.

For some sequences, using the source material wasn't enough.
So in some cases very attentive and repeated listening was required.

Sometimes the unused sequences would mimic the behaviour of known ones, meaning they'd evolve in the same way over time, but, for example, without drums, without specific instruments or with something that isn't present in the used ones.
It was verifiable by lining everything up and rapidly switching between files while carefully looking at the shapes in the spectrogram window.

So a lot of comparisons were made and lots of guesswork.

A more complex and technical approach was to carefully look at the consistency and coherence of waveforms and spectrograms between chunks.
What made it even more time-consuming was the fact that ADPCM in many forms, not just the Sony PlayStation variant, is not a 'seamlessly constant' stream, but it comes in 'packets' of fixed length.
Sony's ADPCM for the PlayStation for example compresses 28 16-bit samples of PCM data (56 bytes) into 16 byte-long packets of ADPCM data (1 for relative waveform positioning, 1 for flags and 14 for ADPCM audio).
That means: if you have a PCM stream that has a number of samples which is not a multiple of 28, you will have silence added as padding to fill up the last ADPCM packet.
This results in every single chunk of the soundtrack being characterized by a 'click' or a 'pop' at the end, which makes this analysis even more time-consuming.
I didn't fully fix them, but I found a way to make them 'a tiny bit smaller'.

The technical detail behind this is that, if you happen to tinker around with the available PlayStation SDKs, Sony's PlayStation ADPCM for some reason tends to add padding zeroes at the beginning (16 '0x00' null bytes at the beginning of each channel) which is always silence that wasn't there before the PCM to ADPCM conversion.
That first part of the problem is at the beginning of the file.
The other problem is at the end.
In order to make a Sony's ADPCM player stop properly, there should be a situation where the second to last chunk has a flag of 0x01 and the last one has a flag of 0x07, but for some reason that doesn't always happen.

What should happen is this (XXs could be any byte values):
XX 01 XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
07 07 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77

But instead what happens is this:
XX 01 XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
07 00 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77

That means that the last chunk gets interpreted as decodeable ADPCM audio which it shouldn't.
The consecutive 0x01 and 0x07 flags are what makes the ADPCM player to stop, but if the player doesn't see that 0x07 it will continue until the end of the file where there is no more data.
In other words the last chunk with that 0x07 flag shouldn't be decoded as audio.

Fixing this minor technical detail made the gaps smaller and it's a tiny improvement that maybe is not worth going through the trouble.
I decided to do it anyway with a few scripts I wrote and here's the difference of before and after:


In the picture you can also spot that these two chunks are meant to be concatenated one after the other given the shape and the slope the waveform is following. Without that padding/trough in the middle they would connect consistently with one another.
Each chunk has a silence padding at the end of different length and there's no easy way to fix it once and for all for every single file without potentially obtaining unwanted results.
The details are even more complicated. It's doable-ish, but it would involve lots of extensive work.

Anyway, huge information dump, sorry ^^'
All of this isn't a perfect approach as it's time-consuming and prone to human error, but with enough time and maniacal/thorough checking, it can be done.
I think I'm happy with the end result, but it's possible that it may not fit for everyone.

Cheers! ^^

[tudexo]

Very good work!
Thanks for the detailed explanation and the files.
Cheers!