R3: Music Extraction Discussion Topic
Moderators: Droolie, English moderators
Forum rules
Please keep the forum rules and guidelines in mind when creating or replying to a topic.
Please keep the forum rules and guidelines in mind when creating or replying to a topic.
R3: Music Extraction Discussion Topic
I'm the master of the Chinese Rayman Fan-club who played Rayman for ten years. And I loved the music in Rayman very much. Today I extracted some files from the data.hos and data.hst file in Rayman 3, But only the sound effects in data.hos is playable. All waves in data.hst is unable to play. I know perhaps it is because the compression cannot be recognised.
Now the master of this site extracted the music successfully. I really want to know the secrect of the data.hst file, or the compress method. I'm nearly crazy this days for the music... Help me!
Now the master of this site extracted the music successfully. I really want to know the secrect of the data.hst file, or the compress method. I'm nearly crazy this days for the music... Help me!
Re: R3: How to extract the background music?
Hello and welcome to Pirate-Community!
We have been doing the same for a very long time, but still without success.
This is why I extracted the music from the PS2 version of the game, which is much easier to extract.
That is the music that can be found on RayTunes.
You could try to ask support for that type of sound on this forum in this topic, though.
More experienced people can reply to this topic if they know how to play these ADPCM files. Don't expect much answers, though...
Drolpiraat.
We have been doing the same for a very long time, but still without success.
This is why I extracted the music from the PS2 version of the game, which is much easier to extract.
That is the music that can be found on RayTunes.
You could try to ask support for that type of sound on this forum in this topic, though.
More experienced people can reply to this topic if they know how to play these ADPCM files. Don't expect much answers, though...
Drolpiraat.
Re: R3: How to extract the background music?
I've seen the post on XeNTaX several days ago by Google Search. Now it seems that we all have no ideas about the file...
But the file is correct. I use Filemon to see what Rayman3.exe do when I play and find that it really read the correct block of data.hst. Perhaps Ubi use a secrect function to coding the music. (perhaps mp3... every block ends up with the string "ump3") Can we search the computer RAM for some useful things?
Perhaps four years later, I graduate from the university and learn more about computer, such as ASM and DASM, memory...
But the file is correct. I use Filemon to see what Rayman3.exe do when I play and find that it really read the correct block of data.hst. Perhaps Ubi use a secrect function to coding the music. (perhaps mp3... every block ends up with the string "ump3") Can we search the computer RAM for some useful things?
Perhaps four years later, I graduate from the university and learn more about computer, such as ASM and DASM, memory...
R3: I've extracted one playable music from data.hst but...
I've extracted the 2d madness music from the data.hst file. After using WinHex to change the compression code from 02h to 01h, the music is playable. It is the correct music, but full of nosie. I think I've made some progress on data.hst because I really heard the music. But the noise is really...
I think that the data in data.hst may be mp3 format because the data is in many frames. One frame is 66Ah in size. I think perhaps we can learn some sound store formats, that we can know what's the correct compression method?
I think that the data in data.hst may be mp3 format because the data is in many frames. One frame is 66Ah in size. I think perhaps we can learn some sound store formats, that we can know what's the correct compression method?
Re: R3: I've extracted one playable music from data.hst but...
Yeah, I know, it's full of noise if you read it as RAW data as well / change the header of the file. Now you're at the point where I got stuck a few years ago...gmsj0001 wrote:I've extracted the 2d madness music from the data.hst file. After using WinHex to change the compression code from 02h to 01h, the music is playable. It is the correct music, but full of nosie. I think I've made some progress on data.hst because I really heard the music. But the noise is really...
I think that the data in data.hst may be mp3 format because the data is in many frames. One frame is 66Ah in size. I think perhaps we can learn some sound store formats, that we can know what's the correct compression method?
And I don't think it's mp3 format... the music quality wouldn't be that good if it was. And I don't think they used any ( commonly ) known format...
And please, don't make so many topics for this. I'll merge the two topics you made... Done.
Re: R3: Music Extraction Discussion Topic
Thank you...
Er... It's not mp3 format... It's ADPCM... If I make a Mp3 Wave format file and read it as raw data, I cann't hear the music any more. But If I make a MS-ADPCM wave file then read it as raw, I can hear the music inside... of cause full of nosie and four times faster...
It is said the ADPCM encoding method can be modify by yourself, So R3 must have its own ADPCM encoding method... I consider if I can read some useful tip from the frame header... Hummmmmmm....
Er... It's not mp3 format... It's ADPCM... If I make a Mp3 Wave format file and read it as raw data, I cann't hear the music any more. But If I make a MS-ADPCM wave file then read it as raw, I can hear the music inside... of cause full of nosie and four times faster...
It is said the ADPCM encoding method can be modify by yourself, So R3 must have its own ADPCM encoding method... I consider if I can read some useful tip from the frame header... Hummmmmmm....
Re: R3: Music Extraction Discussion Topic
Wait, I have an idea.
Could you send me that playable *.wav file you created?
With Yousendit, for example? ( my email address is [email protected] )
I might have found something that could make the noise disappear...
Could you send me that playable *.wav file you created?
With Yousendit, for example? ( my email address is [email protected] )
I might have found something that could make the noise disappear...
Re: R3: Music Extraction Discussion Topic
I've sent them to you now.
I think what we shoud do is not deal with the noise, but crack the ADPCM encode and decode method made by Ubi. The noise when read an ADPCM format file as raw is a normal thing, and four times faster is normal too because ADPCM compression rate is about 1/4.
If we discover the decode method, we are succeed...
I think what we shoud do is not deal with the noise, but crack the ADPCM encode and decode method made by Ubi. The noise when read an ADPCM format file as raw is a normal thing, and four times faster is normal too because ADPCM compression rate is about 1/4.
If we discover the decode method, we are succeed...
Re: R3: Music Extraction Discussion Topic
I am sure it will work out.gmsj0001 wrote:I've sent them to you now.
I think what we shoud do is not deal with the noise, but crack the ADPCM encode and decode method made by Ubi. The noise when read an ADPCM format file as raw is a normal thing, and four times faster is normal too because ADPCM compression rate is about 1/4.
If we discover the decode method, we are succeed...
Re: R3: Music Extraction Discussion Topic
Yeah, I guess. The Rayman 3 for GameCube soundtrack ( which has more tracks, possibly? It's the original version after all... ) has a higher priority for me though.
Re: R3: Music Extraction Discussion Topic
Sorry for posting a reply so long after the previous one.
Maybe like a lot of Rayman2/3 fans, I liked the soundtrack a lot, and really wanted to get the original music files. So I managed to code a little command-line tool to get the Rayman 2 soundtrack, encoded in ADPCM format. The program opens any .apm file and converts it to a standard .wav file, readable with any media player.
The only thing left to do is to merge the created .wav files to get a complete song (as you may know, the soundtrack of Rayman 2 is sliced in parts of 1 or 2 bars, so that the music changes depending on where you are in the level).
Some parts of songs are not played at all in the game ! (for example song endings)
I'll post the executable here in a few days.
Maybe I'll write another tool to get the .bnm files extracted, but I'm not planning to do it now.
=> Already done by Rene Smit, the tool's called F1BNM.
I've discovered Rayman 3 only a few days ago, and I've already taken a look in all these .bnh and .hxc files. Here are the results of the discoveries (maybe some of you have already noticed that):
The .bnh files are just text files renamed to .bnh, and each line of the file looks like:
cuuid( [event type, in hex], [event ID, in hex] ) * fixe [yes/no] * level [yes/no] * transition [yes/no] * file [filename]
The corresponding .hxc files seem to be structured binaries: the file names in the bnh files appear in these files. For each file, there is the name of an event, and a file name. I still don't know what the other bytes mean.
The musics/soundeffects are not in these files. They are all packed in 2 files:
Data.hos [~40 MB], which is, just like .bnm files, a concatenation of standard .wav files;
Data.hst [~498 MB], which contains a lot of sound files in .wav format, and encoded in Microsoft ADPCM.
For those like me who can't manage to play these files, I'm planning to make an extractor-decoder.
The only thing is that I'd like to have the right names for the sound files that will be extracted (at least something more interesting than "sample0001.wav").
I know the file names are in the .bnh/.hxc files, but... which name corresponds to what file?
One more thing. For those who are interested in decoding ADPCM files, I've found good algorithms on MultimediaWiki (they are the ones I use): the standard IMA-ADPCM format and how to decode it, and Microsoft's ADPCM format (0x02 in wav files).
Sorry for this --really-- long reply, but I hope this will help...
EDIT: Corrected some mistakes in this post, now that I know more about BNM and HXC (these formats are alike).
Maybe like a lot of Rayman2/3 fans, I liked the soundtrack a lot, and really wanted to get the original music files. So I managed to code a little command-line tool to get the Rayman 2 soundtrack, encoded in ADPCM format. The program opens any .apm file and converts it to a standard .wav file, readable with any media player.
The only thing left to do is to merge the created .wav files to get a complete song (as you may know, the soundtrack of Rayman 2 is sliced in parts of 1 or 2 bars, so that the music changes depending on where you are in the level).
Some parts of songs are not played at all in the game ! (for example song endings)
I'll post the executable here in a few days.
Maybe I'll write another tool to get the .bnm files extracted, but I'm not planning to do it now.
=> Already done by Rene Smit, the tool's called F1BNM.
I've discovered Rayman 3 only a few days ago, and I've already taken a look in all these .bnh and .hxc files. Here are the results of the discoveries (maybe some of you have already noticed that):
The .bnh files are just text files renamed to .bnh, and each line of the file looks like:
cuuid( [event type, in hex], [event ID, in hex] ) * fixe [yes/no] * level [yes/no] * transition [yes/no] * file [filename]
The corresponding .hxc files seem to be structured binaries: the file names in the bnh files appear in these files. For each file, there is the name of an event, and a file name. I still don't know what the other bytes mean.
The musics/soundeffects are not in these files. They are all packed in 2 files:
Data.hos [~40 MB], which is, just like .bnm files, a concatenation of standard .wav files;
Data.hst [~498 MB], which contains a lot of sound files in .wav format, and encoded in Microsoft ADPCM.
For those like me who can't manage to play these files, I'm planning to make an extractor-decoder.
The only thing is that I'd like to have the right names for the sound files that will be extracted (at least something more interesting than "sample0001.wav").
I know the file names are in the .bnh/.hxc files, but... which name corresponds to what file?
One more thing. For those who are interested in decoding ADPCM files, I've found good algorithms on MultimediaWiki (they are the ones I use): the standard IMA-ADPCM format and how to decode it, and Microsoft's ADPCM format (0x02 in wav files).
Sorry for this --really-- long reply, but I hope this will help...
EDIT: Corrected some mistakes in this post, now that I know more about BNM and HXC (these formats are alike).
Last edited by Synthesis on Sun Jan 25, 2009 6:37 pm, edited 1 time in total.
Re: R3: Music Extraction Discussion Topic
Finally, here it is!
Ray2Get, the command-line ADPCM decoder for Rayman 2 .apm files!
It took me a little time to finish it, and it still doesn't convert mono files properly (that's why "outro1.apm" is not in the .bat file).
Download link:
Ray2Get (zip, 13.6 kiB) (removed for terms of service violation or something)
Ray2Get (zip, 9.45 kiB) (new .bat file included)
For those who are interested in having an mp3 version of the original soundtrack:
Rayman 2 OST - Pack 1 (zip, 78.47 MB)
Rayman 2 OST - Pack 2 (zip, 79.15 MB)
I'm currently starting to code the Rayman 3 version of this tool...
Ray2Get, the command-line ADPCM decoder for Rayman 2 .apm files!
It took me a little time to finish it, and it still doesn't convert mono files properly (that's why "outro1.apm" is not in the .bat file).
Download link:
Ray2Get (zip, 13.6 kiB) (removed for terms of service violation or something)
Ray2Get (zip, 9.45 kiB) (new .bat file included)
For those who are interested in having an mp3 version of the original soundtrack:
Rayman 2 OST - Pack 1 (zip, 78.47 MB)
Rayman 2 OST - Pack 2 (zip, 79.15 MB)
I'm currently starting to code the Rayman 3 version of this tool...
Last edited by Synthesis on Sun Jul 10, 2011 9:41 pm, edited 1 time in total.
Re: R3: Music Extraction Discussion Topic
Wow, just wow! It's absolutely amazing! And you included all the unused ends of the tracks ( and an unused beginning for some of them! )!
You deserve 5000 tings for this. Here they are!
You'll get 5000 more when you release the R3 tool.
Can I put this on RayTunes instead of the current one?
EDIT -> I was right when I thought that I had heard track 89 ( the "ray" bonus track ) before... it's a jingle from the N64 version!
Also, it seems that the Precipice - Part 2 is completely new! o_O
EDIT 2 -> I love the beginning of the Precipice - Part 1. I would have used it as the music of an introduction of the level.
- Attachments
-
- ray2get.zip
- Ray2Get, the command-line ADPCM decoder for Rayman 2 .apm files!
- (13.66 KiB) Downloaded 498 times
Re: R3: Music Extraction Discussion Topic
Thanx Drolpiraat for the Tings!
Maybe you should not replace RayTunes's current Rayman 2 Soundtrack (the quality is better than my PC-version rips).
But you can add my files in a new "category" (or in "Other Rayman Soundtracks", it's up to you).
Actually my external hard-disk crashed a few months ago, and I had to re-code it all from scratch! But this second version creates standard wav files (the previous one only created .raw files that I had to convert to wav).
__________
I've also started to extract sound effects from Rayman 2 .bnm files (using F1BNM), but there are so many BNMs, and I can't automate the extraction process. But it'll be completed in a few weeks, maybe.
__________
Now I'm working hard on Ray3Get.
I've already "reverse-engineered" the HXC file format and wrote a .txt file containing some of its specs (just ask me if you want it).
The program now extracts the standard PCM files in Data.hos (MS-ADPCM will be implemented soon).
The final version will extract every file indexed in the .hxc files (actually yeah, they're indexes). So we'll have musics and also sound effects, with their original file names!
I really want to have it finished...
__________
EDIT: Sorry, I didn't mention it:
The "ump3" tags that can be seen in Data.hst's adpcm files have nothing to see with mp3 format, nor m3u or so.
These seem to be special subchunk tags for wav files, just like "data", "datx", "plst", "list", "labl" and so on... For more info, check out The Sonic Spot.
EDIT 2: Sorry for these over-long posts, I have lots of things to say
Maybe you should not replace RayTunes's current Rayman 2 Soundtrack (the quality is better than my PC-version rips).
But you can add my files in a new "category" (or in "Other Rayman Soundtracks", it's up to you).
Actually my external hard-disk crashed a few months ago, and I had to re-code it all from scratch! But this second version creates standard wav files (the previous one only created .raw files that I had to convert to wav).
__________
I've also started to extract sound effects from Rayman 2 .bnm files (using F1BNM), but there are so many BNMs, and I can't automate the extraction process. But it'll be completed in a few weeks, maybe.
__________
Now I'm working hard on Ray3Get.
I've already "reverse-engineered" the HXC file format and wrote a .txt file containing some of its specs (just ask me if you want it).
The program now extracts the standard PCM files in Data.hos (MS-ADPCM will be implemented soon).
The final version will extract every file indexed in the .hxc files (actually yeah, they're indexes). So we'll have musics and also sound effects, with their original file names!
I really want to have it finished...
__________
EDIT: Sorry, I didn't mention it:
The "ump3" tags that can be seen in Data.hst's adpcm files have nothing to see with mp3 format, nor m3u or so.
These seem to be special subchunk tags for wav files, just like "data", "datx", "plst", "list", "labl" and so on... For more info, check out The Sonic Spot.
EDIT 2: Sorry for these over-long posts, I have lots of things to say
Re: R3: Music Extraction Discussion Topic
Whew!
Didn't know the ADPCM files could be so hard to rip!
In fact, the files are split into very small parts (exactly 1536 bytes each).
That's why there are a lot of "cue ", "LIST" and "labl" tags at the end of each file. For more info, see link in previous post.
Didn't know the ADPCM files could be so hard to rip!
In fact, the files are split into very small parts (exactly 1536 bytes each).
That's why there are a lot of "cue ", "LIST" and "labl" tags at the end of each file. For more info, see link in previous post.
Re: R3: Music Extraction Discussion Topic
News about ray3get:
The format is really tricky to decode: it's not basic IMA-ADPCM format.
It seems to be something like MS-ADPCM.
Do you know any form of compression for which a flat signal is coded by value 0x7 ?
Also, I still don't know what the block headers are used for...
The format is really tricky to decode: it's not basic IMA-ADPCM format.
It seems to be something like MS-ADPCM.
Do you know any form of compression for which a flat signal is coded by value 0x7 ?
Also, I still don't know what the block headers are used for...
Re: R3: Music Extraction Discussion Topic
I wish I knew, but sadly, I don't.Synthesis wrote:Do you know any form of compression for which a flat signal is coded by value 0x7 ?
I will give you another 5000 tings for your hard work though ( and for the great silence-removing tool you made ). Here they are!
Re: R3: Music Extraction Discussion Topic
Thanx again.
Can't wait for the RayTunes site to be updated
I'm sure the key is the block headers, but I still don't know what the bytes corresponds to. I googled for some kind of specification, but never found anything that really fits...
Can't wait for the RayTunes site to be updated
I think it's MS-ADPCM-like, but this format is quite complicated, and when I try to decode I get either distortion or some almost-silent noise.Drolpiraat wrote:I wish I knew, but sadly, I don't.Synthesis wrote:Do you know any form of compression for which a flat signal is coded by value 0x7 ?
I'm sure the key is the block headers, but I still don't know what the bytes corresponds to. I googled for some kind of specification, but never found anything that really fits...
-
- Posts: 7312
- Joined: Sat Jan 12, 2008 3:22 pm
- Location: Poland, of Union of Socialistic European Republics
- Tings: 63250
Re: R3: Music Extraction Discussion Topic
In Poland, it is legal to disassmeble program to see what it do. Maybe go to Poland, disassemble R3 part responsible for playing songs and rewrite it in your programming language?
Re: R3: Music Extraction Discussion Topic
Well, that's a really good idea, but I wish I had enough knowledge about asm...
If anybody here could help...
If anybody here could help...