Upcoming feature (v1.5) - _MEMSOUND (Hi, Petr!)

[FellippeHeitor]

Hello @Petr and everyone else. Look at this:

Code: QB64: [Select]

1. OPTION _EXPLICIT
2. SCREEN _NEWIMAGE(600, 200, 32)
3. DIM bonus AS LONG, x AS LONG, x0 AS LONG, i AS _OFFSET, a%, c~&
4. bonus = _SNDOPEN("sine.ogg")
5. PRINT bonus
6. DO UNTIL _KEYHIT = 32: _LIMIT 10: LOOP
7.  
8. IF bonus THEN _SNDPLAY bonus
9. _DELAY .4
10.  
11. DIM s AS _MEM
12. s = _MEMSOUND(bonus, 1)
13. _DISPLAY
14.  
15. i = s.OFFSET
16. DO
17.     _MEMGET s, i, a%
18.     IF x MOD 2 = 0 THEN
19.         x0 = x0 + 1
20.         c~& = _RGB32(map(a%, -32768, 32767, 0, 255), 0, 255)
21.         LINE (x0, _HEIGHT / 2)-STEP(0, map(a%, -32768, 32767, -_HEIGHT / 2, _HEIGHT / 2)), c~&, BF
22.         IF x0 >= _WIDTH THEN x0 = 0: _DISPLAY: CLS
23.     END IF
24.     x = x + 1
25.  
26.     i = i + 3
27.     LOCATE 1, 1: PRINT i, s.OFFSET + s.SIZE
28.     IF i + 2 >= s.OFFSET + s.SIZE THEN EXIT DO
29. LOOP
30.  
31. FUNCTION map! (value!, minRange!, maxRange!, newMinRange!, newMaxRange!)
32.     map! = ((value! - minRange!) / (maxRange! - minRange!)) * (newMaxRange! - newMinRange!) + newMinRange!
33. END FUNCTION
34.  

_MEMSOUND is supposed to give you access to raw sound data after a file is decoded by using _SNDOPEN - in the same fashion as _MEMIMAGE gives you raw access to an image's data after you use _LOADIMAGE.

The code above won't work with v1.4 and not even with the latest dev build, as it's still in a separate branch for development. But it will yield the following results:

Is that what you had in mind, @Petr?

[Petr]

Hi Fellippe,

That's absolutely great. Perfect. That's exactly what I've wanted for so long. If this feature really will be in the next version of QB64, I will be absolutely happy. It will greatly expand the possibilities for everyone. We will no longer be limited to graphics, in which we have really huge possibilities, but will be able to combine it.

 

MEMSOUND is a fullfiled wish for me!

[Petr]

I think quickly and find that the following is needed:
- Function that returns the number of channels in the audio track:

 - if the data format is RAW, then all audio data will probably be in INTEGER format (if the output RAW format is 16 bits) and the number of channels will determine whether every other INTEGER should be read for the left channel (ie bytes 1.2 for the left track, bytes 3.4 for the right track, bytes 5.6 for the left track .... and so on, or in the case of mono all bytes will belong to the left audio channel), or if this will be handled differently and the left audio channel will be read separately into its own MEM block and right audio channel also in own separated memory block?

That's actually all, because with information about the number of channels in the audio track and RAW audio data, we can add everything else, if the stored RAW format in memory will always be the same type (16 bit). There are also WAV files that contain audio data in the format _UNSIGNED _BYTE (8 bit) or 32 bit, these contain audio data in the SINGLE format. Of course, it can be converted between each other, so one uniform format is enough in memory.

One thing is also needed. Audio sampling frequency. Will it always be the same, or will a new function be needed to return the current state of the sampling frequency? (in RAW MEM block)?

[FellippeHeitor]

Notice in the sample code above that I'm getting the memory block with _MEMSOUND(handle, channel) - if it's mono, I'm getting channel 1, if it's stereo, I can get channel 2. It'll return a memblock of size 0 if you try to read the right channel (channel 2) and it's a mono sound.

[Petr]

I see! It never occurred to me what the number one meant there! In that case, it is already solved! And even it can be loaded separately. This is absolutely perfect! Looks really very, very nice!

[FellippeHeitor]

One thing is also needed. Audio sampling frequency. Will it always be the same, or will a new function be needed to return the current state of the sampling frequency? (in RAW MEM block)?

The data available after _SNDOPEN is used is the result of the decoding and resampling that OpenAL does automatically, which means the output will (i believe) always be in accordance with _SNDRATE (usually returns 44100) and 16 bits per sample.

[Petr]

The data available after _SNDOPEN is used is the result of the decoding and resampling that OpenAL does automatically, which means the output will (i believe) always be in accordance with _SNDRATE (usually returns 44100) and 16 bits per sample.

So in that case, I think it's on the right track! I look forward to it. This feature function is a big amazing news!

[FellippeHeitor]

I can't wait for you to test it, @Petr. I don't know how comfortable you are with cloning the repository (or maybe you already have it cloned) - either way, let me know and I can provide instructions for you to get started with it, even though it still requires more thought before it goes live, even for the dev build.

Code: QB64: [Select]

1. OPTION _EXPLICIT
2. CONST true = -1, false = 0
3.  
4. DIM theSound AS LONG, x0 AS LONG, i AS _OFFSET, j AS _OFFSET
5. DIM a%, c~&, k&, trackerHeight AS INTEGER
6. DIM y0 AS LONG, y1 AS LONG
7. DIM mouseDown AS _BYTE, f$, mouseDownOn AS INTEGER
8. DIM sleft AS _MEM, sright AS _MEM
9. DIM stereo AS _BYTE
10. DIM stereoScreen AS LONG, monoScreen AS LONG
11.  
12. stereoScreen = _NEWIMAGE(882, 400, 32)
13. monoScreen = _NEWIMAGE(882, 200, 32)
14. SCREEN monoScreen
15.  
16. _TITLE "QB64 Sound Player"
17. _ACCEPTFILEDROP
18.  
19. IF _FILEEXISTS(COMMAND$) THEN
20.     f$ = COMMAND$
21. ELSE
22.     start:
23.     INPUT "File to load: ", f$
24. END IF
25.  
26. PRINT "Loading '"; f$; "'..."
27. theSound = _SNDOPEN(f$)
28. IF theSound = 0 THEN PRINT "Load failed.": END
29.  
30. loaded:
31. sleft = _MEMSOUND(theSound, 1)
32. sright = _MEMSOUND(theSound, 2)
33.  
34. IF sright.SIZE > 0 THEN
35.     SCREEN stereoScreen
36.     stereo = true
37.     y0 = _HEIGHT / 4
38.     y1 = _HEIGHT - _HEIGHT / 4
39. ELSE
40.     SCREEN monoScreen
41.     stereo = false
42.     y0 = _HEIGHT / 2
43. END IF
44. _PRINTMODE _KEEPBACKGROUND
45.  
46. trackerHeight = 50
47. _SNDPLAY theSound
48. _SNDPAUSE theSound
49.  
50. DO
51.     WHILE _MOUSEINPUT: WEND
52.     IF _MOUSEBUTTON(1) THEN
53.         IF mouseDown = false THEN
54.             mouseDown = true
55.             IF _MOUSEY < _HEIGHT - trackerHeight THEN
56.                 mouseDownOn = 1 'plot area
57.             ELSE
58.                 mouseDownOn = 2 'track area
59.             END IF
60.         ELSE
61.             IF mouseDownOn = 2 AND _SNDPLAYING(theSound) = 0 THEN
62.                 _SNDSETPOS theSound, map(_MOUSEX, 0, _WIDTH, 0, _SNDLEN(theSound))
63.             END IF
64.         END IF
65.     ELSE
66.         IF mouseDown THEN
67.             'toggle play
68.             IF mouseDownOn = 1 THEN
69.                 toggleSoundState theSound
70.             ELSEIF mouseDownOn = 2 THEN
71.                 _SNDSETPOS theSound, map(_MOUSEX, 0, _WIDTH, 0, _SNDLEN(theSound))
72.             END IF
73.             mouseDown = false
74.             mouseDownOn = 0
75.         END IF
76.     END IF
77.  
78.     k& = _KEYHIT
79.     IF k& = 32 THEN toggleSoundState theSound
80.  
81.     IF _TOTALDROPPEDFILES > 0 THEN
82.         IF _FILEEXISTS(_DROPPEDFILE$(1)) THEN
83.             _SNDSTOP theSound
84.             _SNDCLOSE theSound
85.             theSound = 0
86.             f$ = _DROPPEDFILE$(1)
87.             _PRINTSTRING (_WIDTH / 2 - _PRINTWIDTH("Loading...") / 2, _HEIGHT / 2 - _FONTHEIGHT), "Loading..."
88.             _DISPLAY
89.             theSound = _SNDOPEN(f$)
90.             IF theSound THEN
91.                 _FINISHDROP
92.                 GOTO loaded
93.             ELSE
94.                 PRINT "Load failed."
95.                 END
96.             END IF
97.         END IF
98.         _FINISHDROP
99.     END IF
100.  
101.     DIM alpha AS INTEGER
102.     alpha = 50 'IF _SNDPLAYING(theSound) THEN alpha = 40 ELSE alpha = 20
103.     LINE (0, 0)-(_WIDTH - 1, _HEIGHT - 1), _RGB32(0, alpha), BF
104.     GOSUB render
105.     IF _SNDPLAYING(theSound) = 0 THEN _PRINTSTRING (_WIDTH / 2 - 16, _HEIGHT / 2 - _FONTHEIGHT), CHR$(221) + CHR$(222)
106.  
107.     _DISPLAY
108.     _LIMIT 60
109. LOOP
110.  
111. render:
112. DIM o AS _OFFSET
113. o = INT((_SNDGETPOS(theSound)) * _SNDRATE)
114. WHILE o MOD 2 <> 0: o = o + 1: WEND
115. i = sleft.OFFSET + o
116. IF stereo THEN j = sright.OFFSET + o
117. FOR x0 = 0 TO _WIDTH
118.     _MEMGET sleft, i, a%
119.     IF a% < 0 THEN
120.         c~& = _RGB32(67, 150, 33, map(a%, 0, -32767, 80, 255))
121.     ELSE
122.         c~& = _RGB32(67, 150, 33, map(a%, 0, 32767, 80, 255))
123.     END IF
124.     LINE (x0, y0)-STEP(0, map(a%, -32768, 32767, -_HEIGHT / 4, _HEIGHT / 4)), c~&, BF
125.  
126.     IF stereo THEN
127.         _MEMGET sright, j, a%
128.         IF a% < 0 THEN
129.             c~& = _RGB32(67, 150, 33, map(a%, 0, -32767, 80, 255))
130.         ELSE
131.             c~& = _RGB32(67, 150, 33, map(a%, 0, 32767, 80, 255))
132.         END IF
133.         LINE (x0, y1)-STEP(0, map(a%, -32768, 32767, -_HEIGHT / 4, _HEIGHT / 4)), c~&, BF
134.     END IF
135.  
136.     i = i + 2
137.     IF stereo THEN j = j + 2
138.     IF i + 2 >= sleft.OFFSET + sleft.SIZE THEN EXIT FOR
139. NEXT
140.  
141. 'tracker:
142. LINE (0, _HEIGHT - trackerHeight / 3)-STEP(map(_SNDGETPOS(theSound), 0, _SNDLEN(theSound), 0, _WIDTH), -trackerHeight / 3), _RGB32(255, 180), BF
143. IF _MOUSEY > _HEIGHT - trackerHeight THEN
144.     LINE (0, _HEIGHT - trackerHeight)-STEP(_WIDTH, trackerHeight), _RGB32(255, 30), BF
145. END IF
146.  
147. RETURN
148.  
149. FUNCTION map! (value!, minRange!, maxRange!, newMinRange!, newMaxRange!)
150.     map! = ((value! - minRange!) / (maxRange! - minRange!)) * (newMaxRange! - newMinRange!) + newMinRange!
151. END FUNCTION
152.  
153. SUB toggleSoundState (theSound AS LONG)
154.     IF _SNDPLAYING(theSound) THEN
155.         _SNDPAUSE theSound
156.     ELSE
157.         DIM i AS INTEGER
158.         FOR i = -20 TO 20
159.             LINE (_WIDTH / 2 + 20, _HEIGHT / 2 - 5)-STEP(-40, i)
160.         NEXT
161.         _SNDLOOP theSound
162.     END IF
163. END SUB
164.  

[Petr]

I didn't know it was possible to download it from GitHub. I downloaded a version with the last update eight hours ago, now it's unpacking. The computer says it will unpack for an hour because the ZIP contains 31067 files. That means it will end sometime around midnight of my time. Then I run setup-win.bat and after finishing I will try it :)

[FellippeHeitor]

Hm, that might not be enough, unfortunately. Can you confirm you downloaded the memsound branch?

[SMcNeill]

Hm, that might not be enough, unfortunately. Can you confirm you downloaded the memsound branch?

If it’s the development branch, you should be able to test out the quick insert feature I just ended up pushing.  I’m thinking it should be working as intended now, without any issues, but it’d be nice to hear how it performs for others as well. ;)

[Petr]

Oops... An error has occurred ... I'm already downloading the correct version! Where did I look?

[FellippeHeitor]

Oops... An error has occurred ... I'm already downloading the correct version! Where did I look?

Assuming you're using Windows:

• After the memsound branch gets downloaded, run setup_win.bat. That'll generate qb64.exe - but this still doesn't contain _MEMSOUND
• Launch QB64 and load source/qb64.bas. Make sure "Output exe to source folder" is unchecked in the Run menu.
• Hit F11 and wait for compilation.
• Once compiled, you will have a qb64(2).exe file in your new QB64 folder. That's the version that will allow you to test _MEMSOUND

Let me know how it goes.

PS: The 7Zip version will likely unpack faster, but I don't know if you have 7Zip.

[Petr]

I use internal decompressors in Total Commander to unpack, 7zip can also. I'm already unpacking it to disk.

[SMcNeill]

Quick question:  Are you looking to implement a _NEWSOUND type command so we can _MEMPUT information to it directly?

For example:

I =  _NEWIMAGE(640,480,32)
M = _MEMIMAGE(I)
_MEMPUT M, M.OFFSET, image_data()

The above can take a memblock that we saved to disk, and upload it up into a new image handle, so we can then _PUTIMAGE it, or work with it the same as any other image.

Will we be able to do something like:

S = _NEWSOUND(123, 441000, 1) 123 seconds at 44100 sndrate, with 1 channel
DIM M AS _MEM
M = _MEMSOUND(S, 1)
CALL Load_Routine ‘Load sound data from disk
_MEMPUT M, M.OFFSET, Sound_Data()
SNDPLAY S