Using setTempo and putSamples in SoundTouch - soundtouch

Which samples does SoundTouch::setTempo affect, those already in the FIFO or those that are going to be put into the FIFO by SoundTouch::putSamples? That is, does SoundTouch::setTempo commit or prepare a tempo change?


mediaPlayer.getCurrentPosition() > mediaPlayer.getDuration() at the end of playing mp3 file

I'm playing mp3 file streamed from the network in my application, some mp3 files has weird behavior: mediaPlayer.getCurrentPosition() is larger than mediaPlayer.getDuration() at the end, for about 3 seconds.
The mp3 files are CBR encoded.
What might be the reason of this?
Finally solved the problem by converting the mp3 files, this is the command I'm using:
lame --mp3input -t -m s -b 128 --cbr input.mp3 output.mp3
There is a few reasons you can get this behavior.
First it appears that people had better results using mp3 files at exactly 44100Hz, because apparently the MediaPlayer class is assuming this value and scale the time accordingly, making strange values for files not using this sampling.
You also need to check the mode of your channels, and try using Joint Stereo or forced L/R Stereo. Joint should be the default, but your files might have been previously bad encoded, so it's worth trying. It's interesting to note that Forced L/R Stereo might loose quality for the same bitrate as Joint.
It would also be useful to check the output of soxi which is part of the sox package (you can also do it with ffmpeg), that will give you the number of channels, Sample rate, Bit Rate and Number of Channels.
Also you might want to check the raw content of the mp3 file if you did some treatment on them using any app for the presence of garbage xml content that might have been inserted during the export.
If you have the possibility to modify the mp3 files you're streaming, (which sounds like you do since you can tell the bitrate) these are what I would try first. If it's more like user-upload kind of stuff, maybe you should have a look to another solution instead, like ExoPlayer which has a few thousands stars and active development. It wraps the MediaPlayer api still, but worth a try.
You also have to consider that it might be a threading problem, where the player would stop playing, but the timer would actually keep going, giving you this result where it's superior to the actual duration of the song. 3 seconds seems a bit too much to explain it by that, but that's just a thought.

Controlling the state of appsink pipeline depending on its RTSP appsrc clients

I have a hierarchy like below:
GstElement pipeline(has a videotestsrc and appsink)
a GstAppSrc pointer array to push samples
GstRTSPMediaFactory (has a GstAppSrc named "appsrc0" and mounted on
GstRTSPMediaFactory (has a GstAppSrc named "appsrc1" and mounted on "/test2")
My caps on both appsink and appsrc are
caps = "video/x-raw,width=640,height=512,format=GRAY8"
So, on media factories I listen for media-constructed signals and register appsrc pointers to source pipeline. Also when their media changes state to GST_STATE_NULL i remove the pointer from appsrc array in SourcePipeline.
On the source pipeline, appsink pushes the samples to appsrc one by one. When there are no appsrcs left on the array, pipeline's state is changed to GST_STATE_NULL until the first appsrc joins again.
I have some questions and problems:
When 1st client connects to RTSP, the client instantly gets the stream. When the 2nd one joins on he 2nd mountpoint, stream pauses when GstRTSPMedia changes its state to GST_STATE_PLAYING, and after 5 - 6 seconds stream resumes again. This sometimes doesn't happen, though. The stream fails and I can't get it up again before restarting the program.
Is my approach of controlling SourcePipeline correct? How should I do it on an RTSP server?
I set appsrc's block property on TRUE. If I don't set it to true, It uses all the memory, until system becomes unresponsive. Again, what is the right approach here?
I am currently using push_sample to push buffers to appsrcs. What is the difference between push_sample and push_buffer? Which is more effective?
When 2 clients on different mountpoints are watching the stream, it breaks the stream down when 1 of them disconnects or stops the stream. I check new-state signal on GstRTSPMedia to know RTSP pipeline's states. Clearly this approach doesn't work, what is the correct one here?
I didn't need to do any synchronization between appsink and appsrc after I got time-stamps solved with
GST_BUFFER_TIMESTAMP(buf) = timestamps[i];
GST_BUFFER_DURATION(buf) = bufferDuration;
timestamps[i] += GST_BUFFER_DURATION(buf);
I already had suspicions about buffer time-stamps but couldn't figure out how to solve them. These 3 lines easily handle the problem, as long as the same clock is used between source pipeline and sink pipelines.

BASS/LAME Enc: How to properly continue MP3 recording / encoding?

I've got an app which records audio from microphone and uses BASSenc to encode the recording as MP3 utilizing LAME.
If you exit the app and restart it later, you can load and continue the (recording) session.
Everything is working fine so far. However, I'm wondering what is the proper way to continue the recording / encoding stream?
At the moment I open the old file using BASS_StreamCreateFile, start the recording and encoder and read through the whole file using BASS_ChannelGetData and BASS_Encode_Write to "transfer" the previous data over.
I'm concerned tho that this might not be the "right" way? This step will take longer and longer the bigger the recording gets. And I also think it is lossy, because I load a lossy MP3 and reencode it, losing even more information. Isn't it? Or am I wrong?
Is there a way to tell BASSenc / LAME to just use the existing file and append stuff to it directly? Or do I already use the proper approach?
(Delphi XE7, Windows, VCL)
You cannot directly append new, uncompressed data to a compressed stream without decompressing it first. So your approach seems plausible.
Size might indeed become an issue depending on the maximum intended length of your recordings. I'm handling uncompressed and compressed streams up to 1 GB, so far without experiencing major issues with delays. At least not on modern desktop machines.
However, a way to optimize the process would be to not load the old session data before recording the new data but instead to read it before saving the new session to disk. That way you can start recording immediately without allocating more and more RAM to the process each time.
"Right" and "wrong" becomes evident with the results. The best method is always the one that works best with the least effort to achieve it.
Multiple re-encoding, to my best knowledge, should not have any effect on quality if you didn't use a lossy codec.
Let me know if you need supporting source code for that.

Buffering URI on OpenSL, Android

I have been trying the OpenSL for a few weeks now. I'm trying to access to the buffer while playing a file on sdcard via SL_DATALOCATOR_URI as a source. I want to write a few effects of my own and need the buffer.
Currently in the code, I'm creating two audio players. One is reading the file to buffer, other is writing the buffer to the output. When I'm testing the code with the microphone (recorder), everything is fine. Sound in-out works as expected.
When I switch the recorder with a uri audioplayer, queue goes haywire. Streaming is not listening to thread locks (it occurs async as I understand) and buffer calls are not fired correctly, time flies away.
I've put logs to every method, so the result appears something like this:
V/PDecoder( 1292): Position : 15023
V/PDecoder( 1292): Position : 16044
V/PDecoder( 1292): Position : 17043
V/PDecoder Native PL1( 1292): bqPlayerCallback
V/PDecoder Native PL1( 1292): Notify thread lock
V/PDecoder Native PL1( 1292): android_AudioIn 32768
V/PDecoder Native PL1( 1292): Wait thread lock
V/PDecoder Native PL1( 1292): android_AudioOut 32768
V/PDecoder Native PL1( 1292): android_AudioIn 32768
V/PDecoder Native PL1( 1292): android_AudioOut 32768
V/PDecoder Native PL1( 1292): Wait thread lock
V/PDecoder Native PL1( 1292): bqRecorderCallback
V/PDecoder Native PL1( 1292): Notify thread lock
V/PDecoder( 1708): Position : 18041
V/PDecoder( 1708): Position : 19040
V/PDecoder( 1708): Position : 20038
Seconds fly away before queue callbacks are even fired.
So the question is, how can I correct this problem? Is there a way for audioplayer > buffer > output solution for uri playback? What am I doing wrong? If someone can point me to the right direction, it is greatly appreciated.
The code is a little long for pasting here, so here are the gists
H file gist
C file gist
After loosing myself in the code I gave in the question, decided to write it again, as clean as possible.
I found out that I were not locking the uri player after all. I'm adding the working final code at the end of the answer. The code is good for playing local file or url, but needs to be run in a thread started from java, or you'll lock the gui thread.
PS. The buffer is using stack, so you might want to move it to the heap, and probably save the pointer in the struct. Also, play, pause, destroy methods are not finished. If you want to use the code, you can easily implement these functions.
Bonus. The code also includes a simple way to call java instance methods (without the dreaded *env sent from java part). If you need it, look at JNI_OnLoad, then playStatusCallback() and then callPositionChanged() methods.
The code is a little long for pasting here, so here are the gists
H and C Files in a single Gist
Emrah, this is the precise problem I've been having for my project right now. I've been following this blog:
which is the circular buffer implementation of this, from the same blog:
In any case, upon studying the code it looks like he has his versions of your opensl-native h and c files, named opensl_io. He also has another class, opensl_example, that has an inbuffer and an outbuffer with a bit of simple processing in between. It seems like his recorder object fills the inbuffer of this opensl_example class, and his outbuffer populates his audioplayer object to play to sink. From what it sounds like, you were doing that as well.
Basically, I'm trying to replace the recorder object with an input stream from file, since I have to have access to chunks of buffer from the file if I want to process each chunk differently during stream, for example. you are using SLDATA_locator from the utf8 converted URI, which I'm trying to do now but I'm not exactly sure how to get a stream from it.
Right now, how the blog example works is that it takes the audio input from the recorder object in stream, puts it to the circular buffers as they fill, and put it through processing to output. I'm trying to just replace the source of the recorder buffers with my chunks of buffer from mp3. Again, it sounds like your code does precisely that. The audioprogramming blog's example is particularly complicated to alter to me because I'm not entirely sure with how SWIG works. But since you're using JNI, it might be easier.
Can you advise me in how yours works? Do you simply call StartPDecoderNative and then DecodeOn from Java with the uri string as a parameter?
Ok, tried running the c and h code with a simple java mainactivity that runs both those functions, in that order, on a button click.
Also, it looks like you also need a positionchanged method in java. What are you running in there? I can comment the part with jmethod out and the music plays, so that's working. Is it for seek?
Finally, maybe I'm just having a bit of trouble understanding it but which buffer are you doing the processing on, and where does it reside? Is it outbuffer? If I just wanted to, say, apply an fft or more simply just a scalar multiplication to the output sound, would I just multiply it to outbuffer before playing it out the final sink?

dynamically replacing elements in a playing gstreamer pipeline

I'm looking for the correct technique, if one exists, for dynamically replacing an element in a running gstreamer pipeline. I have a gstreamer based c++ app and the pipeline it creates looks like this (using gst-launch syntax) :
souphttpsrc location="http://localhost/local.ts" ! mpegtsdemux name=d ! queue ! mpeg2dec ! xvimagesink d. ! queue ! a52dec ! pulsesink
During the middle of playback (i.e. GST_STATE_PLAYING is the pipeline state and the user is happily watching video), I need to remove souphttpsrc from the pipeline and create a new souphttpsrc, or even a new neonhttpsource, and then immediately add that back into the pipeline and continue playback of the same uri source stream at the same time position where playback was before we performed this operation. The user might see a small delay and that is fine.
We've barely figured out how to remove and replace the source, and we need more understanding. Here's our best attempt thus far:
gst_element_unlink(source, demuxer);
gst_element_set_state(source, GST_STATE_NULL);
gst_bin_remove(GST_BIN(pipeline), source);
source = gst_element_factory_make("souphttpsrc", "src");
g_object_set(G_OBJECT(source), "location", url, NULL);
gst_bin_add(GST_BIN(pipeline), source);
gst_element_link(source, demuxer);
This doesn't work perfectly because the source is playing back from the beginning and the rest of the pipeline is waiting for the correct timestamped buffers (I assume) because after several seconds, playback picks back up. I tried seeking the source in multiple ways but nothing has worked.
I need to know the correct way to do this. It would be nice to know a general technique, if one exists, as well, in case we wanted to dynamically replace the decoder or some other element.
I think this may be what you are looking for:
(starting at line 115)