Java Sound Resources: FAQ: Performance Issues

Let's collect some ethernal wisdom:

As several applications prove (see links section), Java is sufficient to build audio editors, multitrack recording systems and MIDI processing software. Try it out!

If your still dont believe it, here some pointers to "scientific" work:

For some tips on how to measure Java Sound performance see Florian's mail. (Matthias)

At least for JIT compilers, Java is very fast (under certain conditions it reaches or tops C++ performance!) - and I think even on no-JIT compilers, FFT analysis is no problem in real time, of course depending on available processor power and resources. (Florian)

There are several possibilities:

(Matthias)

First of all, you shouldn't use it. sun.misc.Perf is not an official API. This means that the interface may change without notice, or the class may disappear completely. And Java implementation other than those based on the Sun code base are not likely to provide this class at all.

But OK, there might be reasons to use it. So here is the sketch:

					// may throw SecurityException
					sun.misc.Perf perf = sun.misc.Perf.getPerf();
					long ticksPerSecond = perf.highResFrequency();
					long currTick = perf.highResCounter();
					long milliSeconds = (currTick * 1000) / ticksPerSecond;

For more details, including javadoc, see Florian's mail. (Matthias)

The following table is based on measurements by YIP Chi Lap [Beta] (raw data):

(Matthias)

The fundamental problem with short sleeps is that a call to sleep finishes the current scheduling time slice. Only after all other threads/process finished, the call can return.

For the Sun JDK, Thread.sleep(1) is reported to be quite precise on Windows. For Linux, it depends on the timer interrupt of the kernel. If the kernel is compiled with HZ=1000 (the default on alpha), the precision is reported to be good. For HZ=100 (the default on x86) it typically sleeps for 20 ms.

Using Thread.sleep(millis, nanos) doesn't improve the results. In the Sun JDK, the nanosecond value is just rounded to the nearest millisecond. (Matthias)

. (Matthias)

You'll never be able to completely remove latency. Digital audio on computers is processed in "chunks", i.e. a small number of samples. When recording, you have to wait until one chunk is filled, then it gets passed to your application. So there is a minimum delay of the time of a chunk latency for recording. For playback the same "problem": the soundcard gets chunks from the application, so when one chunk is passed, the soundcard starts playing with the first sample of the chunk and it takes the time of the chunk that the last samples has been played. So, for playback you'll get the same delay. Added to that, the software (i.e. the different hardware layers copying chunks, etc.) and the hardware (hardware latency) add to the total delay.

There are approaches like DirectSound, ASIO or Alsa that all try to minimize the software part's latency and to allow very small chunks. When used efficiently on high-quality soundcards, an audio loop is possible with a barely noticeable delay (e.g. 20ms).

Java Sound (and therefore also JMF) actually adds one layer to the software part, so latency is slightly increased compared to native audio applications. You control the chunk size with the buffer size argument when constructing a SourceDataLine or TargetDataLine in Java Sound. I don't know about JMF's concepts. I also experienced some problems with Java Sound recording when using small buffers (like 1024 samples at 44.1 kHz). (Florian)

The delay, of when the data arrives regarded to when you started recording, should be quite fixed. However, recording and playback may have different delays. The delays depend on the soundcard, the operating system (including the soundcard drivers) and the implementation of Java Sound you use (and also a bit on what JDK you use).

Also, the Java Sound API does not provide means for querying the device's latency. This would be quite difficult anyway for the Java Sound subsystem. However, normal soundcards have very little hardware delay (even the cheap ones), so once they receive the command to start recording, they really start it in a matter of some milliseconds - typically 20 - 30ms.

Using the soundcard via Java Sound, there are many layers until the data arrives in your program, so some delay is added. However, I haven't noticed a relevant delay when recording. I haven't tried to do your approach of measuring the delay, though.

So what you should is, is to call the start() method both on TargetDataLine and SourceDataLine "at once" with the minimum overhead possible. Notice that the buffers of the recording line arrive when they have been finished recording (i.e. when the amount of time has been elapsed), while playback lines need a buffer prior that they are played. This may not be obvious at first. So when you do record and playback at once, the first received block from the recording line corresponds in time to the first played buffer (which has been submitted right with "start").

For playback of the 2 files, you should not use 2 playback lines, but mix them from your program. Like that, you can at least control the exact sample-delay of the 2 playback lines and adjust it (probably only possible by ear).

These are handmade solutions. The "real" solution is to make use of Mixer.synchronize(). This is the proposed solution to your problem from the Java Sound API. Like that you can synchronize the recording line and the playback line - or the 2 playback lines. However, it seems that it isn't implemented in current versions of Java Sound (although I haven't tested it). (Florian)

SourceDataLine.write() is very efficient: it just copies the passed data to a buffer. Being useful, it can't do less. If your program depends on the execution time of SourceDataLine.write(), it's typically a misconception of how it should be used. Audio data is handled in blocks of at least a few hundred bytes, often several kBytes. If you call SourceDataLine.write() with single samples or frames of 1, 2 or 4 bytes, you're indeed likely to run into a performance issue.

And a quote from Florian:

(Matthias)

Calling these methods triggers a lazy download of .jar files. This is a bug not specific to Java Sound. See also Topic: lazy download with ImageIO?, bug #5047548and Q: 2.2 (Matthias)