Comment 9 for bug 792549

Found a solution. Apparently there is a problem with Pulse Audio on linux, when you do start/stop too frequently.

http://wiki.audacityteam.org/wiki/Linux_Issues

PulseAudio: Audacity's PulseAudio support generally works well. PulseAudio doesn't support ALSA's mixer interface so Audacity's Mixer Toolbar input slider is disabled under PulseAudio. Audacity can freeze using PulseAudio if playback or recording are started or stopped very rapidly or frequently. Playback can also occasionally be distorted or excessively fast. Patches are welcomed to improve our PulseAudio support. Meantime a reported workaround for freezes or glitched or skipped playback is to launch Audacity from the terminal with the following command:
env PULSE_LATENCY_MSEC=30 audacity