Comment 3 for bug 883485

Looking into this a little, xfce4-volumed uses whatever the active-card in xfconf is. For me, I set it with:
xfconf-query -c xfce4-mixer -p /active-card -s PlaybackInternalAudioAnalogStereoPulseAudioMixer

The following command will list all the properties under xfce4-mixer.
xfconf-query -lc xfce4-mixer

Then you can query them individually with:
xfconf-query -c xfce4-mixer -p /sound-card

More details about xfconf are here: http://wiki.xfce.org/releng/4.6/general-info
This should fix the mute problem, it does for me.