(some) gnome keybindings only work after restarting gnome-settings-daemon

Binary package hint: gnome-control-center

I have a special keyboard that has addiational (multimedia) keys. I use setkeycodes to make the scancodes available for the system and xmodmap to make them available in X. My customized xmodmap configuration also contains other changes like some accent keys made dead while others remain "non-dead".
I found that the "official" way to apply those changes is to have the xmodmap config at /etc/X11/Xmodmap, which is read by /etc/gdm/Init/Default when starting gdm. This works fine, when I run "xmodmap -pke" after login, I have my keys correctly set up and they emit evets as expected.
However, when I assign functions to my speial multimedia keys via System -> Preferences -> Keybindings, they don't work. xev shows me that the keys work fine. If I do a "killall gnome-settings-daemon" and wait for it to restart automatically, the keybindings work correctly.
I searched for similar bugs and found that for some people it works if they put their xmodmap config to ~/.xmodmap, but this doesn't work for me. I already tried loading my xmodmap config at different points of the startup chain by putting "xmodmap /etc/X11/Xmodmap" to different scripts in both, gdm and Xsession.d, but I always got the same result: xmodmap seems to work, the keys work and have to correct keysyms, but the keybindings in gnome don't work before I restart gnome-settings-daemon.
I don't have anything related to it in my ~/.xsession-errors (somewhere someone mentioned an error line there which indicated that the keysyms were not registered (yet)).

Steps to reproduce:
1. Assign some additional keysyms to keycodes and put the corresponding xmodmap file to either /etc/X11/Xmodmap or ~/.xmodmap or start it manually using some of the many initialization scripts in gdm and xsession.
(for me, the multimedia keys have keycodes above 128)
2. Use Gnome's keybindings dialog to assign functions to the keys
3. Restart X

Result:
The keysyms are correctly assigned, the assigned functions for additional multimedia keys do not work.

Workaround:
1. Restart gnome-settings-daemon after login (this also works if the restart is performed using System -> Preferences -> Sessions -> Startup programs)
2. (alternatively) Reassign the keybindings to the exact same keys. This works without restarting gnome-settings-daemon.

Affects:
Ubuntu Dapper Drake 6.06.1 with latest packages from security, backports and updates repositories.
gnome-settigs-daemon version: 2.14.2