nscd and nss_ldap does not fail over

Ubuntu 14.04
openldap 2.4.31

Two ldap servers with replication, server1 and server2

On a client, in /etc/ldap.conf, configure 2 servers

host server1 server2

Failover works fine. If a server is down, the other server answers authentication requests.

Next, enable nscd, on the client.

service nscd start

Now during a failover event, ssh is broken.

Mar 30 02:00:18 client1 nscd: nss_ldap: could not search LDAP server - Server is unavailable
Mar 30 02:00:18 client1 sshd[31007]: Invalid user user1 from 10.1.2.3
Mar 30 02:00:18 client1 sshd[31007]: input_userauth_request: invalid user user1 [preauth]
Mar 30 02:00:24 client1 sshd[31007]: pam_unix(sshd:auth): check pass; user unknown
Mar 30 02:00:24 client1 sshd[31007]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.1.2.3
Mar 30 02:00:24 client1 sshd[31007]: pam_ldap: error trying to bind as user "uid=user1,ou=users,dc=company1,dc=net" (Invalid credentials)
Mar 30 02:00:24 client1 sshd[31007]: pam_warn(sshd:auth): function=[pam_sm_authenticate] service=[sshd] terminal=[ssh] user=[user1] ruser=[<unknown>] rhost=[10.1.2.3]
Mar 30 02:00:26 client1 sshd[31007]: Failed password for invalid user user1 from 10.1.2.3 port 28607 ssh2
Mar 30 02:00:32 client1 sshd[31007]: pam_unix(sshd:auth): check pass; user unknown
Mar 30 02:00:32 client1 sshd[31007]: pam_ldap: error trying to bind as user "uid=user1,ou=users,dc=company1,dc=net" (Invalid credentials)
Mar 30 02:00:32 client1 sshd[31007]: pam_warn(sshd:auth): function=[pam_sm_authenticate] service=[sshd] terminal=[ssh] user=[user1] ruser=[<unknown>] rhost=[10.1.2.3]
Mar 30 02:00:34 client1 sshd[31007]: Failed password for invalid user user1 from 10.1.2.3 port 28607 ssh2