getent group on trusty returns only local groups

I forgot to comment that I added extra line breaks to the output of "groups user" for readability of the bug report.

Attached smb.conf with workgroup and hostnames changed to generics.

Status changed to 'Confirmed' because the bug affects multiple users.

description was updated to clean up grammar and add clarity. Nothing substantive was changed.

The behavior of BUILTIN\ is not a bug but is intended like this. The idmap_ad plugin is only used for the WORKGROUP domain. everything else is up to idmap config * : range = 100000-300000. See man idmap_ad

If you try setting a gid to the groups in the AD, does this workaround the problem? (to be sure the -1 are comming from the idmap_ad backend)

I added a gid to two different groups within the AD/OU. This does fix the -1 GIDs, as evidenced by the replacement of one of the -1's with GIDs 2002 and 2003 for the two different groups, one for each user:

For the same user as shown above (see the appearance of the 2003):

user@Host:~$ wbinfo -r user
2001
-1
2003
10000
-1
-1
100002
100001

For an alternate user who had 2003 appear in place of the -1:

user@Host:~$ wbinfo -r user2
2001
-1
2002
100002

Just to add a few data points, we've been running Winbind for years to use accounts from our university's domain. We don't have Domain Admin access though so a lot of accounts belong to AD groups out of our control and that we can't assign GIDs to. This has never been an issue because Winbind would not report membership in such groups to Linux. Compare the output of Ubuntu 12.04 and 14.04 for the same account:

  ### 12.04 + winbind 2:3.6.3-2ubuntu2.10
  truffle:~$ groups mhatrak
  mhatrak : domain users ling-mayberrylab BUILTIN\users

  truffle:~$ id mhatrak
  uid=100051358(mhatrak) gid=513(domain users) groups=513(domain users),1310022(ling-mayberrylab),287(BUILTIN\users)

  ### 14.04 + winbind 2:4.1.6+dfsg-1ubuntu2
  enoki:~$groups mhatrak
  mhatrak : domain users groups: cannot find name for group ID 4294967295
  4294967295 groups: cannot find name for group ID 4294967295
  4294967295 ling-mayberrylab libuuid

  enoki:~$id mhatrak
  uid=100051358(mhatrak) gid=513(domain users) groups=513(domain users),4294967295,4294967295,1310022(ling-
  mayberrylab),101(libuuid)

The new behavior breaks sudo because it can't verify all the groups. It may be that this is the only program affected but I haven't done any exhaustive testing.

  enoki:~$sudo bash
  sudo: unable to set runas group vector: Invalid argument

Interestingly, SSSD works better now. It also reports an error but at least sudo is still functional.

  ### 14.04 + sssd 1.11.5-1ubuntu3
  porcini:~$ groups mhatrak
  mhatrak : groups: cannot find name for group ID 1000002
  1000002 ling-mayberrylab domain users

  porcini:~$ id mhatrak
  uid=100051358(mhatrak) gid=1000002 groups=1000002,1310022(ling-mayberrylab),513(domain users)

  porcini:~$ sudo bash
  porcini:~#

It seems to me the old Winbind behavior is desirable. There's little point in telling the OS about group membership for groups that are not going to work due to their lack of a GID. It should be easy to filter them out automatically or at least provide an option to do so.

Español:
El mismo problema aparece en Kubuntu 14.04. Lo descubri cuando investigaba errores en la generacion de los directorios de usuario. La instalacion es nueva, ya que se reemplazo un servidor preexistente que operaba con Kubuntu 12.04. La configuración de samba fue modificada para permitir un acceso de emergencia a todos los usuarios sin restricción. En este momento solo esta abierta a los usuarios un grupo restringido de carpetas de red con acceso completo. La creacion de carpetas de usuario "al vuelo" es operacional. Sin embargo el acceso esta supeditado a que hayan sido creadas previamente. Sospecho que el error puede influir sobre las ACL, aunque debido a la migracion de los discos entre ambos servidores no puedo asegurarlo. Adjunto los archivos de configuración relevantes. Los datos personales han sido retirados por seguridad

English:
The same problem occurs in Kubuntu 14.04. What I discovered when investigating errors in the generation of user directories. The installation is new, as it was replaced an existing server operating with Kubuntu 12.04. The samba configuration was modified to allow emergency access to all users without restriction. Currently is only open to a limited group of users network folders with full access. The creation of user boxes "on the fly" is operational. However access to this subject that have been previously created. I suspect the error may influence the ACL, although due to the migration of the discs between the two servers can not be sure. Attached the relevant configuration files. The personal details have been removed for security.

I'm facing the same problem... I'm desperately looking for a winbind replacement. Can someone, please, point one?! NSS + LDAP ?

Winbind have so many problems... Right now, `wbinfo -u` works, but `getent passwd` shows only half of AD users... Just terrible... It behaves differently from machine to machine, even using the very same setup across an office... :-/

FWIW, I ended up compiling the latest 3.6 version of Samba from the official source and it works just fine. I also tried sss with AD auth for a bit but it has its own set of issues in our environment.

I actually fixed this by doing the following steps:
# service smbd stop
# service winbind stop
# cd /var/lib/samba
# mv winbindd_cache.tdb winbindd_cache.tdb.backup
# mv winbindd_idmap.tdb winbindd_idmap.tdb.backup
# service smbd start
# service winbind start

then
# getent passwd
and
# getent group
both worked!

My smb.conf:
I[global]
        smb ports = 445
        security = ads
        realm = GE.LAN
        workgroup = GELAN
        server string = labo-opi1
        wins server = x.x.x.x # replace with ip of your actual winbind server
        encrypt passwords = yes
        client use spnego = yes
        client ntlmv2 auth = yes
        unix extensions = no
        winbind enum users = yes
        winbind enum groups = yes
        winbind refresh tickets = yes
        idmap config * : backend = tdb
        idmap config * : range = 16777216-33554431
        log file = /var/log/samba/%m.log
        max log size = 50
        socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
        load printers = no
        dns proxy = no
        unix charset = UTF-8

Sorry, the "I" before the [Global] section in my previous comment is of course a typo, it should not be there.
To futher get a bit of context here: I had this server running without problems on ubuntu server 12.04 and upgraded to 14.04.1 LTS using do-release-upgrade. This is when getent group stopped working. So I guess something has changed in winbindd_idmap.tdb?

I also switched from he old idmap syntax which is now deprecated:
   idmap uid = 10000-20000
   idmap gid = 10000-20000
to the new syntax:
        idmap config * : backend = tdb
        idmap config * : range = 16777216-33554431

Another (last) comment:
When doing this, the id mapping get "reassigned", so you'll have to change the rights in the filesystem (chowning) to the desired ad users/groups.

The solution I posted above is NOT stable. After some time, getent group only returns local groups again. The bug is not resolved by this.

The Canonical Server Team doesn't expect to be able to get round to this, unless we have a supported customer who requests it. I'd rather the bug status not set false expectations, so I'm unassigning this bug. If anybody else can drive this, then please do, and we'll try to help with any process or review issues.

For all those stuck with this issue. There's a real alternative to winbind, sssd. The following guide should get you started:
https://fedorahosted.org/sssd/wiki/Configuring_sssd_with_ad_server
Using the "realm" command is especially easy.

Once a getent passwd <email address hidden> returns something useful, this can easily be integrated with samba, without using winbind. The smb.conf should then include
   security = ads
   realm = GE.LAN
   kerberos method = secrets and keytab
in its [global] section.

I believe I have tracked down the source of this bug, which will hopefully lead to an easy fix.

The problem appears to be the inability of SID S-1-18-1 to be mapped (See https://support.microsoft.com/kb/2830145 for an explanation why). Winbind gets a list of all groups, and that SID is returned, then attempts to map them to GIDs but fails because that SID cannot be mapped.

If one runs:

wbinfo -U [uid]

then takes the SID that results and does

wbinfo --user-sids=[users SID]

a list of groups will be returned, along with the users SID. I am able to map all of them back to objects/groups in the domain, except for the S-1-18-1 SID.

This nicely matches the output of

groups [user]

which on my machine returns all of the groups I belong to, except for one, for which the command returns "groups: cannot find name for group ID 100000", where 100000 is the beginning of the idmap * range in smb.conf. I am almost certain the GID 100000 corresponds to the unmappable S-1-18-1 SID and is the reason "getent group" only returns local groups.

A patch may be as simple as winbind just ignoring S-1-18-1 and S-1-18-2 when returned as an SID for a group.

This appears to have been the behavior for earlier versions of winbind, as running

wbinfo -s [user SID]

on a centos 6 machine using Samba 3.6 returns all of the SIDs for the user's groups, except the bad S-1-18-1 SID.

Andreas Schneider seem to provide a fix for unmapped groups - which seems to be the background problem here. You can find the Bug and it's patch at https://bugzilla.samba.org/show_bug.cgi?id=10824
The fix seems to be included in samba 4.1.13 (to be verified), patch notes: http://www.samba.org/samba/history/samba-4.1.13.html

When will it be fixed?

Try to paste in smb.conf

idmap config * : range = 5000-16777216

idmap config ADM : backend = rid
idmap config ADM : range = 5000-33554431

@Sector

That may be a good workaround for some, but for us it won't help as changing to an RID backend would redo all of the UID numbers for users. That would break a number of configured services, and require a significant amount of time to fix.

Per Michael Wodniok's post, Samba 4.1.13 appears to have fixed the bug with groups that don't have a GID in AD. Samba/Winbind 4.1.13 is available in Ubuntu 15.04 (Vivid).

The issue with trying to map a group to SID S-1-18-1 still occurs, and I still see a "groups: cannot find name for group ID 100000". This appears to interfere with "getent groups" still.

On the other hand, setting a domain group as privileged with visudo and then running sudo commands as a user in that group appears to work, resolving the problem that prompted this bug report.

I can confirm this issue on Ubuntu Server 14.04 and Samba 4.1.6 with winbind. I can also confirm replacing winbind with sssd as suggested in post #20 is an easy workaround. sssd does not have this problem and all AD groups get enumerated correctly.

I also confirm this issue for Trusty with v. 4.1.6. It seems that the order in which the gids are evaluated is crtical, in that NO groups (including those with valid gids) are resolved after the first incidence of a group with missing gid, and therefore group memberships after this are simply missing. This ordering seems consistent for a given user, but varies between users.

I was able to apply the upstream patch to the 4.1.6 source package which is current in Trusty. Recompiling & reapplying the samba packages resolved the issue. The patch is quite simple (4 lines) & it would be nice if this could be backported to trusty (and utopic?) at some point; would be especially useful to prevent future updates to these packages from (re)breaking gid resolution.