# FreeIPA / Red Hat IdM

> Identity, Kerberos, host enrolment, DNS and certificates in one domain, with the commands for enrolment, RBAC and replication problems.

Canonical: https://www.wiki.jodisand.me/idm/
Reviewed: 2026-09-24
Related: [DNS](https://www.wiki.jodisand.me/dns/index.md), [TLS and certificates](https://www.wiki.jodisand.me/tls/index.md), [SSH](https://www.wiki.jodisand.me/ssh/index.md), [systemd](https://www.wiki.jodisand.me/systemd/index.md)


FreeIPA (packaged and supported as Red Hat Identity Management) is a domain controller for Linux hosts. One enrolment gives a host a Kerberos identity, a certificate path, DNS records and a single policy source, so you stop managing local users, `sudoers` and TLS trust on each box. Reach for it when more than a handful of Linux hosts need shared accounts, single sign-on or host-based access control; a small static fleet is usually better served by configuration management alone.

## Cheatsheet

| Task | Command |
| --- | --- |
| Get a ticket | `kinit admin` |
| Show tickets | `klist` |
| Server health | `ipactl status` |
| Add a user | `ipa user-add alice --first=Alice --last=Bloggs` |
| Reset a password | `ipa passwd alice` |
| Add to a group | `ipa group-add-member ops --users=alice` |
| Enrol a host | `ipa-client-install --mkhomedir --domain=example.test` |
| Host keytab | `ipa-getkeytab -s ipa-01.example.test -p host/web-01.example.test@EXAMPLE.TEST -k /etc/krb5.keytab` |
| Service principal | `ipa service-add HTTP/web-01.example.test` |
| Request a certificate | `ipa-getcert request -f /etc/pki/tls/certs/web.crt -k /etc/pki/tls/private/web.key -K HTTP/web-01.example.test` |
| Certificate status | `getcert list` |
| DNS record | `ipa dnsrecord-add example.test web --a-rec=192.0.2.5` |
| Sudo rule | `ipa sudorule-add ops-all` |
| HBAC test | `ipa hbactest --user=alice --host=web-01.example.test --service=sshd` |
| Replication status | `ipa-replica-manage list -v ipa-01.example.test` |
| Client cache reset | `sss_cache -E` then `systemctl restart sssd` |

## What the pieces are

IdM runs several servers behind one command surface. 389 Directory Server holds the identity data, MIT Kerberos issues tickets, Dogtag signs certificates, BIND with an LDAP backend answers DNS, and SSSD on each client caches all of it and enforces policy locally. You administer the lot through the `ipa` CLI (or the web UI on port 443), which talks to the framework over a Kerberos-authenticated JSON-RPC API.

| Component | Responsibility | Fails as |
| --- | --- | --- |
| 389-ds | Users, groups, hosts, policies | LDAP timeouts, replication conflicts |
| KDC | Tickets and service principals | Clock skew, missing SPN, wrong realm |
| Dogtag | Issuing and renewing certificates | Expired CA subsystem certificates |
| BIND-DYNDB-LDAP | Forward and reverse DNS | Missing PTR, stale SRV records |
| SSSD (client) | Caching, HBAC, sudo, home directories | Stale cache, offline mode |

Kerberos depends on [DNS](https://www.wiki.jodisand.me/dns/) and on time. The client finds servers through `_kerberos` and `_ldap` SRV records, and a Kerberos exchange rejects any timestamp more than five minutes out. Those two facts, a missing SRV record and clock skew, are behind most enrolment and login failures. The default ticket lifetime is 24 hours and the maximum renewable age is seven days, both set per Kerberos policy.

## Kerberos tickets

A `kinit` proves identity once to the KDC and receives a ticket-granting ticket (TGT), cached in a credentials cache (ccache). Every later service access exchanges the TGT for a service ticket without the password, which is what single sign-on means here. The ccache location comes from `KRB5CCNAME`; a keyring cache (`KEYRING:persistent:$UID`, the Fedora and RHEL default) survives until logout, while a file cache (`FILE:/tmp/krb5cc_$UID`) is what most scripts and older tools expect.

```sh
kinit alice                                # password prompt; TGT valid for the policy lifetime
kinit -kt /etc/krb5.keytab host/web-01.example.test   # from a keytab, no password, for services and cron
kinit -R                                   # renew the current TGT without re-authenticating, up to the renew lifetime
klist -f                                   # flags: F forwardable, R renewable, and the renew-until time
klist -e                                   # encryption types; aes256-cts-hmac-sha1-96 or aes256-cts-hmac-sha384-192 expected, never RC4 (arcfour)
klist -kt /etc/krb5.keytab                 # principals and key version numbers (KVNO) held in a keytab
kdestroy                                   # discard the current ccache
kswitch -p alice@EXAMPLE.TEST              # switch between cached principals when a collection cache holds several
```

Service tickets are cached alongside the TGT; a stale service ticket after a keytab rotation shows as a KVNO mismatch even though `kinit` works, so `kdestroy` then re-request. `kvno HTTP/web-01.example.test` asks the KDC for a fresh service ticket and prints the version, which is the quick check that a service principal exists and its keytab is current.

```sh
kvno HTTP/web-01.example.test              # request a service ticket; fails if the SPN is missing
ipa krbtpolicy-mod --maxlife=8h --maxrenew=$((7*24))h   # tighten the default ticket lifetime domain-wide
ipa krbtpolicy-mod alice --maxlife=1h      # per-user override for a privileged account
KRB5_TRACE=/dev/stderr kvno HTTP/web-01.example.test    # every step, for diagnosing "no support for encryption type"
```

Delegation is the sharp edge. Unconstrained delegation lets a service impersonate the user to any other service and is a serious finding; prefer constrained delegation (`ipa servicedelegationrule`) that names the target services. Forwardable tickets (`kinit -f`) are what allow SSH with GSSAPI to carry your identity to the next host; disable forwarding for accounts that never need it.

## Users and groups

The user and group objects are the identities every other policy references. Disable a departed user rather than deleting: disabling revokes access at once while keeping the UID, group memberships and file ownership, so audit trails and on-disk permissions stay intact. Delete only when you are certain nothing on any host still refers to the UID.

```sh
ipa user-add alice --first=Alice --last=Bloggs --email=alice@example.test --shell=/bin/bash
ipa user-mod alice --title='Platform Engineer' --sshpubkey="$(cat ~/.ssh/id_ed25519.pub)"
ipa user-find --all --raw alice        # --raw shows LDAP attribute names, useful for scripting
ipa user-disable alice                 # blocks authentication; keeps the object and memberships
ipa user-del alice --preserve          # moves to the deleted-users container, recoverable with user-undel
ipa passwd alice                       # sets a one-time password the user must change at next login
```

```sh
ipa group-add ops --desc='Platform on-call'
ipa group-add-member ops --users=alice --groups=contractors   # groups can nest
ipa group-show ops --all
```

Password policy is per group and evaluated by priority (lower number wins when a user is in several). Set a stricter policy on privileged groups than on the global default.

```sh
ipa pwpolicy-add ops --maxlife=90 --minlength=14 --history=10 --priority=1
ipa pwpolicy-show ops
```

Groups have a type. A POSIX group has a GID and appears in `getent group`; a non-POSIX group exists only in the directory for RBAC or as a mail list; an external group holds references to identities from a trusted Active Directory and is how AD users receive IdM policy. Convert a plain group to POSIX with `ipa group-mod --posix`; you cannot go back. User private groups (one per-user group matching the UID) are created automatically unless the install disabled them.

```sh
ipa group-add app-owners --nonposix                       # RBAC only, no GID
ipa group-add-member ops --external='EXAMPLE\\ad-admins'  # AD group via a trust; needs --external on a group created with --external
ipa group-add ops-posix --posix                           # allocates a GID from the ID range
ipa idrange-find                                          # the UID/GID ranges IdM assigns from
```

Automember rules add a new user or host to groups automatically from an attribute, so joiners land in the right groups without a manual step. Rebuild membership after adding a rule; automember only fires on creation otherwise.

```sh
ipa automember-add ops --type=group
ipa automember-add-condition ops --type=group --key=title --inclusive-regex='Platform'
ipa automember-rebuild --type=group                       # apply to existing users; can be heavy on a large directory
```

## Hosts and services

A host must exist as an object before it can hold a keytab, and enrolment creates that object and its `host/` principal. A service (a web server, a database) gets its own principal so it can accept Kerberos tickets or hold a certificate independently of the host.

```sh
ipa host-add web-01.example.test --ip-address=192.0.2.5
ipa-client-install --domain=example.test --realm=EXAMPLE.TEST \
  --principal=admin --mkhomedir --enable-dns-updates   # run on the client; needs root
```

```sh
ipa service-add HTTP/web-01.example.test
ipa service-allow-retrieve-keytab HTTP/web-01.example.test --hosts=web-01.example.test
ipa-getkeytab -s ipa-01.example.test -p HTTP/web-01.example.test -k /etc/httpd.keytab
klist -kt /etc/httpd.keytab                            # list principals and key versions in the keytab
```

> [!WARNING]
> By default `ipa-getkeytab` resets the principal's key and writes the new one to the keytab, which invalidates every keytab already issued for that principal. Two hosts sharing a principal will knock each other offline. Use one principal per host per service, or pass `-r` to retrieve the existing key without regenerating it (retrieval must be permitted with `service-allow-retrieve-keytab` first).

```sh
ipa-client-install --uninstall         # clean removal on the client before re-enrolment
ipa host-del web-01.example.test       # then remove the stale host entry on a server
```

## DNS

IdM can run the domain's DNS so that enrolment maintains the SRV, A and PTR records clients depend on. Use the integrated DNS when hosts come and go; keep an external DNS if you already operate one, but then you must publish the `_kerberos` and `_ldap` SRV records yourself.

```sh
ipa dnszone-add example.test --dynamic-update=TRUE            # allow enrolled hosts to update their records
ipa dnsrecord-add example.test web --a-rec=192.0.2.5
ipa dnsrecord-add 2.0.192.in-addr.arpa 5 --ptr-rec=web.example.test.   # note the trailing dot
ipa dnsrecord-find example.test
ipa dnsconfig-mod --forwarder=192.0.2.53 --forward-policy=only
dig +short SRV _kerberos._udp.example.test                   # what a client resolves to find the KDC
```

If the SRV records are wrong or missing, enrolment fails with an unhelpful message about the realm rather than about DNS. Check them first. See [DNS](https://www.wiki.jodisand.me/dns/#a-name-that-will-not-resolve) for resolving the underlying lookup.

## Certificates

Dogtag issues X.509 certificates from the IdM CA, and `certmonger` tracks each one on the host, renewing it before expiry and running a post-save command to reload the service. A certificate issued outside this tracking is the one that expires unnoticed. See [TLS](https://www.wiki.jodisand.me/tls/) for verifying the resulting chain.

```sh
ipa-getcert request -f /etc/pki/tls/certs/web.crt -k /etc/pki/tls/private/web.key \
  -K HTTP/web-01.example.test -D web-01.example.test        # -K principal, -D SAN dNSName
getcert list                      # tracked certificates: expiry, state, and the post-save command
getcert resubmit -i <request-id>  # force renewal now instead of waiting for the auto-renewal window
ipa cert-find --subject=web-01
ipa cert-show 12 --out=cert.pem
ipa cert-revoke 12 --revocation-reason=4    # reason 4 = superseded
```

```sh
ipa-cacert-manage renew            # renew the CA certificate; run on the CA renewal master only
ipa-certupdate                     # refresh the local trust store on every host after a CA change
```

> [!IMPORTANT]
> Keep at least two replicas with CA capability and confirm the CA renewal master is healthy before certificates approach expiry. If the CA subsystem certificates lapse, issuance and renewal stop for the whole domain.

## Access control

HBAC (host-based access control) decides who may reach which service on which host; sudo rules decide what they may then run as root. SSSD on the client evaluates both against cached data, so a policy change is not live until the cache refreshes. Service names (`sshd`, `login`) map to PAM services on the client, so an HBAC rule that gates [SSH](https://www.wiki.jodisand.me/ssh/) must name `sshd`.

```sh
ipa hbacrule-add ops-ssh
ipa hbacrule-add-user ops-ssh --groups=ops
ipa hbacrule-add-host ops-ssh --hostgroups=web-servers
ipa hbacrule-add-service ops-ssh --hbacsvcs=sshd
ipa hbactest --user=alice --host=web-01.example.test --service=sshd   # simulate before you rely on it
```

```sh
ipa sudorule-add ops-restart
ipa sudorule-add-user ops-restart --groups=ops
ipa sudorule-add-host ops-restart --hostgroups=web-servers
ipa sudorule-add-allow-command ops-restart --sudocmds='/bin/systemctl restart nginx'
ipa sudorule-mod ops-restart --sudoopt='!authenticate'    # no re-prompt for the password
```

> [!WARNING]
> The default `allow_all` HBAC rule permits every user on every host. Disable it only after replacement rules pass `hbactest`, or the next login locks everyone out.

```sh
ipa hbacrule-disable allow_all
```

HBAC service groups let one rule cover several PAM services. The distinction between `sshd` and `login` matters: `sshd` gates network SSH, `login` the local console, and a rule that only names `sshd` still lets someone at the keyboard in. `sudo` is a separate PAM service again, so an HBAC rule that omits it does not stop `sudo` from prompting, only the sudo rules do.

```sh
ipa hbacsvcgroup-add remote-access
ipa hbacsvcgroup-add-member remote-access --hbacsvcs=sshd,cockpit
ipa hbacrule-add-service ops-ssh --hbacsvcgroups=remote-access
ipa hbacrule-find --all | grep -A3 'Rule name: ops-ssh'
```

Sudo rules mirror HBAC in shape: subjects (users or groups), hosts (or host groups), and what they may run (command or command group), plus run-as and options. Build command groups so a rule references one named set rather than a list that drifts.

```sh
ipa sudocmd-add '/usr/bin/systemctl restart nginx'
ipa sudocmdgroup-add web-restart
ipa sudocmdgroup-add-member web-restart --sudocmds='/usr/bin/systemctl restart nginx'
ipa sudorule-add-allow-command ops-restart --sudocmdgroups=web-restart
ipa sudorule-mod ops-restart --runasusers=root --runasgroups=root
ipa sudorule-add-option ops-restart --sudooption='!authenticate'
ipa sudorule-add-option ops-restart --sudooption='logfile=/var/log/sudo-ops.log'
ssh alice@web-01 sudo -l                 # confirm from the client after sss_cache -E; the client compiles the rules, not the server
```

The client fetches sudo rules over LDAP through SSSD, not from `/etc/sudoers`. `sssctl` shows what the client actually resolved, which is the fastest way to tell a rule problem from a cache problem.

```sh
sssctl user-checks alice -s sudo         # what SSSD returns for sudo for this user
sudo -l -U alice                          # the compiled sudo policy, run as root on the client
```

Delegate administration with roles instead of adding people to `admins`. A role holds privileges, a privilege holds permissions, and a permission grants a specific right on specific attributes. This keeps a helpdesk able to reset passwords without being able to change group membership or policy.

```sh
ipa role-add 'Helpdesk'
ipa privilege-add 'Password Reset'
ipa permission-add 'Reset user password' --type=user --right=write --attrs=userPassword
ipa privilege-add-permission 'Password Reset' --permissions='Reset user password'
ipa role-add-privilege 'Helpdesk' --privileges='Password Reset'
ipa role-add-member 'Helpdesk' --groups=helpdesk
```

## Replication

Replicas hold full read-write copies of the directory, so any one can serve the whole domain and no single server is a point of failure. Run at least two, and at least two with CA capability. Replication is multi-master, which means a write on two replicas to the same entry can collide and produce a conflict that needs manual resolution.

```sh
ipa-replica-manage list -v ipa-01.example.test   # agreements and last-update time per peer
ipa-replica-manage force-sync --from ipa-01.example.test
ipa-csreplica-manage list                        # certificate-server (Dogtag) replication
ipa-healthcheck --failures-only                  # run on each server; the fastest triage
```

Replication conflicts appear as directory entries carrying `nsds5ReplConflict`. Resolve them by choosing the surviving entry and deleting the conflict copy.

```sh
ldapsearch -Y GSSAPI -b "dc=example,dc=test" "(nsds5ReplConflict=*)" \* nsds5ReplConflict
```

## Keycloak and OIDC

FreeIPA authenticates Linux hosts with Kerberos and LDAP; web and API single sign-on wants OIDC or SAML, which is Keycloak's job. A common layout is Keycloak as the OIDC provider federated to FreeIPA over LDAP for the account source, so people have one password and applications speak OIDC. Keycloak calls its tenant a realm (unrelated to a Kerberos realm), and each application is a client.

Every realm exposes standard OIDC endpoints under `/realms/<realm>/protocol/openid-connect/`, discoverable at the well-known URL. Point applications at the discovery URL and let them read the rest.

```sh
# Discovery document: every endpoint, supported grant types and signing keys
curl -s https://sso.example.com/realms/example/.well-known/openid-configuration | jq '{authorization_endpoint, token_endpoint, userinfo_endpoint, end_session_endpoint, jwks_uri, grant_types_supported}'
```

| Endpoint | Path | Use |
| --- | --- | --- |
| Authorization | `.../auth` | Browser redirect that starts the authorization-code flow |
| Token | `.../token` | Exchange a code, refresh token or client credentials for tokens |
| UserInfo | `.../userinfo` | Claims about the user, given an access token |
| Logout | `.../logout` | End the SSO session (RP-initiated logout) |
| JWKS | `.../certs` | Public keys to verify a token signature |

The flows, and when to use each:

- **Authorization code with PKCE**: interactive users in a browser or mobile app. The app never sees the password; it receives a short-lived code and swaps it for tokens server-side (or with PKCE, safely in a public client). This is the default and the only one to use for people.
- **Client credentials**: a service authenticating as itself, no user. Returns an access token for the client's own service account.
- **Refresh token**: swap a refresh token for a new access token without re-prompting, within the refresh token's lifetime.
- **Device code**: input-constrained devices; the user visits a URL and enters a code shown on the device.
- **Direct access grants (password)**: the app collects the password and sends it. Only for trusted first-party tooling or migration; disable it on every client that does not need it, because it defeats MFA and the SSO session.

```sh
# Client credentials: a service getting its own token
curl -s -X POST https://sso.example.com/realms/example/protocol/openid-connect/token \
  -d grant_type=client_credentials -d client_id=my-service -d client_secret="$CLIENT_SECRET" | jq -r .access_token

# Inspect a token you already hold (header and claims; does not verify the signature)
jq -R 'split(".") | .[1] | @base64d | fromjson' <<< "$ACCESS_TOKEN"

# Verify and introspect a token server-side (needs a confidential client's credentials)
curl -s -X POST https://sso.example.com/realms/example/protocol/openid-connect/token/introspect \
  -u "my-service:$CLIENT_SECRET" -d token="$ACCESS_TOKEN" | jq '{active, sub, preferred_username, realm_access}'
```

`kcadm.sh` is the admin API client; it authenticates once and stores the token, then creates and edits realm objects. Store the admin password out of the command line.

```sh
kcadm.sh config credentials --server https://sso.example.com --realm master --user admin --password-stdin <<< "$KC_ADMIN_PASSWORD"
kcadm.sh create realms -s realm=example -s enabled=true
kcadm.sh create clients -r example -s clientId=my-app -s 'redirectUris=["https://my-app.example.com/*"]' \
  -s publicClient=false -s standardFlowEnabled=true -s directAccessGrantsEnabled=false -s 'attributes."pkce.code.challenge.method"=S256'
kcadm.sh get clients -r example --fields id,clientId | jq -r '.[] | [.clientId, .id] | @tsv'
kcadm.sh create clients/<client-id>/client-secret -r example        # generate a secret
kcadm.sh get clients/<client-id>/client-secret -r example -F value
kcadm.sh create users -r example -s username=alice -s enabled=true -s email=alice@example.test
kcadm.sh set-password -r example --username alice --temporary   # prompts; user must reset at first login
kcadm.sh add-roles -r example --uusername alice --rolename platform-admin
```

Roles arrive in the token under `realm_access.roles` (realm roles) or `resource_access.<client>.roles` (client roles); a client maps them into the token with a role mapper, and a group mapper adds `groups`. An application authorises on those claims. Map FreeIPA groups through the LDAP federation's group mapper so IdM stays the source of truth for membership, and see [Vault OIDC](https://www.wiki.jodisand.me/vault/#oidc) for the same claims driving Vault policy.

## Client troubleshooting

Most login problems are visible from the client with a ticket request and the SSSD logs. Trace the Kerberos exchange when the error is unclear, and check the clock first when nothing else explains it.

```sh
klist -e                                   # current tickets and their encryption types
kinit -V alice                             # verbose ticket request
KRB5_TRACE=/dev/stderr kinit alice         # every step of the AS/TGS exchange
id alice                                   # does the client resolve the user through SSSD at all
getent passwd alice
sss_cache -E && systemctl restart sssd     # clear the SSSD cache after a policy change
journalctl -u sssd -f                      # follow SSSD while you reproduce the failure
realm list                                 # domain membership from the client's point of view
chronyc tracking                           # clock offset; skew over five minutes breaks Kerberos
```

| Symptom | Cause | Check |
| --- | --- | --- |
| `Clock skew too great` | Time drift beyond five minutes | `chronyc tracking`; fix NTP before anything else |
| `Server not found in Kerberos database` | Missing SPN, or a name with no matching principal | `ipa service-find`; confirm the exact principal name |
| `KDC has no support for encryption type` | Mismatched enctypes after an upgrade | Regenerate the keytab; check the principal's `krbEncType` |
| Login works, sudo does not | Sudo rule unmatched, or the SSSD cache is stale | `sss_cache -E`; verify `sudorule` host and user members |
| User exists in IdM but `id` fails | SSSD domain disabled, or the host is not enrolled | `realm list`; `sssctl domain-status example.test` |
| Everything fails after a CA renewal | Clients hold the old CA trust | `ipa-certupdate` on each client |
| Intermittent failures | One replica unhealthy and still in rotation | `ipa-healthcheck --failures-only` on every server |
| `kinit` works but a service rejects the ticket | Stale service ticket after a keytab rotation, KVNO mismatch | `kdestroy`; `kvno HTTP/host`; compare `klist -kt` KVNO with the KDC |
| SSH single sign-on stops carrying identity | Ticket not forwardable, or GSSAPI delegation off | `klist -f` for the F flag; `kinit -f`; `GSSAPIDelegateCredentials yes` |
| `id` slow or times out on the client | SSSD querying an unreachable or slow replica | `sssctl domain-status example.test`; set `ldap_uri` failover; `journalctl -u sssd` |
| Automember rule added but existing users not in the group | Automember only fires on creation | `ipa automember-rebuild --type=group` |
| Sudo rule shows in `ipa sudorule-show` but not `sudo -l` on the host | SSSD cache stale, or sudo not in an HBAC service allowed on the host | `sss_cache -E`; `sssctl user-checks alice -s sudo` |
| OIDC login loops or `invalid redirect_uri` | The client's `redirectUris` does not match the app's callback exactly | `kcadm.sh get clients/<id>` redirectUris; add the exact URL |
| Token accepted by one app, rejected by another | Audience (`aud`) or issuer (`iss`) mismatch, or clock skew to Keycloak | Decode the token's `aud`/`iss`; check the app's expected issuer and NTP |
| Keycloak login fails for an IdM user | LDAP federation bind or search base wrong, or the account is disabled in IdM | Keycloak server log; `ipa user-show`; test the bind DN with `ldapsearch` |
| MFA bypassed for some logins | A client has direct access grants (password grant) enabled | `kcadm.sh get clients -r example --fields clientId,directAccessGrantsEnabled` |

## Oneliners

```sh
# Last successful Kerberos authentication per user
ipa user-find --all --raw | awk '/^ *uid:/ {u=$2} /^ *krbLastSuccessfulAuth:/ {print u, $2}'

# Every member of a group, flattened
ipa group-show ops --all --raw | grep -E '^ *member(user|group):'

# Hosts that exist but were never enrolled (no keytab)
ipa host-find | awk '/Host name:/ {h=$3} /Keytab: False/ {print h}'

# Tracked certificates with their expiry and status
getcert list | grep -E 'Request ID|status:|expires:|certificate:'

# All HBAC rules and who they apply to
ipa hbacrule-find --all --raw | grep -E '^ *(cn|memberuser|memberhost|memberservice):'

# Test access for a user before telling them it works
ipa hbactest --user=alice --host=web-01.example.test --service=sshd

# Replication agreements and their last update
ipa-replica-manage list -v ipa-01.example.test | grep -E 'last update|status'

# Confirm the DNS SRV records a client will use to find servers
dig +short SRV _ldap._tcp.example.test _kerberos._tcp.example.test

# Force a full client refresh after policy changes
sss_cache -E && systemctl restart sssd && id alice

# Users whose password expires in the next 14 days
ipa user-find --all --raw | awk '/^ *uid:/ {u=$2} /^ *krbPasswordExpiration:/ {print $2, u}' | awk -v d="$(date -u -d '+14 days' +%Y%m%d)" '$1 < d""000000Z'

# Disabled but not yet deleted accounts (offboarding follow-up)
ipa user-find --disabled --raw | grep -E '^ *uid:'

# Accounts that have never logged in
ipa user-find --all --raw | awk '/^ *uid:/ {u=$2; s=0} /^ *krbLastSuccessfulAuth:/ {s=1} /^$/ {if (u && !s) print u; u=""}'

# Every host and whether it is enrolled, one per line
ipa host-find --raw | awk '/^ *fqdn:/ {h=$2} /^ *has_keytab: True/ {print h, "enrolled"} /^ *has_keytab: False/ {print h, "NOT enrolled"}'

# Hosts that have not contacted the domain recently (stale enrolments)
ipa host-find --all --raw | awk '/^ *fqdn:/ {h=$2} /^ *krbLastSuccessfulAuth:/ {print $2, h}' | sort

# Tracked certificates expiring within 30 days on this host
getcert list | awk '/Request ID/ {id=$0} /expires:/ {print id, $0}' | grep -v "$(date -d '+30 days' +%Y-%m-%d)"

# Which principals a keytab holds and their key versions
klist -kt /etc/krb5.keytab | awk 'NR>3 {print $4, $1}' | sort -u

# Request a fresh service ticket to confirm an SPN resolves and the keytab is current
for s in HTTP ldap host; do printf '%-6s ' "$s"; kvno "$s/$(hostname -f)" 2>&1 | tail -1; done

# All group memberships for one user, resolved through SSSD (indirect groups included)
id -Gn alice | tr ' ' '\n' | sort

# HBAC rules that apply to a given host group
ipa hbacrule-find --all --raw | awk '/^ *cn:/ {r=$2} /web-servers/ {print r}'

# Members of the admins group (the accounts to watch most closely)
ipa group-show admins --all --raw | grep -E '^ *member' | sed 's/^ *//'

# Confirm every server is healthy in one pass
for s in ipa-01 ipa-02; do echo "== $s =="; ssh "$s" ipa-healthcheck --failures-only 2>/dev/null || echo 'healthcheck reported failures'; done

# Kerberos ticket lifetime remaining, in minutes
klist | awk '/krbtgt/ {print $3, $4}' | while read -r d t; do echo $(( ( $(date -d "$d $t" +%s) - $(date +%s) ) / 60 )) min left; done

# Decode a Keycloak access token's claims without verifying
jq -R 'split(".")[1] | @base64d | fromjson | {sub, preferred_username, exp, realm_access}' <<< "$ACCESS_TOKEN"

# Keycloak: list clients that still allow the password grant (an MFA gap)
kcadm.sh get clients -r example --fields clientId,directAccessGrantsEnabled | jq -r '.[] | select(.directAccessGrantsEnabled) | .clientId'

# Keycloak: users who have not set OTP but are in an MFA-required group
kcadm.sh get users -r example -q briefRepresentation=true --fields id,username | jq -r '.[] | [.id, .username] | @tsv'

# Verify the SRV records a joining client depends on, both protocols
for r in _ldap._tcp _kerberos._tcp _kerberos._udp; do printf '%-16s ' "$r"; dig +short SRV "$r.example.test" | head -1; done

# Confirm client clock is within Kerberos tolerance of a server
offset=$(chronyc tracking | awk '/Last offset/ {print $4}'); echo "offset ${offset}s (must be under 300)"
```

## Scripts

Offboarding: disable a user, remove them from privileged groups, kill live tickets by expiring the account, and print what was done for the audit trail. Disables rather than deletes so file ownership and audit history survive. Run on an IdM server with a fresh `kinit admin`.

```sh
#!/usr/bin/env bash
# usage: offboard.sh USERNAME   (disables the account and strips privileged group membership)
set -euo pipefail
user=${1:?username required}
klist -s || { echo 'no valid ticket; run kinit admin first' >&2; exit 1; }
ipa user-show "$user" >/dev/null || { echo "no such user: $user" >&2; exit 1; }

printf '== before ==\n'
ipa user-show "$user" --all --raw | grep -E '^ *(uid|nsaccountlock|memberof_group):' || true

# Remove from privileged groups; ignore "not a member" errors
for g in admins ops platform-admins helpdesk; do
  ipa group-remove-member "$g" --users="$user" 2>/dev/null && printf 'removed from %s\n' "$g" || true
done

ipa user-disable "$user"
# Expire the password so cached credentials and any keytab-less access stop working
ipa user-mod "$user" --setattr=krbPasswordExpiration=19700101000000Z >/dev/null

printf '\n== after ==\n'
ipa user-show "$user" --all --raw | grep -E '^ *(uid|nsaccountlock|memberof_group):' || true
printf '\nDisabled %s at %s. Object preserved; delete with `ipa user-del --preserve` only after confirming no host references the UID.\n' "$user" "$(date -u +%FT%TZ)"
```

Certificate expiry report across the fleet: asks each enrolled host for its certmonger-tracked certificates and flags any inside the warning window, so a stalled renewal is caught before an outage. Read-only; uses SSH.

```sh
#!/usr/bin/env bash
# usage: cert-report.sh hosts.txt [WARN_DAYS]   (one host per line; needs SSH and getcert on each)
set -euo pipefail
hosts=${1:?hosts file}; warn=${2:-21}
rc=0
while IFS= read -r h; do
  [[ $h =~ ^[[:space:]]*(#|$) ]] && continue
  # getcert prints "expires: 2026-12-01 ..." lines; compute days for each
  out=$(ssh -o ConnectTimeout=10 -o BatchMode=yes "$h" 'getcert list 2>/dev/null | grep -E "status:|expires:|certificate:"' 2>/dev/null) \
    || { printf 'FAIL %-30s unreachable\n' "$h"; rc=1; continue; }
  awk -v host="$h" -v warn="$warn" '
    /status:/   { status=$2 }
    /expires:/  { exp=$2 " " $3 }
    /certificate:/ {
      cmd="date -d \"" exp "\" +%s"; cmd | getline e; close(cmd)
      days=int((e - systime())/86400)
      if (status != "MONITORING" || days < warn)
        printf "%-4s %-30s %4d days  %s\n", (days<warn?"WARN":"ok"), host, days, status
    }' <<< "$out" || true
  grep -q 'WARN' <<< "$out" && rc=1 || true
done < "$hosts"
exit "$rc"
```

Bulk user import from a CSV (`uid,first,last,email,group`), idempotent: existing users are updated and re-added to the group rather than failing the run. Creates accounts, so review the CSV first.

```sh
#!/usr/bin/env bash
# usage: import-users.sh users.csv   (header: uid,first,last,email,group)
set -euo pipefail
csv=${1:?csv file}
klist -s || { echo 'run kinit admin first' >&2; exit 1; }

tail -n +2 "$csv" | while IFS=, read -r uid first last email group; do
  [[ -n $uid ]] || continue
  if ipa user-show "$uid" >/dev/null 2>&1; then
    ipa user-mod "$uid" --first="$first" --last="$last" --email="$email" >/dev/null
    printf 'updated %s\n' "$uid"
  else
    ipa user-add "$uid" --first="$first" --last="$last" --email="$email" --random >/dev/null
    printf 'created %s (random password set; user must reset)\n' "$uid"
  fi
  if [[ -n ${group:-} ]]; then
    ipa group-show "$group" >/dev/null 2>&1 || ipa group-add "$group" --desc="imported" >/dev/null
    ipa group-add-member "$group" --users="$uid" >/dev/null 2>&1 || true
  fi
done
```

## Further reading

- [Red Hat Identity Management documentation](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/managing_idm_users_groups_hosts_and_access_control_rules/index): users, groups, hosts, HBAC and sudo.
- [FreeIPA documentation](https://freeipa.readthedocs.io/en/latest/): architecture, replication and troubleshooting.
- [MIT Kerberos user commands](https://web.mit.edu/kerberos/krb5-latest/doc/user/user_commands/index.html): `kinit`, `klist`, `kvno` and ccache types.
- [SSSD documentation](https://sssd.io/docs/introduction.html) and `man sssd.conf`, `man sssctl`.
- [Keycloak Server Administration Guide](https://www.keycloak.org/docs/latest/server_admin/) and [securing applications](https://www.keycloak.org/docs/latest/securing_apps/) for OIDC clients and flows.