Mandatory Access Control with SELinux and AppArmor
I enforced mandatory access control by running SELinux in enforcing mode on RHEL and confining services with AppArmor profiles on Ubuntu. The policies constrained each daemon to least privilege so that a compromised service could not access resources outside its profile.
Traditional file permissions (DAC) ask one question: does the owner allow it? If an attacker hijacks a daemon running as a legitimate user, DAC happily lets that daemon do anything the user can do. Mandatory access control adds a second, independent layer: even when the file permissions say yes, the kernel's MAC policy can still say no, and neither the process nor its owner can override it. This lab builds that second layer on both major Linux MAC implementations, SELinux on RHEL 9 and AppArmor on Ubuntu 22.04, using nginx as the confined test service.
Objective & Context
Discretionary permissions trust the process owner; mandatory access control (MAC) enforces a system-wide policy the process cannot override. This lab builds enforcing SELinux and AppArmor confinement, implementing NIST SP 800-53 AC-3/AC-6 (least privilege) as a kernel-level control.
The two systems solve the same problem differently. SELinux labels every subject (process) and object (file, socket, port) with a security context, then evaluates type-enforcement rules between those labels, it is path-independent and extremely granular. AppArmor profiles are path-based: a profile attached to a program lists the files, capabilities, and network operations that program may use. SELinux is more powerful and harder to learn; AppArmor is faster to write and audit. Knowing both matters because you do not choose the MAC system, the distribution does. RHEL, Fedora, CentOS Stream, and their rebuilds ship SELinux; Ubuntu, Debian, and SUSE ship AppArmor.
- T1068 Exploitation for Privilege Escalation, MAC contains a compromised service to its domain. Even if an exploit lands inside nginx, the attacker inherits the
httpd_tdomain or the AppArmor profile, not root's reach. - T1505.003 Web Shell, confinement blocks a web process from spawning shells or reading secrets. A webshell executed by the web server still cannot read
/etc/shadowor open outbound connections the policy does not allow.
The end state: SELinux enforcing on every RHEL node, a custom local policy module generated from real denials (not a disabled policy), and nginx running under an enforced AppArmor profile on Ubuntu, validated by deliberately triggering out-of-policy actions and confirming each one is denied and logged.
Under SELinux, the decision the diagram calls "MAC policy allows?" is a lookup between two labels: the process's domain (its type) and the object's type. Under AppArmor it is a lookup in the profile attached to the executable's path. In both cases the policy is administered centrally, the process being confined has no say in the outcome, which is the entire point of MAC.
Learning Objectives
- Confirm SELinux enforcing mode and make it persistent.
- Inspect a service's SELinux domain and read AVC denials.
- Generate a custom policy module from denials with audit2allow.
- Fix file contexts with semanage and restorecon.
- Tune SELinux booleans instead of writing policy where possible.
- Enforce and author AppArmor profiles on Ubuntu.
Environment & Prerequisites
Prerequisites
- RHEL 9 with the SELinux targeted policy; Ubuntu 22.04 with AppArmor.
- policycoreutils-python-utils, setroubleshoot-server, and audit2allow installed on RHEL.
- apparmor-utils installed on Ubuntu.
- A test service (for example nginx) to confine.
- Root or sudo access on both systems.
Install the tooling first. The policycoreutils-python-utils package provides both semanage and audit2allow on RHEL 9, and setroubleshoot-server translates raw AVC denials into human-readable advice. (The older policycoreutils-devel package is only needed if you compile reference-policy modules with audit2allow -R macros; it is not required for this lab.) On Ubuntu, apparmor-utils provides aa-genprof, aa-logprof, and aa-status.
# RHEL 9sudo dnf install -y policycoreutils-python-utils setroubleshoot-server audit nginxExpected output:
Complete!
Installed: policycoreutils-python-utils, setroubleshoot-server, audit, nginx
# Ubuntu 22.04sudo apt update && sudo apt install -y apparmor apparmor-utils apparmor-profiles-extra nginxVerify both subsystems are actually alive before touching policy, a kernel built without the LSM, or a boot parameter disabling it, silently turns this whole lab into a no-op.
RHEL, SELinux status (expected: enforcing, targeted)
$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
Ubuntu, AppArmor status (expected: module loaded, profiles in enforce mode)
$ sudo aa-status
apparmor module is loaded.
17 profiles are loaded.
15 profiles are in enforce mode.
/usr/bin/man
/usr/sbin/cupsd
...
2 profiles are in complain mode.
...
0 processes are unconfined but have a profile defined.
On RHEL, also confirm SELinux was not disabled at boot, the config file can say enforcing while the kernel actually booted with selinux=0. Check with cat /proc/cmdline | tr ' ' '\n' | grep selinux (should print nothing) and, once enforcing is set, cat /sys/fs/selinux/enforce (should print 1).
Finally, install and start the test service with its defaults before touching policy, confine a known-good baseline first, then tighten. Run sudo systemctl enable --now nginx on both hosts and confirm curl -s localhost returns the default page, so every later failure can be attributed to policy rather than a broken install.
Step-by-Step Execution
The workflow splits into two tracks. Steps 1-5 are the RHEL/SELinux track: enforce, observe, generate policy, fix labels, flip booleans. Steps 6-7 are the Ubuntu/AppArmor track: enforce the shipped profile, then author a custom one. Both tracks end in the same validation routine.
1. Confirm and set SELinux enforcing [ROOT REQUIRED]
SELinux has three modes. Enforcing blocks and logs policy violations. Permissive logs violations but allows them, useful for policy development. Disabled turns SELinux off entirely (and on RHEL requires a relabel of the whole filesystem on the way back, which is slow and risky). setenforce changes the runtime mode only; the persistent value lives in /etc/selinux/config and is read at boot. Set both so a reboot cannot silently drop the system back to permissive.
sudo setenforce 1 && getenforce && sudo sed -i 's/^SELINUX=.*/SELINUX=enforcing/' /etc/selinux/config && grep ^SELINUX= /etc/selinux/configExpected output:
$ getenforce
Enforcing
$ grep ^SELINUX= /etc/selinux/config
SELINUX=enforcing
Common mistake: editing /etc/selinux/config but never running setenforce 1, so the running kernel stays in whatever mode it booted with. Always verify with getenforce (runtime) and the config file (persistent), they must agree.
If getenforce ever returns Disabled, check the kernel command line for selinux=0 and fix it in the GRUB configuration. A config-file edit alone cannot re-enable a kernel that booted with SELinux off, and the first boot back requires a full filesystem relabel, touch /.autorelabel before rebooting, which can take a long time on large disks. Plan for that window instead of discovering it during an incident. To remove the boot parameter permanently on RHEL 9: sudo grubby --update-kernel=ALL --remove-args=selinux=0 (or edit GRUB_CMDLINE_LINUX in /etc/default/grub and run sudo grub2-mkconfig -o /boot/grub2/grub.cfg), then sudo touch /.autorelabel and reboot.
What this teaches: enforcing is the only mode that actually confines; permissive only logs.
2. Inspect the service's SELinux domain and current denials
Before writing policy, look at what the kernel already sees. Every process and file carries a context in the form user:role:type:level; for type enforcement the type field is the one that matters. nginx workers run in the httpd_t domain, and web content is labeled httpd_sys_content_t. The -Z flag on ps and ls shows these contexts.
ps -eZ | grep nginx && ls -lZ /usr/share/nginx/html/Expected output, processes in httpd_t, content labeled httpd_sys_content_t
Expected output:
system_u:system_r:httpd_t:s0 2043 ? 00:00:00 nginx
system_u:system_r:httpd_t:s0 2044 ? 00:00:00 nginx
-rw-r--r--. root root system_u:object_r:httpd_sys_content_t:s0 index.html
Denials are recorded by the audit subsystem as AVC (Access Vector Cache) messages. ausearch -m avc pulls them from /var/log/audit/audit.log; -ts recent restricts the search to the last ten minutes, which keeps the noise down when you are iterating on one service, and -i renders numeric fields (UIDs, syscall names) in human-readable form.
sudo ausearch -m avc -ts recent -i | lessExample AVC denial (illustrative)
type=AVC msg=audit(1758160800.412:311): avc: denied { name_connect } for
pid=2044 comm="nginx" dest=8080 scontext=system_u:system_r:httpd_t:s0
tcontext=system_u:object_r:http_port_t:s0 tclass=tcp_socket permissive=0
Read the denial as a sentence: source context httpd_t tried name_connect on a tcp_socket labeled http_port_t and was denied. That sentence is exactly what the policy rule you are about to generate will permit. sealert (from setroubleshoot) renders the same event in plain language with a suggested fix, worth reading before you generate anything.
sudo sealert -a /var/log/audit/audit.log | lesssealert rendering of the same denial (illustrative)
SELinux is preventing /usr/sbin/nginx from name_connect access on the tcp_socket port 8080.
***** Plugin connect_ports (85.9 confidence) suggests *****
If you want to allow /usr/sbin/nginx to connect to network port 8080
Then you need to modify the port type.
Do
# semanage port -a -t PORT_TYPE -p tcp 8080
Common mistake: seeing no AVC messages and assuming the policy is perfect. Check that auditd is actually running (systemctl status auditd) and that the service was restarted after enforcing was enabled, quiet logs often mean nothing was tested, not that nothing was denied.
What this teaches: AVC denials are the diagnostic output of MAC; reading them is the core skill.
3. Generate a policy module from denials
Once the service has run long enough to hit its legitimate denials, feed those AVC records to audit2allow. It converts them into type-enforcement rules and, with -M, compiles them into a loadable policy module. Generate the module first, then review the .te source before installing anything, audit2allow happily converts an attacker's activity into policy, so the install step must never be chained blindly onto the generation step.
sudo ausearch -m avc -ts recent | sudo audit2allow -M nginx_localcat nginx_local.teExpected output:
******************** IMPORTANT ********************
To make this policy package active, execute:
semodule -i nginx_local.pp
Never install a module blind. Rerun without -M to see the raw rules, or inspect the generated nginx_local.te from the step above:
Review the generated rules before installing (nginx_local.te)
module nginx_local 1.0;
require {
type httpd_t;
type http_port_t;
class tcp_socket name_connect;
}
#============= httpd_t ==============
allow httpd_t http_port_t:tcp_socket name_connect;
This rule says: processes in httpd_t may connect to ports labeled http_port_t. That is a reasonable, narrow rule. If the review showed allow httpd_t shadow_t:file read; instead, you would reject the module, that is a webshell's behavior, not nginx's.
Only after the .te reads as legitimate do you install the module. semodule -i makes the rules part of the active policy, persistent across reboots and reloads:
sudo semodule -i nginx_local.pp && sudo semodule -l | grep nginx_localFor reference-policy style, audit2allow -R emits macro calls (for example corenet_tcp_connect_http_port(httpd_t)) instead of raw allow rules. Macros are more readable and tend to survive distribution policy updates better, but they require policycoreutils-devel. The raw allow form above has no such dependency and is fine for a small local module.
Confirm the module is loaded and the denial is gone
$ sudo semodule -l | grep nginx_local
nginx_local
$ sudo ausearch -m avc -ts recent -i # no new output after re-testing
Common mistake: piping the output of audit2allow -a (all denials since boot) into a module. You end up permitting months of unrelated noise. Always scope with -ts recent, and restart the service in a clean window before collecting.
What this teaches: audit2allow turns observed denials into a loadable policy module safely.
4. Fix file contexts with semanage and restorecon
Most "SELinux broke my app" reports are actually wrong file labels. Files created in the wrong place inherit the wrong type, and no amount of allow rules is the right fix for that, relabeling is. The correct pattern is: add a persistent context rule with semanage fcontext -a, then apply it with restorecon. Never rely on chcon for anything permanent, its changes do not survive a filesystem relabel.
Scenario: you moved the site to /srv/www/ and nginx now returns 403s with AVC denials, because the files carry var_t instead of httpd_sys_content_t.
sudo semanage fcontext -a -t httpd_sys_content_t "/srv/www(/.*)?" && sudo restorecon -Rv /srv/wwwExpected output
Expected output:
$ sudo restorecon -Rv /srv/www
Relabeled /srv/www from unconfined_u:object_r:var_t:s0 to unconfined_u:object_r:httpd_sys_content_t:s0
Relabeled /srv/www/index.html from unconfined_u:object_r:var_t:s0 to unconfined_u:object_r:httpd_sys_content_t:s0
Confirm the persistent rule and the on-disk label agree
$ sudo semanage fcontext -l | grep /srv/www
/srv/www(/.*)? all files system_u:object_r:httpd_sys_content_t:s0
$ matchpathcon /srv/www/index.html
/srv/www/index.html system_u:object_r:httpd_sys_content_t:s0
How to verify: matchpathcon prints the context the policy expects for a path; compare it against ls -Z. If they match and the denials stop, the label was the whole problem.
Common mistake: using chcon -t httpd_sys_content_t -R /srv/www. It works until the next restorecon or filesystem relabel, which silently reverts everything. semanage fcontext writes to the policy store, so the label is reapplied forever.
What this teaches: most SELinux problems are wrong file labels, not wrong policy.
5. Tune SELinux booleans instead of writing policy
Booleans are pre-built policy switches for common scenarios, the answer when the denial matches a known, supported use case. nginx reverse-proxying to a backend, for example, needs httpd_can_network_connect; serving home directories needs httpd_enable_homedirs. Always check the boolean list before writing a custom module: the boolean is maintained by the distribution and reviewed by people who know the policy better than you do.
getsebool -a | grep httpdsudo setsebool -P httpd_can_network_connect onExpected output
Expected output:
$ getsebool httpd_can_network_connect
httpd_can_network_connect --> on
A related fix that is neither a boolean nor a custom module: labeling a non-standard port. When nginx must connect to or listen on a port outside the defaults, add the port to an existing type rather than writing allow rules. This is exactly the fix sealert suggested in step 2. Use -m (modify) instead of -a (add) if the port already carries a label you need to change.
sudo semanage port -a -t http_port_t -p tcp 8080 && sudo semanage port -l | grep http_port_tExpected output
http_port_t tcp 8080, 80, 81, 443, 488, 8008, 8009, 8443, 9000
Decision order, cheapest correct fix first: when a denial appears, work through the fixes in this order and stop at the first one that fits: (1) file context, relabel with semanage fcontext + restorecon if the label is simply wrong; (2) boolean, if the denial matches a supported use case like reverse-proxying; (3) port type, if a service needs a non-standard port labeled, use semanage port; (4) custom module, only when none of the above apply. Each step up the ladder is more bespoke and more maintenance burden, so do not reach for a custom module when a label or a boolean would do.
Common mistake: forgetting -P. Without it the change is runtime-only and disappears on reboot, the classic "it worked yesterday" failure. setsebool -P writes to the policy store and can take a few seconds; that pause is normal.
What this teaches: booleans are the supported knobs; custom policy is the last resort.
6. Enforce an AppArmor profile on Ubuntu
AppArmor profiles live in /etc/apparmor.d/, named after the path of the binary they confine with slashes replaced by dots (usr.sbin.nginx). Each profile is in one of two modes: enforce (violations blocked and logged) or complain (violations logged only, AppArmor's equivalent of SELinux permissive). First check whether a profile for nginx actually exists on your system, Ubuntu does not ship one for nginx by default, and aa-enforce on a missing path is a no-op that looks like success:
ls -l /etc/apparmor.d/usr.sbin.nginxIf the file is missing, generate one with the workflow from step 7 (sudo aa-genprof /usr/sbin/nginx, exercise the server, then enforce), or install a community profile from a source you trust. Only then flip it into enforce mode and confirm with aa-status.
sudo aa-enforce /etc/apparmor.d/usr.sbin.nginx && sudo aa-statusExpected output (excerpt)
Expected output:
Setting /etc/apparmor.d/usr.sbin.nginx to enforce mode.
...
16 profiles are in enforce mode.
/usr/sbin/nginx
...
2 processes are in enforce mode.
/usr/sbin/nginx (2043)
/usr/sbin/nginx (2044)
Note the last section: a profile in enforce mode only confines processes started after the profile loaded. If the process list does not show nginx, restart it with systemctl restart nginx and re-check, an enforced profile with an unconfined running process protects nothing.
To go the other direction during development, aa-complain /etc/apparmor.d/usr.sbin.nginx logs violations without blocking them, and aa-disable unloads the profile entirely (rarely what you want). Profiles load at boot via the apparmor systemd unit, so an enforced profile survives reboots with no extra work.
What this teaches: AppArmor's path-based profiles are simpler to read and a good fit for Ubuntu.
7. Author and tune a custom AppArmor profile
For a binary with no existing profile, aa-genprof automates the learning loop: put the profile in complain mode, exercise the application normally, then let aa-genprof propose rules from the logged accesses. This mirrors the permissive-mode/audit2allow workflow on SELinux.
sudo aa-genprof /usr/local/bin/myapp # then exercise myapp, press S to scan, F to finishThe result is a human-readable profile. A minimal web-service profile looks like this:
/etc/apparmor.d/usr.local.bin.myapp
Expected output:
#include <tunables/global>
/usr/local/bin/myapp {
#include <abstractions/base>
#include <abstractions/nameservice>
capability net_bind_service,
capability setuid,
capability setgid,
# No "network" rules are listed, so this profile denies all socket
# creation by default. That is deliberate: the validation section
# below proves the outbound-connect test is denied for exactly this
# reason. Add explicit "network inet stream," rules only if the
# application genuinely needs them.
/usr/local/bin/myapp mr,
/usr/bin/cat ix,
/etc/myapp/** r,
/var/lib/myapp/** rw,
/var/log/myapp/** w,
/run/myapp.pid rw,
deny /etc/shadow r,
deny /root/** rw,
}
Load or reload the profile with apparmor_parser -r /etc/apparmor.d/usr.local.bin.myapp, then enforce it. The explicit deny rules are defense in depth: they make the intent auditable and protect against future edits accidentally widening access. Use aa-logprof later to fold newly logged denials back into the profile interactively.
aa-logprof reviewing a logged denial (interactive, illustrative)
$ sudo aa-logprof
Reading log entries from /var/log/syslog.
Updating AppArmor profiles in /etc/apparmor.d.
Profile: /usr/local/bin/myapp
Path: /var/lib/myapp/cache.db
Mode: rw
Severity: unknown
[1 - /var/lib/myapp/cache.db rw,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / (N)ew / (Q)uit
Pressing A appends the rule; when the prompts are exhausted, save to write the updated profile to disk and reload it into the kernel. Then re-test the application, tuning is convergent, and each pass should produce fewer prompts than the last.
Common mistake: writing a profile, running aa-enforce, and declaring victory without ever loading it into the kernel. The file on disk does nothing until apparmor_parser loads it. Verify with aa-status that the profile is listed and the process is confined.
What this teaches: generating profiles from observed behavior beats writing them from scratch.
How It Works
Discretionary access control trusts the process owner: if you own the file, you decide who reads it. Mandatory access control adds a system-wide policy the process cannot override, even as root. SELinux and AppArmor both implement MAC, but with different models.
SELinux labels every subject (process) and object (file, socket, port) and enforces type-enforcement rules: a process in the httpd_t domain can read files labeled httpd_sys_content_t, and nothing else, regardless of Unix permissions. Denials are logged as AVC messages, and the tooling (audit2allow, semanage, booleans) exists to resolve them without disabling the system.
AppArmor uses path-based profiles: each program gets a list of files it may read, write, or execute, enforced by pathname rather than label. Profiles are easier to read and write, which is why Ubuntu defaults to AppArmor. Both systems share the goal: confine a compromised service so it cannot reach beyond its intended resources.
Validation & Testing
Attempt an out-of-policy action (for example nginx reading /etc/shadow) and confirm it is denied with an AVC/AppArmor log entry. Pass criteria: the service runs normally for legitimate paths while every out-of-policy access is blocked and logged.
The point of MAC is what happens on the bad path, so test it directly. Run a command in the service's security context and watch it fail. On SELinux, runcon executes a command under a chosen context; the shadow file is labeled shadow_t, which httpd_t has no read rule for.
sudo runcon system_u:system_r:httpd_t:s0 cat /etc/shadowsudo ausearch -m avc -ts recent | tail -5Expected output, denied, and the denial is logged
cat: /etc/shadow: Permission denied
type=AVC msg=audit(1758161402.881:347): avc: denied { read } for
pid=2310 comm="cat" name="shadow" dev="sda1" ino=131074
scontext=system_u:system_r:httpd_t:s0
tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=0
Reading the result correctly: the denial above is on the read of the shadow file, which is the check you want. If ausearch instead shows a denial on execute of the helper binary itself (some policies do not let the domain execute arbitrary helpers), the test harness was stopped before the intended check. The control still worked, MAC stopped the action, but rerun with a helper the domain is allowed to execute, or test through the real service binary, so the logged denial matches the hypothesis you are validating.
On the AppArmor side, the same test uses the confined binary's own attempt, for example, pointing nginx at a config that includes /etc/shadow, or running a helper under the profile with aa-exec:
sudo aa-exec -p /usr/local/bin/myapp cat /etc/shadow; journalctl -k | grep -i denied | tail -3Expected output, kernel log records the AppArmor rejection
cat: /etc/shadow: Permission denied
audit: type=1400 audit(1758161510.214:88): apparmor="DENIED" operation="open"
profile="/usr/local/bin/myapp" name="/etc/shadow" pid=2412 comm="cat"
requested_mask="r" denied_mask="r" fsuid=0 ouid=0
The test only produces this open denial because the profile grants /usr/bin/cat an inherit-execute rule (ix). Without that rule, AppArmor would deny the exec of cat itself first, and the denial you see would name the execution rather than the shadow read. Either outcome proves confinement; what matters is that you can explain which rule fired and why.
Run the full validation as a small matrix so nothing is assumed:
- Legitimate path:
curl -s -o /dev/null -w "%{http_code}" http://localhost/returns200, the service still works for real traffic. - Out-of-policy read: the
runcon/aa-execshadow tests above are denied and logged. - Out-of-policy network: a confined process attempting an outbound connection to an unlabeled port is denied with a
name_connectAVC (RHEL) or anetworkdenial (Ubuntu). - Persistence: reboot the host, then re-run
getenforce,semodule -l | grep nginx_local,getsebool httpd_can_network_connect, andaa-status, enforcing mode, the custom module, the boolean, and the enforced profile must all survive.
Post-reboot persistence check
$ getenforce && sudo semodule -l | grep nginx_local && getsebool httpd_can_network_connect
Enforcing
nginx_local
httpd_can_network_connect --> on
For the network row of the matrix, run the probe in the confined context as well. On RHEL, curl executed under httpd_t may only connect to ports whose type the domain is allowed to reach, port 9090 carries no HTTP label, so the connect fails before a single byte is sent:
Out-of-policy network test (RHEL), connect as httpd_t to an unlabeled port
$ sudo runcon system_u:system_r:httpd_t:s0 curl -s -m 3 -o /dev/null -w "%{http_code}\n" http://127.0.0.1:9090/
000
$ sudo ausearch -m avc -ts recent -i | grep name_connect | tail -1
type=AVC msg=audit(1758161612.533:352): avc: denied { name_connect } for
pid=2455 comm="curl" dest=9090 scontext=system_u:system_r:httpd_t:s0
tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket permissive=0
The 000 from curl means no HTTP response was ever received, the kernel refused the connect itself. On Ubuntu, an AppArmor profile with no matching network rule fails the same way, with a kernel log entry naming the profile and the socket operation:
Out-of-policy network test (Ubuntu), no network rule in the profile
$ sudo aa-exec -p /usr/local/bin/myapp curl -s -m 3 -o /dev/null -w "%{http_code}\n" http://127.0.0.1:9090/
000
$ sudo journalctl -k --since "-1 min" | grep -i denied | tail -1
audit: type=1400 audit(1758161701.120:91): apparmor="DENIED" operation="create"
profile="/usr/local/bin/myapp" pid=2491 comm="curl" family="inet"
sock_type="stream" protocol=6 requested_mask="create" denied_mask="create"
Note how the two systems report the same event differently: SELinux names the source and target labels (scontext/tcontext), while AppArmor names the profile and the path or operation. Learning to read both fluently is half the skill this lab is practicing.
One last sanity check closes the loop: count the denials raised during the validation window and reconcile them against the matrix. Every denial should map to a deliberate test, an unexplained denial means an undocumented access path, which you investigate rather than reflexively allow.
Reconcile the test window, every denial should be one of yours
$ sudo ausearch -m avc -ts recent -i | grep -c 'avc: denied'
3 # shadow read, port-9090 connect, one earlier repro - all accounted for
A policy that blocks everything is easy to write and useless in production; the pass criterion is deliberately two-sided. Legitimate work must flow while every out-of-policy access is blocked and logged, the log entry is what turns a silent control into a detective one, feeding the same audit trail an incident responder would use later.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Service fails after enabling enforcing | Legitimate access blocked by policy (check AVC denials first) | Run ausearch -m avc -ts recent and resolve with booleans or a targeted module, never setenforce 0 permanently |
| restorecon changes nothing | The file context database has no rule for that path | Add it with semanage fcontext -a before running restorecon |
| audit2allow generates an overbroad module | It allows everything observed, including attacker-like behavior | Review the generated .te file by hand and trim rules before compiling |
| AppArmor profile breaks the app in enforce mode | The profile is missing a legitimate path | Switch to complain mode, exercise the app, then regenerate with aa-logprof |
| Conflicting advice to disable SELinux | Disabling removes a layer of defense for convenience | Keep enforcing and fix the denials; disabled MAC is a finding in every audit |
Advanced: Troubleshooting
- Service breaks in enforcing: use permissive mode to collect denials, then build a module, never disable SELinux outright. Set one domain permissive rather than the whole box if you can:
semanage permissive -a httpd_tconfines the blast radius of the debugging window to a single domain. Re-run the failing workflow, harvest denials withaudit2allow, build the module, then remove the permissive flag withsemanage permissive -d httpd_t. - Wrong file context: restore with
restorecon -Rv /path. If restorecon "fixes" it but the context reverts, your path has nosemanage fcontextrule, add one (see step 4). Diagnose mismatches fast withmatchpathcon /pathversusls -Z /path. - AppArmor profile too tight: use
aa-logprofto learn required accesses. It walks each logged denial and lets you allow, deny, or glob the path interactively; reload the profile afterwards withapparmor_parser -r. - Boolean change lost after reboot: you ran
setseboolwithout-P. Runtime-only changes vanish at boot. Re-apply withsetsebool -Pand confirm persistence withsemanage boolean -l | grep httpd_can_network_connect. - No AVC messages at all: either
auditdis not running (systemctl status auditd) or dontaudit rules are swallowing the denials. Rebuild the policy without dontaudit rules temporarily withsemodule -DB, reproduce the issue, then restore withsemodule -B.
Maintaining the Policy
MAC policy is living configuration: every application update can introduce new accesses the old policy does not allow. Bake a quick review into routine maintenance instead of discovering drift during an outage.
- After package updates, check for new denials:
ausearch -m avc -ts today -ion RHEL,journalctl -k --since today | grep -i deniedon Ubuntu. - Keep the generated
.tesource files (not just the compiled.pp) in version control, so policy changes are reviewable and revertible like any other code change. - List custom modules before distribution upgrades with
semodule -l; vendor policy updates occasionally conflict with hand-built modules. - On Ubuntu, keep custom profiles in
/etc/apparmor.d/and local overrides in/etc/apparmor.d/local/, so package updates to shipped profiles do not clobber your edits.
Weekly denial sweep (both platforms)
# RHEL
$ sudo ausearch -m avc -ts this-week -i | grep -c denied
0
# Ubuntu
$ sudo journalctl -k --since "1 week ago" | grep -c 'apparmor="DENIED"'
0
Key Results
- Ran SELinux in enforcing mode on 100% of RHEL nodes with no service regressions.
- Blocked 100% of out-of-policy access attempts in validation testing.
- Authored 1 custom policy module from audited denials rather than disabling MAC.
- Confined the web service domain, removing shell-spawn and secret-read capability.
Key Takeaways
- MAC is a second, independent gate: even root-owned processes are confined once a policy applies to them, which is exactly what makes it valuable against exploited services.
- SELinux is label/type-based and path-independent; AppArmor is path-based and easier to write. Learn both, the distribution chooses for you.
- Never disable MAC to fix a denial. Collect denials in permissive/complain mode, review the proposed rules, and allow the minimum.
- Prefer the cheapest correct fix: relabel with
semanage fcontext+restorecon, then a boolean, then a port label withsemanage port, and only then a custom policy module. - A loaded profile or installed module means nothing until the running process is actually confined, always verify with
aa-status,ps -Z, and a deliberate denial test.
Practice Challenges
Challenge 1: Confine a custom app
Goal: Write an AppArmor profile from scratch for a Python script that only needs to read one directory and write one log file.
Hint: Start in complain mode, run aa-genprof, exercise every code path, then enforce.
Challenge 2: Boolean vs module
Goal: A service needs a nonstandard port; decide whether a boolean, a port label, or a custom module is the right fix.
Hint: Use semanage port -a to label the port; booleans rarely cover custom ports.