no way to log network connections and traffic of applications (processes)

the log viewer only displays system logs, it's not the software recording those

It isn't the syslogd's responsibility to do a lookup of the pid, it is the application and the application is the one specifying the pid anyway.

Based on the example given, it sounds like what is desired is for the firewall to log the pid of the application that generated the request. This is the kernel's responsibility via netfilter, and netfilter is controlled via iptables (and up above maybe another tool like ufw). However, iptables does not support this (see 'man iptables' for more information). It used to have the '--cmd-owner' option, but this was removed long ago because it was deemed unfixably broken.

@Jamie Strandboge, is there anything that can be done to get this capability in Ubuntu? Perhaps some other solution can be created or put on the planning board? A lot of people want to have a log of the outgoing internet connections of applications. Unless users are willing to sit and stare at the monitor while connections occur every second they are on their computer, users have no way of learning what apps are making outgoing connections on their computers. There's got to be some way give Ubuntu users this capability.

The capability to log "process names" has been requested by numerous users over the years, here's some links:

Ubuntu Brainstorm: http://brainstorm.ubuntu.com/idea/28288/

Superuser: http://superuser.com/questions/34782/with-linux-iptables-is-it-possible-to-log-the-process-command-name-that-initiat

nethogs and netstat can connect pids to program names, so in theory someone could add this functionality to iptables. You could also setup a cron job to regularly log netstat output for all network connections to a file. For example, the following command ran as root will timestamp and log all network connections every 5 seconds to log.txt:
   # while [ 1 == 1 ]; do date >> log.txt; netstat -pn -A inet --wide >> log.txt; sleep 5 ; done
You can run without root privledge, however process you don't own won't be included. Hope this helps a little.

To be clear, the kernel is doing the logging, not iptables. Iptables configures netfilter, the part of the kernel that does all this.

That said, the pid is not logged by the kernel so there is nothing to cross-reference in /proc (which is all netstat is doing). Eg:
Jan 26 15:36:57 localhost kernel: [21281.600175] [UFW BLOCK] IN= OUT=wlan0 SRC=10.0.0.2 DST=91.189.90.41 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=37870 DF PROTO=TCP SPT=38053 DPT=23 WINDOW=14600 RES=0x00 SYN URGP=0

What Robbie suggests could kinda work, but because it is polling a snapshot in time, it is not really a satisfactory solution for people wanting to continually map outgoing connections to a program name. The superuser.com site has tips on how to write a program that could poll various things in /proc, but this is not this bug. This bug is asking for logging the process name for network packets (the PID could in theory satisfy this, but there is still the problem of the polling interval).

Iptables does have a --log-uid option, but that doesn't get us all the way there:
Jan 26 15:45:54 localhost kernel: [21818.931215] [TEST] IN= OUT=wlan0 SRC=10.0.0.2 DST=91.189.89.88 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=24075 DF PROTO=TCP SPT=52517 DPT=23 WINDOW=14600 RES=0x00 SYN URGP=0 UID=1000 GID=1000

Most of the desire surrounding this sort of logging has to do with application firewalls where you have specific firewall rules based on the application producing the network traffic. There used to be a --cmd-owner option for iptables that would configure the firewall to more of less do what you wanted but the kernel as of 2.6.14 stopped supporting this. The Debian bug report referring to the removal of --cmd-owner is: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=492284

and the kernel commit is: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=34b4a4a624bafe089107966a6c56d2a1aca026d4

The kernel commit states:
"[NETFILTER]: Remove tasklist_lock abuse in ipt{,6}owner

Rip out cmd/sid/pid matching since its unfixable broken and stands in the
way of locking changes to tasklist_lock."

LSMs such as AppArmor could be used to help with application firewalls, but at this time AppArmor network mediation is very coarse-grained. Support is planned for better network mediation-- the first cut will allow specifying network rules by port. After that we would tie in with secmark which will allow us to filter based on the contents of the secmark. Both of these would improve the situation for application firewalls to varying degrees, and a creative complain-mode global AppArmor policy could in theory be used to show which applications are making the outgoing connections.

Must say I agree with Nick Rundy's comments on 2012-01-25 (above).

We do need a way to track what application is initiating network traffic, that can (or is) logged to the syslog.

Can we get this progressed to the kernel and iptables teams as a feature request.

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 820895

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

This report does not need any logs attatched to be complete and confirmed.

"[Bug 820895] Re: Log File Viewer does not log Process Name"
nick doc