Fetching systemd logs from Python 3

(Note: This post is originally from 2018, and is copied to this new site for historical reasons. Opinions, network infrastructure, and other things may have changed since this was first written.)

I like to read my mail server logs from time to time. There’s just something I find fascinating about watching a bunch of people trying to use it as a mail relay and failing. I think it’s some variant of schadenfreude.

But recently I kinda wanted to keep track of the number of times someone tries it in a given day. And it’s difficult to do that if you’re just running journalctl -u postfix all the time. So I’ve decided to put it in the nightly email all the servers are sending to me anyway. Remember The Daily Bullshit Report?

So how does it work?

systemd comes with its own set of python bindings. The module of interest here is systemd.journal. In order to read journal entries, one makes an instance of [systemd.journal.Reader][systemd.journal.Reader]. Then you run the add_match method on that instance to filter messages (in my case, I wanted Postfix logs, so I had jrnl.add_match(_SYSTEMD_UNIT="postfix.service"), and use seek_* functions to move to where you want to start from. (In my case, seek_realtime and a datetime object for “yesterday at 19:00”).

After that, it’s as easy as iterating over the journal reader, and reading the "MESSAGE" key from each object. You can also use the "__REALTIME_TIMESTAMP" key if you need a UNIX timestamp for the events.

The full file would look something like this, sorta:

import datetime
import systemd.journal

today = datetime.datetime.today()
yesterday = today.replace(day=today.day - 1, hour=19)

jrnl = systemd.journal.Reader()
jrnl.add_match(_SYSTEMD_UNIT="postfix.service") # the .service is important
jrnl.seek_realtime(yesterday.timestamp())

for entry in jrnl:
    print(entry["MESSAGE"])

Note: The code I’m actually using involves fishing out IP addresses and things from the log messages, but that’s a bit out of scope for this particular article. For now, I’ll just leave you with this scary regex I wrote, which grabs the first thing it finds encased in square brackets: [^\[]*\[([^\]]*)\]