Fluentd: Warning on parser filter: nested repeat operator '+' and '*' was replaced with '*' in regular expression

May 19, 2021 · 2 min read · coredns fluentd kubernetes  ·
Share on:

I needed to add parser settings for CoreDNS working on my Kubernetes cluster for troubleshooting but I got an error "nested repeat operator '+' and '' was replaced with '' in regular expression." I had no idea why it happened because the additional regular expression was valid. The cause of the error was my mistake so let me share what’s the cause of?

First, I added log setting to the CoreDNS configuration like this.

 1$ kubectl get configmaps -n kube-system coredns -o yaml
 2apiVersion: v1
 3data:
 4  Corefile: |
 5    .:53 {
 6        log
 7
 8        errors
 9        health {
10           lameduck 5s
11        }
12        ready
13     :
14     :

And then, I restarted the pod and got logs like this.

1[INFO] 172.31.57.199:54169 - 1916 "A IN dynamodb.us-west-2.amazonaws.com. udp 50 false 512" NOERROR qr,rd,ra 98 0.000921506s

So, I create a regular expression string and test it on https://fluentular.herokuapp.com/. Then, added the following filter to the Fluentd configuration.

 1<filter **>
 2    @type parser
 3    key_name log
 4    reserve_data true
 5    replace_invalid_sequence true
 6    <parse>
 7        @type multi_format
 8        <pattern>
 9            format regexp
10            expression /^\[(?<severity>\S+)\] (?<remote>\d+\.\d+\.\d+\.\d+):(?<port>\d+) +- (?<id>\d+) +*"(?<type>\S+) +(?<class>\S+) +(?<name>[^ ]+)\. +(?<proto>\S+) +(?<size>\d+) +(?<do>\S+) +(?<bufsize>\d+)" +(?<rcode>\S+) +(?<rflags>[^ ]+) +(?<rsize>\d+) +(?<duration>\d+\.\d+)s$/
11        </pattern>
12     :
13     :

After reloading the fluentd pod, then the following error appeared.

12021-05-17 08:57:06 +0000 [info]: adding filter pattern="**" type="parser"
2/usr/local/bundle/gems/fluentd-1.11.4/lib/fluent/config/types.rb:92: warning: nested repeat operator '+' and '*' was replaced with '*' in regular expression

So, I deleted the expression and then I paste each element of the regular expression from the beginning one by one and then I found out the cause of the problem is "(?\d+) +*"(?\S+)." As you see, there is "+*" as my mistake. So, I modify the part like this.

1^\[(?<severity>\S+)\] (?<remote>\d+\.\d+\.\d+\.\d+):(?<port>\d+) +- (?<id>\d+) +"(?<type>\S+) +(?<class>\S+) +(?<name>[^ ]+)\. +(?<proto>\S+) +(?<size>\d+) +(?<do>\S+) +(?<bufsize>\d+)" +(?<rcode>\S+) +(?<rflags>[^ ]+) +(?<rsize>\d+) +(?<duration>\d+\.\d+)s$

And then, I restarted the pod and the errors didn’t appear.