In a previous article, I mentioned the problem of Java program containers generating a lot of zombie processes. Today I ran into it again, but in a different way.

The zombie processes are also generated by a bash script, but this script is executed in the K8S readinessProbe and livenessProbe, not by the java program itself.

Let’s look at some failure diagrams.

You can see that there are many timeout processes that are not cleaned up after execution.

And readinessProbe and livenessProbe look like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

readinessProbe:
  exec:
    command:
    - sh
    -c
    - /health/ping_readiness_local.sh 1
    ...
livenessProbe:
  exec:
    command:
    - sh
    -c
    - /health/ping_liveness_local.sh 5
    ...

After a series of troubleshooting, I found out that it has something to do with the ENTRYPOINT of the image.

The base image is openjdk:8-alpine, and the original one looks like this.

ENTRYPOINT ["/home/java-app/docker-entrypoint.sh"] Change to.

1
2

RUN apk add --no-cache --update tini
ENTRYPOINT ["/sbin/tini", "--", "/home/java-app/docker-entrypoint.sh"]

tini prevents containers from spawning zombie processes.

PS. So far the bot process problem is often found on alpine images.