日常檢查服務的時候，從portainer那里進去看容器日志的時候，發現右上角出現紅色的感嘆號：Unable to retrieve container logs。

因為之前沒出現過這樣的問題，所以就先上服務器上用命令docker logs -f containerID看日志，發現日志也是動不了，還是停留在某個時間的日志記錄上。

想了一下不應該是服務的日志打印出問題，先照著Google搜索了一遍，發現都沒有跟我的問題相匹配的。因為日志有時能收集顯示，有些日志不可以，應該是跟docker設置的日志引擎有問題。

本來是想整一套EFK的，但是感覺現在日志量還不夠大，所以并沒有修改docker的日志引擎，還是默認的journald

[root@ad-official xiaoxiao]# docker info|grep Logging WARNING: You're not using the default seccomp profileLogging Driver: journald

journald的官方文檔上有這么一個說明：

man journald.conf ... RateLimitInterval=, RateLimitBurst= Configures the rate limiting that is applied to all messages generated on the system. If, in the time interval defined by RateLimitInterval=, more messages than specified in RateLimitBurst= are logged by a service, all further messages within the interval are dropped until the interval is over. A message about the number of dropped messages is generated. This rate limiting is applied per-service, so that two services which log do not interfere with each other's limits. Defaults to 1000 messages in 30s. The time specification for RateLimitInterval= may be specified in the following units: "s", "min", "h", "ms", "us". To turn off any kind of rate limiting, set either value to 0. ...

這里寫了默認30秒內只能接收1000條日志，看到這里就能明白了，因為前陣子剛在docker發布了一個單日日志文件大小差不多達到3G的服務，導致到了其他服務的日志也受到了影響，大量的日志被journald丟棄，所以我們修改一下配置就沒有問題了。

打開/etc/systemd/journald.conf文件，將RateLimitBurst從默認的1000修改成5000，根據自己目前的日志輸出量進行調整：

[root@ad-official log]# cat /etc/systemd/journald.conf# This file is part of systemd.## systemd is free software; you can redistribute it and/or modify it# under the terms of the GNU Lesser General Public License as published by# the Free Software Foundation; either version 2.1 of the License, or# (at your option) any later version.## Entries in this file show the compile time defaults.# You can change settings by editing this file.# Defaults can be restored by simply deleting this file.## See journald.conf(5) for details.[Journal]#Storage=auto#Compress=yes#Seal=yes#SplitMode=uid#SyncIntervalSec=5m#RateLimitInterval=30sRateLimitBurst=5000#SystemMaxUse=#SystemKeepFree=#SystemMaxFileSize=#RuntimeMaxUse=#RuntimeKeepFree=#RuntimeMaxFileSize=#MaxRetentionSec=#MaxFileSec=1monthForwardToSyslog=no#ForwardToKMsg=no#ForwardToConsole=noForwardToWall=no#TTYPath=/dev/console#MaxLevelStore=debug#MaxLevelSyslog=debug#MaxLevelKMsg=notice#MaxLevelConsole=info#MaxLevelWall=emerg#LineMax=48K

順便將ForwardToSyslog和ForwardToWall設置成no，因為默認是yes，會導致我們清理了journal的日志文件，而Syslog中的沒有清除掉，慢慢的就會將磁盤占滿。