[EFK on Kubernetes ] Elasticsearch 수동으로 로그 지우기

https://yjwang.tistory.com/entry/EFK-on-k8s

이전에 작성한 위 포스팅을 따라 EFK를 Kubernetes 위에 구축할 수 있다. 기본으로는 각 Node에 있는 docker log들을 수집하며 로그가 삭제되지 않고 지속적으로 수집된다.

하여 수동 혹은 자동으로 로그를 삭제할 필요가 생기게 되는데 오늘은 이에 대한 포스팅을 다루려고한다.

• 참고 : https://blog.chesterwood.io/2019/06/how-to-delete-old-elasticsearch-logs.html

삭제 진행

---

삭제 전 아래 캡쳐와 같이 2월 5일자 Log가 수집돼있는 것을 볼 수 있다.

elasticsearch는 logstash-2019.04.04 과 같이 일별로 로그가 저장되며 2월 5일자 로그를 삭제할 것이다. 즉 logstash-2021.02.05를 삭제하는 것이다.

우선 현재 있는 index list를 확인한다. (pod ip를 사용했다.)

root@master01:~# curl http://10.233.108.81:9200/_cat/indices?v 
health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   logstash-2021.02.08  KA7B-va-RjmZyvIhDFDtkw   1   1      43299            0     34.6mb         17.8mb
green  open   logstash-2021.01.20  beK8R96PRVq4hXNFsJ9aJQ   1   1        810            0    330.5kb        174.8kb
green  open   logstash-2021.01.30  ku1YY9CvSBWmE4v__mJofQ   1   1      71453            0     35.8mb         17.9mb
green  open   logstash-2021.02.01  DupYNPnQSZuTyMmY2zvMsg   1   1     304955            0    136.7mb         68.8mb
green  open   logstash-2021.01.29  _HS3KLVgTCmzZL7nW7QHmQ   1   1      47621            0     25.8mb           13mb
green  open   logstash-2021.02.03  t0cLWR73Rw6Hy9rfkKekNQ   1   1     304594            0    167.9mb           83mb
green  open   logstash-2021.01.31  Soep_X_dT2OT85UgdmgxTQ   1   1    1417395            0      572mb        286.4mb
green  open   logstash-2021.01.27  yo7G5ZoTS4itEQu4YNtryA   1   1      43065            0     26.3mb         13.9mb
green  open   logstash-2021.01.24  aZURguDxQSqwUJBBnaB7sg   1   1        135            0      123kb         61.5kb
green  open   logstash-2021.02.02  qgeHlmVZQdSFOO2t5Yvyxw   1   1      55566            0     29.9mb         14.5mb
green  open   logstash-2021.02.05  hrlfQGqkQY62XGrInB-xEg   1   1    1020985            0    600.1mb        299.4mb
green  open   .kibana_1            eXbVe4AzQ_WUHWf_YixbZQ   1   1         17            2    248.2kb         87.3kb
green  open   .kibana_task_manager ikGF4FC9RrihzkxGlFOxPA   1   1          2            0     59.8kb         29.9kb
green  open   logstash-2021.02.04  BCoyEpySSD6svXsL0B3B-A   1   1     355158            0    171.8mb         85.1mb
green  open   logstash-2021.02.07  W5EOno5GRXKn1mjL4z54Ug   1   1     476631            0    374.4mb        187.2mb
green  open   logstash-2021.01.21  68sr7yAFQqW8Kcab9ahhWQ   1   1        405            0    202.5kb        101.2kb
green  open   logstash-2021.01.28  opGcSmSpRjCDlmjCKDf9Ew   1   1      43458            0     24.8mb         12.8mb
green  open   logstash-2021.02.06  OnGbzVljRKmA3ZbUQRQ_VA   1   1     476244            0    373.7mb        186.8mb
green  open   logstash-2021.01.19  Hnr4wTf6QbqzRHVU-FThOw   1   1        558            0    405.9kb        188.7kb
green  open   logstash-2021.01.22  iWb7ZPGFRNylSOyJrFBxRw   1   1        540            0    255.2kb        118.6kb
green  open   logstash-2021.01.25  UhAI3cUMTluREi2nNWxQ4A   1   1      41506            0     24.3mb         12.2mb
green  open   logstash-2021.01.26  WE7kM3mYRpefUy1hATLPQQ   1   1      66006            0     36.4mb         16.8mb

이후 삭제를 진행한다.

root@master01:~# curl -X DELETE http://10.233.108.81:9200/logstash-2021.02.05
{"acknowledged":true}

삭제 확인

---

아래와 같이 2월 5일자 로그가 없음을 알 수 있다.

root@master01:~# curl http://10.233.108.81:9200/_cat/indices?v 
health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   logstash-2021.02.08  KA7B-va-RjmZyvIhDFDtkw   1   1      45015            0     36.5mb         19.5mb
green  open   logstash-2021.01.20  beK8R96PRVq4hXNFsJ9aJQ   1   1        810            0    330.5kb        174.8kb
green  open   logstash-2021.01.30  ku1YY9CvSBWmE4v__mJofQ   1   1      71453            0     35.8mb         17.9mb
green  open   logstash-2021.02.01  DupYNPnQSZuTyMmY2zvMsg   1   1     304955            0    136.7mb         68.8mb
green  open   logstash-2021.01.29  _HS3KLVgTCmzZL7nW7QHmQ   1   1      47621            0     25.8mb           13mb
green  open   logstash-2021.02.03  t0cLWR73Rw6Hy9rfkKekNQ   1   1     304594            0    167.9mb           83mb
green  open   logstash-2021.01.31  Soep_X_dT2OT85UgdmgxTQ   1   1    1417395            0      572mb        286.4mb
green  open   logstash-2021.01.27  yo7G5ZoTS4itEQu4YNtryA   1   1      43065            0     26.3mb         13.9mb
green  open   logstash-2021.01.24  aZURguDxQSqwUJBBnaB7sg   1   1        135            0      123kb         61.5kb
green  open   logstash-2021.02.02  qgeHlmVZQdSFOO2t5Yvyxw   1   1      55566            0     29.9mb         14.5mb
green  open   .kibana_1            eXbVe4AzQ_WUHWf_YixbZQ   1   1         17            2    174.8kb         87.4kb
green  open   .kibana_task_manager ikGF4FC9RrihzkxGlFOxPA   1   1          2            0     59.8kb         29.9kb
green  open   logstash-2021.02.04  BCoyEpySSD6svXsL0B3B-A   1   1     355158            0    171.8mb         85.1mb
green  open   logstash-2021.02.07  W5EOno5GRXKn1mjL4z54Ug   1   1     476631            0    374.4mb        187.2mb
green  open   logstash-2021.01.21  68sr7yAFQqW8Kcab9ahhWQ   1   1        405            0    202.5kb        101.2kb
green  open   logstash-2021.01.28  opGcSmSpRjCDlmjCKDf9Ew   1   1      43458            0     24.8mb         12.8mb
green  open   logstash-2021.02.06  OnGbzVljRKmA3ZbUQRQ_VA   1   1     476244            0    373.7mb        186.8mb
green  open   logstash-2021.01.19  Hnr4wTf6QbqzRHVU-FThOw   1   1        558            0    405.9kb        188.7kb
green  open   logstash-2021.01.22  iWb7ZPGFRNylSOyJrFBxRw   1   1        540            0    255.2kb        118.6kb
green  open   logstash-2021.01.25  UhAI3cUMTluREi2nNWxQ4A   1   1      41506            0     24.3mb         12.2mb
green  open   logstash-2021.01.26  WE7kM3mYRpefUy1hATLPQQ   1   1      66006            0     36.4mb         16.8mb