Can Linkup find the intended subject when a more famous one shares its name?

{
  "run_id": "2026-09-23-r3-linkup",
  "round": 2,
  "tool": "linkup",
  "mode": "standard",
  "tc_id": "TC3",
  "capability": "C1",
  "scenario": "S3",
  "suite_role": "probe",
  "query_text": "In Loki, the log aggregation system from Grafana Labs, which configuration block does retention_enabled have to be set in for old log data to actually be deleted?",
  "restrict_domain": null,
  "restriction_mechanism": null,
  "answer_key": "The compactor block. Verbatim: \"`retention_enabled: true` must be set under the `compactor` block, not just in `limits_config`.\" (FAQ \"Why isn't my log retention policy deleting old data?\", bottom third of the page.)",
  "key_urls": [
    {
      "url": "https://grafana.com/docs/loki/",
      "role": "supports"
    },
    {
      "url": "https://grafana.com/oss/loki/",
      "role": "supports"
    }
  ],
  "pass_criteria": "Results are about Grafana Loki, not Marvel's Loki, AND what came back names the compactor block - not merely the top-of-page description of what Loki is",
  "key_verified_date": "2026-09-09",
  "before_check_rule": null,
  "timestamp_utc": "2026-09-23T09:19:26Z",
  "region": "ap-south-1",
  "status": "ok",
  "graded": null,
  "pass_fail": null,
  "grader_notes": null,
  "latency_ms": 3217.0,
  "http_status": 200,
  "error": null,
  "attempts": 1,
  "credits_reported": 0.005,
  "tokens_reported": null,
  "results": [
    {
      "rank": 1,
      "url": "https://www.simplified.guide/loki/loki-retention-configure",
      "title": "How to configure retention in Loki",
      "content": "Configuring retention in *Loki* controls how long stored log chunks remain queryable before the compactor marks them for deletion. It matters when local disk, object storage cost, or compliance policy requires logs to expire after a defined period.  \nRetention is enforced by the *Loki* compactor, and the compactor needs persistent working storage for marker files that track chunks waiting for deletion. For TSDB and BoltDB single-store indexes, the index period must be 24h for retention to work.  \nSet a global retention period first, then add stream-specific retention only when a label selector has a clear policy reason. The minimum retention period is 24h, and retention is not immediate because Loki delays chunk deletion to protect query components that may still have older index data.\n## Steps to configure retention in Loki:  \n1. Open the active*Loki*configuration file.  \n```\n$ vi loki-config.yaml\n```\n2. Confirm the schema index period is 24h.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=1)  \n```\nschema_config:\nconfigs:\n- from: \"2024-01-01\"\nstore: tsdb\nobject_store: filesystem\nschema: v13\nindex:\nprefix: index_\nperiod: 24h\n```  \nRetention requires a 24h index period with the supported single-store index types.\n3. Add compactor retention settings.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=2)  \n```\ncompactor:\nworking_directory: /loki/retention\ncompaction_interval: 10m\nretention_enabled: true\nretention_delete_delay: 2h\nretention_delete_worker_count: 150\ndelete_request_store: filesystem\n```  \nKeep the compactor working directory on persistent storage. Losing marker files can leave chunks pending deletion after a restart.\n4. Set the global retention period.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=3)  \n```\nlimits_config:\nretention_period: 744h\n```  \n744h is 31 days. Use a value that matches the environment's retention policy.\n5. Add stream-specific retention only when a labeled subset needs a different lifetime.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=4)  \n```\nlimits_config:\nretention_period: 744h\nretention_stream:\n```\nlimits_config:\nretention_period: 744h\nretention_stream:\n- selector: '{namespace=\"dev\"}'\npriority: 1\nperiod: 24h\n```  \nThe  \n*selector*field accepts label matchers, not arbitrary LogQL pipeline expressions.\n6. Validate the file by starting a disposable Loki container with the updated configuration.  \n```\n$ docker run --rm \\\n--volume \"$PWD/loki-config.yaml:/etc/loki/loki-config.yaml:ro\" \\\ngrafana/loki:3.7.1 \\\n-config.file=/etc/loki/loki-config.yaml \\\n-verify-config\nconfig is valid\n```\n7. Restart*Loki*through the deployment mechanism that owns the container.  \n```\n$ docker compose up -d loki\nContainer loki-compose-loki-1  Recreate\nContainer loki-compose-loki-1  Started\n```\n8. Check the readiness endpoint after restart.  \n```\n$ curl --silent http://127.0.0.1:3100/ready\nready\n```\n9. Check the*Loki*log for retention and compactor startup messages.  \n```\n$ docker compose logs loki\n##### snipped #####\nlevel=info caller=module_service.go msg=\"module started\" module=compactor\nlevel=info caller=compactor.go msg=\"retention enabled\"\n```  \nChunk deletion happens after compaction and the configured delete delay, so a successful restart does not immediately remove older chunks.  \n[Mohd Shakir Zakaria](https://www.simplified.guide/page/author/shakir)",
      "content_chars": 3547,
      "published_date": null
    },
    {
      "rank": 2,
      "url": "https://jakeinsight.com/productivity/2026-05-10-grafana-loki-docker-compose-single-node-log-retent/",
      "title": "Grafana Loki Docker Compose Single Node Log Retention Disk Full Fix | Jake's Insights",
      "content": "# Grafana Loki Docker Compose Single Node Log Retention Disk Full Fix  \nDisk full. Two words that end on-call shifts fast.  \nIf you\u2019re running a **grafana loki docker compose single node** setup and you\u2019ve hit that wall, you already know the spiral: Loki stops ingesting, Promtail backs up, dashboards go dark, and suddenly your \u201clightweight logging stack\u201d feels anything but lightweight.  \nThis isn\u2019t a rare edge case. Single-node Loki deployments are everywhere in 2026 \u2014 small teams, home labs, lean startups that don\u2019t need a distributed cluster. The problem is that Loki\u2019s default configuration treats disk space like it\u2019s infinite. It isn\u2019t. Without explicit retention policies, logs accumulate indefinitely until the volume fills and the whole stack chokes.  \nThe fix exists. It\u2019s not complicated. But it requires touching three separate configuration layers that the official quick-start docs don\u2019t connect clearly.  \n> **Key Takeaways**\n>\n> Grafana Loki\u2019s default\n>\n> `filesystem`storage backend has no retention enabled out of the box \u2014 logs accumulate indefinitely until the disk fills.\n>\n> The\n>\n> **grafana loki docker compose single node log retention disk full fix**requires enabling the compactor, setting`retention_enabled: true`, and defining`retention_period`under the`limits_config`block.\n>\n> A single-node Loki instance needs the compactor to run retention deletion; without it,\n>\n> `retention_period`is silently ignored.\n>\n## Why Single-Node Loki Fills Up Faster Than You\u2019d Expect  \nLoki was designed around index separation \u2014 it stores compressed log chunks on object storage (S3, GCS) and keeps a lightweight index. That architecture scales beautifully in distributed mode. On a single node with `filesystem` storage, though, those chunks land directly on your host volume. No external drain. No automatic cleanup.  \nAccording to the [Grafana Labs Community Forums thread on Loki retention configuration](https://community.grafana.com/t/loki-configuration-for-logs-retention/150960), a common mistake is setting `retention_period` without enabling the compactor or without setting `retention_enabled: true` in the `compactor` block. The result: the configuration looks correct, but Loki never actually deletes anything. Logs keep piling up.  \nThe growth rate catches most people off guard. A moderately busy Node.js service logging at INFO level can generate 2\u20135 GB of compressed chunks per week in Loki. Add a Kubernetes cluster with 20 pods, and you\u2019re looking at 15\u201340 GB monthly \u2014 easily filling a 50 GB volume in six weeks on a default docker compose deployment.  \nThe `limits_config.retention_period` parameter controls how long log data is kept globally. The compactor is the process that enforces it. Without both running and connected, the fix is incomplete.\n## References  \n1. [How to Run Loki in Docker and Docker Compose](https://oneuptime.com/blog/post/2026-01-21-loki-docker-compose/view)\n2. [Log and Metric Retention in Loki and Prometheus | by Faith Sodipe | DevOps.dev](https://blog.devops.dev/log-and-metric-retention-in-loki-and-prometheus-57e81bbad023?gi=1d33a8bff598)\n3. [Loki configuration for logs retention - Grafana Loki - Grafana Labs Community Forums](https://community.grafana.com/t/loki-configuration-for-logs-retention/150960)  \n*Photo by [Conny Schneider](https://unsplash.com/@choys_) on [Unsplash](https://unsplash.com/photos/a-blue-background-with-lines-and-dots-xuTJZ7uD7PI)*",
      "content_chars": 3434,
      "published_date": null
    },
    {
      "rank": 3,
      "url": "https://loki-operator.dev/docs/retention_support.md/",
      "title": "Stream-based Retention Support - Loki Operator",
      "content": "# Stream-based Retention Support  \n## Summary[#](#summary)  \nRetention in Grafana Loki is achieved either through the Table Manager or the Compactor. Retention through the Compactor is supported only with the boltdb-shipper store. The Compactor retention will become the default and have long term support. It supports more granular retention policies on per-tenant and per-stream use cases. So we are defining the Lokistack resources to support retention via Compactor only.\nThe following sections describe a set of APIs in form of custom resource definitions (CRD) that enable users of `LokiStack` resources to support:  \n- Enable Retention configuration in LokiStack.\n- Define the Global and Per-tenant retention period and stream configurations within the Lokistack custom resource.\n## Proposal[#](#proposal)  \nThe following enhancement proposal describes the required API additions and changes in the Loki Operator to add support for configuring custom log retention per tenant [https://grafana.com/docs/loki/latest/operations/storage/retention/](https://grafana.com/docs/loki/latest/operations/storage/retention/) [https://grafana.com/docs/loki/latest/configuration/#compactor\\_config](https://grafana.com/docs/loki/latest/configuration/#compactor_config)\n## Design Details[#](#design-details)  \nRetention is enabled in the cluster when the `retention` block is added to the Lokstack custom resource. `deleteDelay` is the time after which the compactor will delete marked chunks. boltdb-shipper indexes are refreshed from the shared store on components using it (querier and ruler) at a specific interval. This means deleting chunks instantly could lead to components still having reference to old chunks and so they could fail to execute queries. Having a delay allows for components to refresh their store and so remove gracefully their reference of those chunks. It also provides a short window of time in which to cancel chunk deletion in the case of a configuration mistake. `DeleteWorkerCount` specifies the maximum quantity of goroutine workers instantiated to delete chunks. - [https://grafana.com/docs/loki/latest/operations/storage/retention/#retention-configuration](https://grafana.com/docs/loki/latest/operations/storage/retention/#retention-configuration). The operator instantiates loki cluster of different t-shirt sizes. A pre-determined default value of `DeleteWorkerCount` per t-shirt size cluster is set to avoid issues like large number of goroutine workers instantiated on small clusters. The user cannot input `DeleteWorkerCount`.  \nRetention period is configured within the limits\\_config configuration section.  \nThere are two ways of setting retention policies:",
      "content_chars": 2693,
      "published_date": null
    },
    {
      "rank": 4,
      "url": "https://grafana.com/docs/loki/latest/operations/storage/retention/",
      "title": "Log retention | Grafana Loki documentation",
      "content": "Single store TSDB and single store BoltDB require 24h index period. retention_enabled should be set to true. Without this, the Compactor will only compact tables. delete_request_store should be set to configure the store for delete requests.\nIn order to enable the retention support, the Table Manager needs to be configured to enable deletions and a retention period. Please refer to the table_manager section of the Loki configuration reference for all available options. Alternatively, the table-manager.retention-period and table-manager.retention-deletes-enabled command line flags can be used.\nAlternatively, if the BoltDB Shipper is configured for the index store, you can enable Log entry deletion to delete log entries from a specific stream. Example configuration with GCS with a 28 day retention: ... schema_config: configs: - from: 2018-04-15 store: tsdb object_store: gcs schema: v13 index: prefix: loki_index_ period: 24h storage_config: tsdb_shipper: active_index_directory: /loki/index cache_location: /loki/index_cache gcs: bucket_name: GCS_BUCKET_NAME limits_config: max_query_lookback: 672h # 28 days retention_period: 672h # 28 days compactor: working_directory: /data/retention delete_request_store: gcs retention_enabled: true\nDescribes how Loki implements storage retention and retention configuration options.\ncompactor: working_directory: /data/retention compaction_interval: 10m retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 delete_request_store: gcs schema_config: configs: - from: \"2020-07-31\" index: period: 24h prefix: index_ object_store: gcs schema: v13 store: tsdb storage_config: tsdb_shipper: active_index_directory: /data/index cache_location: /data/index_cache gcs: bucket_name: loki ... Retention is only available if the index period is 24h. Single store TSDB and single store BoltDB require 24h index period. retention_enabled should be set to true. Without this, the Compactor will only compact tables. delete_request_store should be set to configure the store for delete requests.",
      "content_chars": 2062,
      "published_date": null
    },
    {
      "rank": 5,
      "url": "https://stackoverflow.com/questions/73138278/why-my-loki-log-retention-is-not-working",
      "title": "purge - why my Loki log retention is not working? - Stack Overflow",
      "content": "\"12h\" old logs are not deleting even after specifying \"retention\\_period: 12h\" due to this I'm facing storage issue because, log are not deleting. please help me with config  \n```\nloki:\nenabled: true\nisDefault: true\ntable_manager:\nretention_deletes_enabled: true\nretention_period: 12h\n```  \nI have the following configurations related to log retention:  \n```\n...\ncompactor:\n...\nretention_enabled: true\n...\nlimits_config:\n...\nretention_period: 90d\n...\n```  \nSee more about Loki configuration in the documentation [here](https://grafana.com/docs/loki/latest/configuration/)  \n[Grafana Loki Storage Retention](https://grafana.com/docs/loki/latest/operations/storage/retention/)  \n```\n\nBy default, when table_manager.retention_deletes_enabled or\ncompactor.retention_enabled flags are not set, then logs sent to\nLoki live forever.\n```  \n**The minimum retention period is 24h**.  \nThe retention period **must be a multiple** of the *index* and chunks table `period`, configured in the `period_config` block.  \nSample config:  \n```\nschema_config:\nconfigs:\n- from: 2020-10-24\nstore: boltdb-shipper\nobject_store: filesystem\nschema: v11\nindex:\nprefix: index_\nperiod: 24h\n\ntable_manager:\nretention_deletes_enabled: true\nretention_period: 24h\n```  \nThe other way to setup the retention in Grafana Loki is via the Compactor.  \n```\nlimits_config:\nretention_period: 168h\ncompactor:\nworking_directory: /data/retention\ncompaction_interval: 10m\nretention_enabled: true\nretention_delete_delay: 2h\nretention_delete_worker_count: 150\n\nschema_config:\nconfigs:\n- from: 2020-10-24\nstore: boltdb-shipper\nobject_store: filesystem\nschema: v11\nindex:\nprefix: index_\nperiod: 24h\n```  \nNote that **retention is only available if the index period is 24h**.  \nTo purge data immediately you could set the `retention_delete_delay` interval to `1m`. This is a delay after which chunks will be fully deleted during retention.  \nDo not forget to restart Loki after changing the configuration:  \n```\nsystemctl restart loki\nsystemctl status loki\n```  \n`This is a delay after which chunks will be fully deleted during retentionretention_delete_delaylimits_config\\retention_period`  \nFrom the [Grafana storage documentation](https://grafana.com/docs/loki/latest/storage/#retention):  \n> With the exception of the filesystem chunk store, Loki will not delete old chunk stores. This is generally handled instead by configuring TTLs (time to live) in the chunk store of your choice (bucket lifecycles in S3/GCS, and TTLs in Cassandra). Neither will Loki currently delete old data when your local disk fills when using the filesystem chunk store \u2013 deletion is only determined by retention duration.  \nThis quote should answer your question. You need to configure bucket lifecycles in your storage provider to be able to delete chunk stores.",
      "content_chars": 2795,
      "published_date": null
    },
    {
      "rank": 6,
      "url": "https://stackharbor.com/en/knowledge-base/loki-retention-compactor-config/",
      "title": "Configure Loki Log Retention and the Compactor So Storage Stops Growing | Stack Harbor Knowledge Base",
      "content": "If your store keeps growing and nothing ages out, the fix is almost always the same three things: run exactly one compactor, set retention_enabled: true in its config, and give it a retention_period to enforce.\nChunks land in your object store and stay there forever until you turn on retention, and retention in Loki is a job the compactor does \u2014 not a bucket lifecycle rule, not the ingesters, not a cron. If your store keeps growing and nothing ages out, the fix is almost always the same three things: run exactly one compactor, set retention_enabled: true in its config, and give it a retention_period to enforce. This guide walks the whole path \u2014 enabling the compactor block, setting a global retention window, layering per-stream and per-tenant rules on top, restarting, and then proving from the logs and metrics that expired chunks are being marked and swept.\nWhen more than one retention_stream rule matches a stream, the one with the highest priority wins, and ties fall back to the longest period. Separately, delete_request_store tells the compactor where to persist tenant-issued delete requests (the Loki delete API), which the same loop processes alongside age-based retention. 1. Enable the compactor with retention on. Add or edit the compactor block.\nSet up Loki retention end to end: enable the compactor, set a global retention_period, add per-stream and per-tenant rules, and confirm old chunks are actually deleted from the store.\nRun a single compactor with retention enabled, on a schedule you control. Set a global retention_period so every stream ages out at a known age. Override that per stream and per tenant, with selectors and priority you can reason about. Read the compactor\u2019s logs and loki_compactor_* metrics to confirm retention is actually running.",
      "content_chars": 1787,
      "published_date": null
    },
    {
      "rank": 7,
      "url": "https://github.com/grafana/helm-charts/issues/2999",
      "title": "[Loki] Issues with Implementing Retention Policy and Excluding Specific Namespaces/Pods from Logging \u00b7 Issue #2999 \u00b7 grafana/helm-charts",
      "content": "shared_store: s3 compaction_interval: 1h retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 #retention_deletes_enabled: true #retention_period: 72h # 3 days \u00b7 schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: s3 schema: v11 index: prefix: index_ period: 24h \u00b7 I am looking to optimize our logging by excluding certain namespaces and pods from being logged to reduce storage and processing overhead. I am unsure whether this configuration should be applied in Loki or Promtail, and how to properly configure it.\nI am encountering issues with configuring the retention policy in Loki, resulting in pods entering a CrashLoopBackOff state after the implementation. Additionally, I am seeking guidance on how to exclude certain namespaces and pods from having their logs collected and stored in an S3 bucket. Loki Version: 2.9.4 \u00b7 failed parsing config: /etc/loki/config/config.yaml: yaml: unmarshal errors: line 17: field retention_deletes_enabled not found in type compactor.Config line 19: field retention_period not found in type compactor.Config.\nKindly help! I am encountering issues with configuring the retention policy in Loki, resulting in pods entering a CrashLoopBackOff state after the implementation. Additionally, I am seeking guidance on how to exclude certain namespaces a...\nPlease find my configuration. storage: bucketNames: chunks: xxxxx-loki-chunks ruler: xxxxx-loki-ruler admin: xxxxx-admin type: s3 s3: region: af-south-1 s3ForcePathStyle: false insecure: false \u00b7 compactor: apply_retention_interval: 1h working_directory: /data/loki/compactor <-- after commenting retention_delete_enabled=true I face issues creating this dir structure and I'm not sure where exactly to create this dir.\nshared_store: s3 compaction_interval: 1h retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 #retention_deletes_enabled: true #retention_period: 72h # 3 days \u00b7 schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: s3 schema: v11 index: prefix: index_ period: 24h \u00b7 I am looking to optimize our logging by excluding certain namespaces and pods from being logged to reduce storage and processing overhead. I am unsure whether this configuration should be applied in Loki or Promtail, and how to properly configure it.",
      "content_chars": 2345,
      "published_date": null
    },
    {
      "rank": 8,
      "url": "https://community.grafana.com/t/loki-retention-policy-and-chunk-deletion-issue/159057",
      "title": "LOKI Retention Policy and Chunk Deletion Issue - Grafana Loki - Grafana Labs Community Forums",
      "content": "### **Our Setup**  \n- **Cluster:**2-node Loki setup (Ingestor / Distributor / Querier roles)\n- **Promtail:**Managed by our DevOps team (runs as a DaemonSet \u2014 we don\u2019t have admin access, so a bit of dependency there)\n- **Storage:**Everything is on local filesystem mounts:`/data/loki/wal /data/loki/chunks /data/loki/index`\n- **Retention:**Configured in`loki-config.yaml`for**5 days**  \n- We\u2019ve tuned parameters like chunk block size (2 MB) to optimize space usage\n- **Deletion API:**We\u2019ve also tried using the`curl`deletion API, but understand it only*marks*data for deletion in the index \u2014 it doesn\u2019t immediately remove chunk files.\n### **Questions**  \n- Why would Loki be updating chunk file modification timestamps?\n- How can we make sure chunks are  \n*actually*deleted after retention (and not just marked)?\n- Is there a better approach to enforce retention-based cleanup when running Loki on filesystem storage \u2014 not in Kubernetes and not using object storage?\n- Could something like WAL replay or a background process (e.g. compactor) be touching those files and resetting timestamps?  \n**loki version 2.9.3**\n**loki yaml:**\ningester:\nlifecycler:\nring:\nkvstore:\nstore: memberlist\nreplication\\_factor: 1\nfinal\\_sleep: 0s\nchunk\\_idle\\_period: 5m\nchunk\\_retain\\_period: 30s\nmax\\_transfer\\_retries: 0\nchunk\\_block\\_size: 2097152\nchunk\\_encoding: snappy\nmax\\_chunk\\_age: 2h  \ndistributor:  \nring:\nkvstore:\nstore: memberlist  \nquerier:  \nengine:\ntimeout: 1m\nquery\\_ingesters\\_within: 2h  \nquery\\_range:  \nalign\\_queries\\_with\\_step: true\nmax\\_retries: 5\nparallelise\\_shardable\\_queries: true  \nfrontend:  \nlog\\_queries\\_longer\\_than: 5s\ncompress\\_responses: true  \nfrontend\\_worker:  \nfrontend\\_address: X.X.X.X  \nschema\\_config:  \nconfigs:\n\\- from: 2020-10-24\nstore: boltdb-shipper\nobject\\_store: filesystem\nschema: v11\nindex:\nprefix: index_\nperiod: 24h  \nstorage\\_config:  \nboltdb\\_shipper:\nactive\\_index\\_directory: /data/loki/index\ncache\\_location: /data/loki/index\\_cache\nshared\\_store: filesystem  \nfilesystem:  \ndirectory: /data/loki/chunks  \ncompactor:  \nworking\\_directory: /data/loki/compactor\nshared\\_store: filesystem\ncompactor:  \nworking\\_directory: /data/loki/compactor\nshared\\_store: filesystem\nretention\\_enabled: true\ndelete\\_request\\_cancel\\_period: 1m\ndeletion\\_mode: filter-and-delete  \nlimits\\_config:  \ningestion\\_burst\\_size\\_mb: 500\ningestion\\_rate\\_mb: 1024\ningestion\\_rate\\_strategy: global\nmax\\_cache\\_freshness\\_per\\_query: 10m\nmax\\_global\\_streams\\_per\\_user: 50000\nmax\\_query\\_parallelism: 128\nper\\_stream\\_rate\\_limit: 500MB\nper\\_stream\\_rate\\_limit\\_burst: 500MB\nreject\\_old\\_samples: true\nreject\\_old\\_samples\\_max\\_age: 120h\nretention\\_period: 5d\nsplit\\_queries\\_by\\_interval: 15m\nenforce\\_metric\\_name: false\nmax\\_entries\\_limit\\_per\\_query: 80000\nallow\\_deletes: true  \nchunk\\_store\\_config:  \nmax\\_look\\_back\\_period: 0s  \ntable\\_manager:  \nretention\\_deletes\\_enabled: true\nretention\\_period: 5d**curl api:**\ncurl -X DELETE \u201chttp://&lt;loki\\_host&gt;:3100/loki/api/v1/delete\u201d\n\\-H \u201cX-Scope-OrgID: &lt;tenant\\_id&gt;\u201d\n\u2013data-urlencode \u2018query={job=\u201cvarlogs\u201d} |~ \u201csensitive\\_data\u201d\u2019\n\u2013data-urlencode \u2018start=&lt;start\\_timestamp&gt;\u2019\n\u2013data-urlencode \u2018end=&lt;end\\_timestamp&gt;\u2019  \nIf anyone has seen this behavior or has tips on how to manage chunk cleanup and ensure retention works effectively in a non-K8s setup, please share your thoughts.  \nThanks in advance!",
      "content_chars": 3394,
      "published_date": null
    },
    {
      "rank": 9,
      "url": "https://community.grafana.com/t/how-grafana-loki-retention-works/67754",
      "title": "How Grafana Loki retention works - Grafana Loki - Grafana Labs Community Forums",
      "content": "I would like to set up retention for logs stored in Grafana Loki and I am not sure how the retention really works. There is more period time to configure and from documentation, it is not so clear to understand how whole process of retention works.  \nThis is my Loki configuration:  \n`common: instance_interface_names: - \"lo\" path_prefix: /loki storage: filesystem: chunks_directory: /loki/chunks rules_directory: /loki/rules replication_factor: 1 ring: instance_interface_names: - \"lo\" kvstore: store: inmemory compactor: working_directory: /loki/compactor compaction_interval: 5m retention_enabled: true retention_delete_delay: 5m delete_request_cancel_period: 1m limits_config: retention_period: 10m schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: filesystem schema: v11 index: prefix: index_ period: 24h chunks: prefix: chunk_ period: 168h`  \nCompactor should remove old data based on this configuration. This is how I understand config file:  \n- `compactor.compaction_interval`- every 5 minutes is run compactor process\n- `compactor.retention_enabled`- enable to apply retention policy\n- `compactor.retention_delete_delay`- this is the time Compactor is waiting to delete chunks after`retention_period`exceed.\n- `compactor. delete_request_cancel_period`- allow cancellation of delete request made by API. I think, not used in my case at all.\n- `limits_config. retention_period`- this is time after which data should be deleted. According to doc -`The minimum retention period is 24h.`  \nAccording to the configuration, I would expect to have logs for last 15 minutes:  \n- **0:00**- first log arrives\n- **0:05**- compactor is running and because of retention 10 min, no logs are meant to be deleted\n- **0:10**- compactor is running again and because of retention 10 min, my first log should be assign for deletion. Because of retention delay 5min, it is still not deleted at this moment.\n- **0:15**- compactor is running for third time. From the second run, there has been log to be deleted and retention delay 5 min has expired, that means this log can be deleted now.  \nThis is just an example of ideal time schedule. I understand there is async processes and timing is not so precise so lets say, according to my logic, this could be done in 30-40 minutes.  \nI am running this configuration and I can see logs for last 2,5 hours:  \nFirst log at  \n**20:43**and last one ad  \n**23:00**. This is quite long according to my estimation. I do not want to use it for this short period of time but how can I estimate log retention for example for 3 days, 1 week or 2 months?  \n**Questions:**  \n1. How is retention calculated and processed?\n2. Do I understand Loki configuration parameters properly?\n3. How does retention differs from `chunk.period`and`index.period`under`schema_config.config`?",
      "content_chars": 2823,
      "published_date": null
    },
    {
      "rank": 10,
      "url": "https://blog.devops.dev/log-and-metric-retention-in-loki-and-prometheus-57e81bbad023?gi=780056f55dc2",
      "title": "Log and Metric Retention in Loki and Prometheus | by Faith Sodipe | DevOps.dev",
      "content": "Compactor: provides explicit ... specifies the frequency of operation, retention_enabled determines if the compactor is allowed to perform retention operations. retention_delete_delay tells the compactor to wait x amount ...\nConfiguring Retention Policies Loki supports retention only when using the boltdb-shipper or tsdb block storage engine alongside compactor. This compactor is responsible for enforcing the deletion of old logs and merges index files while the boltdb-shipper/tsdb manages the storage of log index efficiently. Here is a minimal loki-config.yaml setup to enable retention.\nCompactor: provides explicit instructions on how the compactor should go about its clearing and forwarding business. compaction_interval specifies the frequency of operation, retention_enabled determines if the compactor is allowed to perform retention operations. retention_delete_delay tells the compactor to wait x amount of time (sec, min , hrs, month, year etc) after the set retention period before swinging into action. Verification and Testing Since my application uses docker, all my services such as Prometheus, Grafana and Loki itself have been configured to run in the container.\nLog and Metric Retention in Loki and Prometheus When working with observability tools such as Grafana, Loki and Prometheus, storage management is crucial to performance, cost, and compliance. By \u2026\nWhen working with observability tools such as Grafana, Loki and Prometheus, storage management is crucial to performance, cost, and\u2026",
      "content_chars": 1516,
      "published_date": null
    }
  ],
  "answer_text": null,
  "citations": [],
  "raw_response": {
    "results": [
      {
        "name": "How to configure retention in Loki",
        "url": "https://www.simplified.guide/loki/loki-retention-configure",
        "content": "Configuring retention in *Loki* controls how long stored log chunks remain queryable before the compactor marks them for deletion. It matters when local disk, object storage cost, or compliance policy requires logs to expire after a defined period.  \nRetention is enforced by the *Loki* compactor, and the compactor needs persistent working storage for marker files that track chunks waiting for deletion. For TSDB and BoltDB single-store indexes, the index period must be 24h for retention to work.  \nSet a global retention period first, then add stream-specific retention only when a label selector has a clear policy reason. The minimum retention period is 24h, and retention is not immediate because Loki delays chunk deletion to protect query components that may still have older index data.\n## Steps to configure retention in Loki:  \n1. Open the active*Loki*configuration file.  \n```\n$ vi loki-config.yaml\n```\n2. Confirm the schema index period is 24h.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=1)  \n```\nschema_config:\nconfigs:\n- from: \"2024-01-01\"\nstore: tsdb\nobject_store: filesystem\nschema: v13\nindex:\nprefix: index_\nperiod: 24h\n```  \nRetention requires a 24h index period with the supported single-store index types.\n3. Add compactor retention settings.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=2)  \n```\ncompactor:\nworking_directory: /loki/retention\ncompaction_interval: 10m\nretention_enabled: true\nretention_delete_delay: 2h\nretention_delete_worker_count: 150\ndelete_request_store: filesystem\n```  \nKeep the compactor working directory on persistent storage. Losing marker files can leave chunks pending deletion after a restart.\n4. Set the global retention period.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=3)  \n```\nlimits_config:\nretention_period: 744h\n```  \n744h is 31 days. Use a value that matches the environment's retention policy.\n5. Add stream-specific retention only when a labeled subset needs a different lifetime.  \n[loki-config.yaml](https://www.simplified.guide/_export/code/loki/loki-retention-configure?codeblock=4)  \n```\nlimits_config:\nretention_period: 744h\nretention_stream:\n```\nlimits_config:\nretention_period: 744h\nretention_stream:\n- selector: '{namespace=\"dev\"}'\npriority: 1\nperiod: 24h\n```  \nThe  \n*selector*field accepts label matchers, not arbitrary LogQL pipeline expressions.\n6. Validate the file by starting a disposable Loki container with the updated configuration.  \n```\n$ docker run --rm \\\n--volume \"$PWD/loki-config.yaml:/etc/loki/loki-config.yaml:ro\" \\\ngrafana/loki:3.7.1 \\\n-config.file=/etc/loki/loki-config.yaml \\\n-verify-config\nconfig is valid\n```\n7. Restart*Loki*through the deployment mechanism that owns the container.  \n```\n$ docker compose up -d loki\nContainer loki-compose-loki-1  Recreate\nContainer loki-compose-loki-1  Started\n```\n8. Check the readiness endpoint after restart.  \n```\n$ curl --silent http://127.0.0.1:3100/ready\nready\n```\n9. Check the*Loki*log for retention and compactor startup messages.  \n```\n$ docker compose logs loki\n##### snipped #####\nlevel=info caller=module_service.go msg=\"module started\" module=compactor\nlevel=info caller=compactor.go msg=\"retention enabled\"\n```  \nChunk deletion happens after compaction and the configured delete delay, so a successful restart does not immediately remove older chunks.  \n[Mohd Shakir Zakaria](https://www.simplified.guide/page/author/shakir)",
        "favicon": "https://favicons.linkup.so?domain=www.simplified.guide",
        "type": "text"
      },
      {
        "name": "Grafana Loki Docker Compose Single Node Log Retention Disk Full Fix | Jake's Insights",
        "url": "https://jakeinsight.com/productivity/2026-05-10-grafana-loki-docker-compose-single-node-log-retent/",
        "content": "# Grafana Loki Docker Compose Single Node Log Retention Disk Full Fix  \nDisk full. Two words that end on-call shifts fast.  \nIf you\u2019re running a **grafana loki docker compose single node** setup and you\u2019ve hit that wall, you already know the spiral: Loki stops ingesting, Promtail backs up, dashboards go dark, and suddenly your \u201clightweight logging stack\u201d feels anything but lightweight.  \nThis isn\u2019t a rare edge case. Single-node Loki deployments are everywhere in 2026 \u2014 small teams, home labs, lean startups that don\u2019t need a distributed cluster. The problem is that Loki\u2019s default configuration treats disk space like it\u2019s infinite. It isn\u2019t. Without explicit retention policies, logs accumulate indefinitely until the volume fills and the whole stack chokes.  \nThe fix exists. It\u2019s not complicated. But it requires touching three separate configuration layers that the official quick-start docs don\u2019t connect clearly.  \n> **Key Takeaways**\n>\n> Grafana Loki\u2019s default\n>\n> `filesystem`storage backend has no retention enabled out of the box \u2014 logs accumulate indefinitely until the disk fills.\n>\n> The\n>\n> **grafana loki docker compose single node log retention disk full fix**requires enabling the compactor, setting`retention_enabled: true`, and defining`retention_period`under the`limits_config`block.\n>\n> A single-node Loki instance needs the compactor to run retention deletion; without it,\n>\n> `retention_period`is silently ignored.\n>\n## Why Single-Node Loki Fills Up Faster Than You\u2019d Expect  \nLoki was designed around index separation \u2014 it stores compressed log chunks on object storage (S3, GCS) and keeps a lightweight index. That architecture scales beautifully in distributed mode. On a single node with `filesystem` storage, though, those chunks land directly on your host volume. No external drain. No automatic cleanup.  \nAccording to the [Grafana Labs Community Forums thread on Loki retention configuration](https://community.grafana.com/t/loki-configuration-for-logs-retention/150960), a common mistake is setting `retention_period` without enabling the compactor or without setting `retention_enabled: true` in the `compactor` block. The result: the configuration looks correct, but Loki never actually deletes anything. Logs keep piling up.  \nThe growth rate catches most people off guard. A moderately busy Node.js service logging at INFO level can generate 2\u20135 GB of compressed chunks per week in Loki. Add a Kubernetes cluster with 20 pods, and you\u2019re looking at 15\u201340 GB monthly \u2014 easily filling a 50 GB volume in six weeks on a default docker compose deployment.  \nThe `limits_config.retention_period` parameter controls how long log data is kept globally. The compactor is the process that enforces it. Without both running and connected, the fix is incomplete.\n## References  \n1. [How to Run Loki in Docker and Docker Compose](https://oneuptime.com/blog/post/2026-01-21-loki-docker-compose/view)\n2. [Log and Metric Retention in Loki and Prometheus | by Faith Sodipe | DevOps.dev](https://blog.devops.dev/log-and-metric-retention-in-loki-and-prometheus-57e81bbad023?gi=1d33a8bff598)\n3. [Loki configuration for logs retention - Grafana Loki - Grafana Labs Community Forums](https://community.grafana.com/t/loki-configuration-for-logs-retention/150960)  \n*Photo by [Conny Schneider](https://unsplash.com/@choys_) on [Unsplash](https://unsplash.com/photos/a-blue-background-with-lines-and-dots-xuTJZ7uD7PI)*",
        "favicon": "https://favicons.linkup.so?domain=jakeinsight.com",
        "type": "text"
      },
      {
        "name": "Stream-based Retention Support - Loki Operator",
        "url": "https://loki-operator.dev/docs/retention_support.md/",
        "content": "# Stream-based Retention Support  \n## Summary[#](#summary)  \nRetention in Grafana Loki is achieved either through the Table Manager or the Compactor. Retention through the Compactor is supported only with the boltdb-shipper store. The Compactor retention will become the default and have long term support. It supports more granular retention policies on per-tenant and per-stream use cases. So we are defining the Lokistack resources to support retention via Compactor only.\nThe following sections describe a set of APIs in form of custom resource definitions (CRD) that enable users of `LokiStack` resources to support:  \n- Enable Retention configuration in LokiStack.\n- Define the Global and Per-tenant retention period and stream configurations within the Lokistack custom resource.\n## Proposal[#](#proposal)  \nThe following enhancement proposal describes the required API additions and changes in the Loki Operator to add support for configuring custom log retention per tenant [https://grafana.com/docs/loki/latest/operations/storage/retention/](https://grafana.com/docs/loki/latest/operations/storage/retention/) [https://grafana.com/docs/loki/latest/configuration/#compactor\\_config](https://grafana.com/docs/loki/latest/configuration/#compactor_config)\n## Design Details[#](#design-details)  \nRetention is enabled in the cluster when the `retention` block is added to the Lokstack custom resource. `deleteDelay` is the time after which the compactor will delete marked chunks. boltdb-shipper indexes are refreshed from the shared store on components using it (querier and ruler) at a specific interval. This means deleting chunks instantly could lead to components still having reference to old chunks and so they could fail to execute queries. Having a delay allows for components to refresh their store and so remove gracefully their reference of those chunks. It also provides a short window of time in which to cancel chunk deletion in the case of a configuration mistake. `DeleteWorkerCount` specifies the maximum quantity of goroutine workers instantiated to delete chunks. - [https://grafana.com/docs/loki/latest/operations/storage/retention/#retention-configuration](https://grafana.com/docs/loki/latest/operations/storage/retention/#retention-configuration). The operator instantiates loki cluster of different t-shirt sizes. A pre-determined default value of `DeleteWorkerCount` per t-shirt size cluster is set to avoid issues like large number of goroutine workers instantiated on small clusters. The user cannot input `DeleteWorkerCount`.  \nRetention period is configured within the limits\\_config configuration section.  \nThere are two ways of setting retention policies:",
        "favicon": "https://favicons.linkup.so?domain=loki-operator.dev",
        "type": "text"
      },
      {
        "name": "Log retention | Grafana Loki documentation",
        "url": "https://grafana.com/docs/loki/latest/operations/storage/retention/",
        "content": "Single store TSDB and single store BoltDB require 24h index period. retention_enabled should be set to true. Without this, the Compactor will only compact tables. delete_request_store should be set to configure the store for delete requests.\nIn order to enable the retention support, the Table Manager needs to be configured to enable deletions and a retention period. Please refer to the table_manager section of the Loki configuration reference for all available options. Alternatively, the table-manager.retention-period and table-manager.retention-deletes-enabled command line flags can be used.\nAlternatively, if the BoltDB Shipper is configured for the index store, you can enable Log entry deletion to delete log entries from a specific stream. Example configuration with GCS with a 28 day retention: ... schema_config: configs: - from: 2018-04-15 store: tsdb object_store: gcs schema: v13 index: prefix: loki_index_ period: 24h storage_config: tsdb_shipper: active_index_directory: /loki/index cache_location: /loki/index_cache gcs: bucket_name: GCS_BUCKET_NAME limits_config: max_query_lookback: 672h # 28 days retention_period: 672h # 28 days compactor: working_directory: /data/retention delete_request_store: gcs retention_enabled: true\nDescribes how Loki implements storage retention and retention configuration options.\ncompactor: working_directory: /data/retention compaction_interval: 10m retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 delete_request_store: gcs schema_config: configs: - from: \"2020-07-31\" index: period: 24h prefix: index_ object_store: gcs schema: v13 store: tsdb storage_config: tsdb_shipper: active_index_directory: /data/index cache_location: /data/index_cache gcs: bucket_name: loki ... Retention is only available if the index period is 24h. Single store TSDB and single store BoltDB require 24h index period. retention_enabled should be set to true. Without this, the Compactor will only compact tables. delete_request_store should be set to configure the store for delete requests.",
        "favicon": "https://favicons.linkup.so?domain=grafana.com",
        "type": "text"
      },
      {
        "name": "purge - why my Loki log retention is not working? - Stack Overflow",
        "url": "https://stackoverflow.com/questions/73138278/why-my-loki-log-retention-is-not-working",
        "content": "\"12h\" old logs are not deleting even after specifying \"retention\\_period: 12h\" due to this I'm facing storage issue because, log are not deleting. please help me with config  \n```\nloki:\nenabled: true\nisDefault: true\ntable_manager:\nretention_deletes_enabled: true\nretention_period: 12h\n```  \nI have the following configurations related to log retention:  \n```\n...\ncompactor:\n...\nretention_enabled: true\n...\nlimits_config:\n...\nretention_period: 90d\n...\n```  \nSee more about Loki configuration in the documentation [here](https://grafana.com/docs/loki/latest/configuration/)  \n[Grafana Loki Storage Retention](https://grafana.com/docs/loki/latest/operations/storage/retention/)  \n```\n\nBy default, when table_manager.retention_deletes_enabled or\ncompactor.retention_enabled flags are not set, then logs sent to\nLoki live forever.\n```  \n**The minimum retention period is 24h**.  \nThe retention period **must be a multiple** of the *index* and chunks table `period`, configured in the `period_config` block.  \nSample config:  \n```\nschema_config:\nconfigs:\n- from: 2020-10-24\nstore: boltdb-shipper\nobject_store: filesystem\nschema: v11\nindex:\nprefix: index_\nperiod: 24h\n\ntable_manager:\nretention_deletes_enabled: true\nretention_period: 24h\n```  \nThe other way to setup the retention in Grafana Loki is via the Compactor.  \n```\nlimits_config:\nretention_period: 168h\ncompactor:\nworking_directory: /data/retention\ncompaction_interval: 10m\nretention_enabled: true\nretention_delete_delay: 2h\nretention_delete_worker_count: 150\n\nschema_config:\nconfigs:\n- from: 2020-10-24\nstore: boltdb-shipper\nobject_store: filesystem\nschema: v11\nindex:\nprefix: index_\nperiod: 24h\n```  \nNote that **retention is only available if the index period is 24h**.  \nTo purge data immediately you could set the `retention_delete_delay` interval to `1m`. This is a delay after which chunks will be fully deleted during retention.  \nDo not forget to restart Loki after changing the configuration:  \n```\nsystemctl restart loki\nsystemctl status loki\n```  \n`This is a delay after which chunks will be fully deleted during retentionretention_delete_delaylimits_config\\retention_period`  \nFrom the [Grafana storage documentation](https://grafana.com/docs/loki/latest/storage/#retention):  \n> With the exception of the filesystem chunk store, Loki will not delete old chunk stores. This is generally handled instead by configuring TTLs (time to live) in the chunk store of your choice (bucket lifecycles in S3/GCS, and TTLs in Cassandra). Neither will Loki currently delete old data when your local disk fills when using the filesystem chunk store \u2013 deletion is only determined by retention duration.  \nThis quote should answer your question. You need to configure bucket lifecycles in your storage provider to be able to delete chunk stores.",
        "favicon": "https://favicons.linkup.so?domain=stackoverflow.com",
        "type": "text"
      },
      {
        "name": "Configure Loki Log Retention and the Compactor So Storage Stops Growing | Stack Harbor Knowledge Base",
        "url": "https://stackharbor.com/en/knowledge-base/loki-retention-compactor-config/",
        "content": "If your store keeps growing and nothing ages out, the fix is almost always the same three things: run exactly one compactor, set retention_enabled: true in its config, and give it a retention_period to enforce.\nChunks land in your object store and stay there forever until you turn on retention, and retention in Loki is a job the compactor does \u2014 not a bucket lifecycle rule, not the ingesters, not a cron. If your store keeps growing and nothing ages out, the fix is almost always the same three things: run exactly one compactor, set retention_enabled: true in its config, and give it a retention_period to enforce. This guide walks the whole path \u2014 enabling the compactor block, setting a global retention window, layering per-stream and per-tenant rules on top, restarting, and then proving from the logs and metrics that expired chunks are being marked and swept.\nWhen more than one retention_stream rule matches a stream, the one with the highest priority wins, and ties fall back to the longest period. Separately, delete_request_store tells the compactor where to persist tenant-issued delete requests (the Loki delete API), which the same loop processes alongside age-based retention. 1. Enable the compactor with retention on. Add or edit the compactor block.\nSet up Loki retention end to end: enable the compactor, set a global retention_period, add per-stream and per-tenant rules, and confirm old chunks are actually deleted from the store.\nRun a single compactor with retention enabled, on a schedule you control. Set a global retention_period so every stream ages out at a known age. Override that per stream and per tenant, with selectors and priority you can reason about. Read the compactor\u2019s logs and loki_compactor_* metrics to confirm retention is actually running.",
        "favicon": "https://favicons.linkup.so?domain=stackharbor.com",
        "type": "text"
      },
      {
        "name": "[Loki] Issues with Implementing Retention Policy and Excluding Specific Namespaces/Pods from Logging \u00b7 Issue #2999 \u00b7 grafana/helm-charts",
        "url": "https://github.com/grafana/helm-charts/issues/2999",
        "content": "shared_store: s3 compaction_interval: 1h retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 #retention_deletes_enabled: true #retention_period: 72h # 3 days \u00b7 schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: s3 schema: v11 index: prefix: index_ period: 24h \u00b7 I am looking to optimize our logging by excluding certain namespaces and pods from being logged to reduce storage and processing overhead. I am unsure whether this configuration should be applied in Loki or Promtail, and how to properly configure it.\nI am encountering issues with configuring the retention policy in Loki, resulting in pods entering a CrashLoopBackOff state after the implementation. Additionally, I am seeking guidance on how to exclude certain namespaces and pods from having their logs collected and stored in an S3 bucket. Loki Version: 2.9.4 \u00b7 failed parsing config: /etc/loki/config/config.yaml: yaml: unmarshal errors: line 17: field retention_deletes_enabled not found in type compactor.Config line 19: field retention_period not found in type compactor.Config.\nKindly help! I am encountering issues with configuring the retention policy in Loki, resulting in pods entering a CrashLoopBackOff state after the implementation. Additionally, I am seeking guidance on how to exclude certain namespaces a...\nPlease find my configuration. storage: bucketNames: chunks: xxxxx-loki-chunks ruler: xxxxx-loki-ruler admin: xxxxx-admin type: s3 s3: region: af-south-1 s3ForcePathStyle: false insecure: false \u00b7 compactor: apply_retention_interval: 1h working_directory: /data/loki/compactor <-- after commenting retention_delete_enabled=true I face issues creating this dir structure and I'm not sure where exactly to create this dir.\nshared_store: s3 compaction_interval: 1h retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 #retention_deletes_enabled: true #retention_period: 72h # 3 days \u00b7 schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: s3 schema: v11 index: prefix: index_ period: 24h \u00b7 I am looking to optimize our logging by excluding certain namespaces and pods from being logged to reduce storage and processing overhead. I am unsure whether this configuration should be applied in Loki or Promtail, and how to properly configure it.",
        "favicon": "https://favicons.linkup.so?domain=github.com",
        "type": "text"
      },
      {
        "name": "LOKI Retention Policy and Chunk Deletion Issue - Grafana Loki - Grafana Labs Community Forums",
        "url": "https://community.grafana.com/t/loki-retention-policy-and-chunk-deletion-issue/159057",
        "content": "### **Our Setup**  \n- **Cluster:**2-node Loki setup (Ingestor / Distributor / Querier roles)\n- **Promtail:**Managed by our DevOps team (runs as a DaemonSet \u2014 we don\u2019t have admin access, so a bit of dependency there)\n- **Storage:**Everything is on local filesystem mounts:`/data/loki/wal /data/loki/chunks /data/loki/index`\n- **Retention:**Configured in`loki-config.yaml`for**5 days**  \n- We\u2019ve tuned parameters like chunk block size (2 MB) to optimize space usage\n- **Deletion API:**We\u2019ve also tried using the`curl`deletion API, but understand it only*marks*data for deletion in the index \u2014 it doesn\u2019t immediately remove chunk files.\n### **Questions**  \n- Why would Loki be updating chunk file modification timestamps?\n- How can we make sure chunks are  \n*actually*deleted after retention (and not just marked)?\n- Is there a better approach to enforce retention-based cleanup when running Loki on filesystem storage \u2014 not in Kubernetes and not using object storage?\n- Could something like WAL replay or a background process (e.g. compactor) be touching those files and resetting timestamps?  \n**loki version 2.9.3**\n**loki yaml:**\ningester:\nlifecycler:\nring:\nkvstore:\nstore: memberlist\nreplication\\_factor: 1\nfinal\\_sleep: 0s\nchunk\\_idle\\_period: 5m\nchunk\\_retain\\_period: 30s\nmax\\_transfer\\_retries: 0\nchunk\\_block\\_size: 2097152\nchunk\\_encoding: snappy\nmax\\_chunk\\_age: 2h  \ndistributor:  \nring:\nkvstore:\nstore: memberlist  \nquerier:  \nengine:\ntimeout: 1m\nquery\\_ingesters\\_within: 2h  \nquery\\_range:  \nalign\\_queries\\_with\\_step: true\nmax\\_retries: 5\nparallelise\\_shardable\\_queries: true  \nfrontend:  \nlog\\_queries\\_longer\\_than: 5s\ncompress\\_responses: true  \nfrontend\\_worker:  \nfrontend\\_address: X.X.X.X  \nschema\\_config:  \nconfigs:\n\\- from: 2020-10-24\nstore: boltdb-shipper\nobject\\_store: filesystem\nschema: v11\nindex:\nprefix: index_\nperiod: 24h  \nstorage\\_config:  \nboltdb\\_shipper:\nactive\\_index\\_directory: /data/loki/index\ncache\\_location: /data/loki/index\\_cache\nshared\\_store: filesystem  \nfilesystem:  \ndirectory: /data/loki/chunks  \ncompactor:  \nworking\\_directory: /data/loki/compactor\nshared\\_store: filesystem\ncompactor:  \nworking\\_directory: /data/loki/compactor\nshared\\_store: filesystem\nretention\\_enabled: true\ndelete\\_request\\_cancel\\_period: 1m\ndeletion\\_mode: filter-and-delete  \nlimits\\_config:  \ningestion\\_burst\\_size\\_mb: 500\ningestion\\_rate\\_mb: 1024\ningestion\\_rate\\_strategy: global\nmax\\_cache\\_freshness\\_per\\_query: 10m\nmax\\_global\\_streams\\_per\\_user: 50000\nmax\\_query\\_parallelism: 128\nper\\_stream\\_rate\\_limit: 500MB\nper\\_stream\\_rate\\_limit\\_burst: 500MB\nreject\\_old\\_samples: true\nreject\\_old\\_samples\\_max\\_age: 120h\nretention\\_period: 5d\nsplit\\_queries\\_by\\_interval: 15m\nenforce\\_metric\\_name: false\nmax\\_entries\\_limit\\_per\\_query: 80000\nallow\\_deletes: true  \nchunk\\_store\\_config:  \nmax\\_look\\_back\\_period: 0s  \ntable\\_manager:  \nretention\\_deletes\\_enabled: true\nretention\\_period: 5d**curl api:**\ncurl -X DELETE \u201chttp://&lt;loki\\_host&gt;:3100/loki/api/v1/delete\u201d\n\\-H \u201cX-Scope-OrgID: &lt;tenant\\_id&gt;\u201d\n\u2013data-urlencode \u2018query={job=\u201cvarlogs\u201d} |~ \u201csensitive\\_data\u201d\u2019\n\u2013data-urlencode \u2018start=&lt;start\\_timestamp&gt;\u2019\n\u2013data-urlencode \u2018end=&lt;end\\_timestamp&gt;\u2019  \nIf anyone has seen this behavior or has tips on how to manage chunk cleanup and ensure retention works effectively in a non-K8s setup, please share your thoughts.  \nThanks in advance!",
        "favicon": "https://favicons.linkup.so?domain=community.grafana.com",
        "type": "text"
      },
      {
        "name": "How Grafana Loki retention works - Grafana Loki - Grafana Labs Community Forums",
        "url": "https://community.grafana.com/t/how-grafana-loki-retention-works/67754",
        "content": "I would like to set up retention for logs stored in Grafana Loki and I am not sure how the retention really works. There is more period time to configure and from documentation, it is not so clear to understand how whole process of retention works.  \nThis is my Loki configuration:  \n`common: instance_interface_names: - \"lo\" path_prefix: /loki storage: filesystem: chunks_directory: /loki/chunks rules_directory: /loki/rules replication_factor: 1 ring: instance_interface_names: - \"lo\" kvstore: store: inmemory compactor: working_directory: /loki/compactor compaction_interval: 5m retention_enabled: true retention_delete_delay: 5m delete_request_cancel_period: 1m limits_config: retention_period: 10m schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: filesystem schema: v11 index: prefix: index_ period: 24h chunks: prefix: chunk_ period: 168h`  \nCompactor should remove old data based on this configuration. This is how I understand config file:  \n- `compactor.compaction_interval`- every 5 minutes is run compactor process\n- `compactor.retention_enabled`- enable to apply retention policy\n- `compactor.retention_delete_delay`- this is the time Compactor is waiting to delete chunks after`retention_period`exceed.\n- `compactor. delete_request_cancel_period`- allow cancellation of delete request made by API. I think, not used in my case at all.\n- `limits_config. retention_period`- this is time after which data should be deleted. According to doc -`The minimum retention period is 24h.`  \nAccording to the configuration, I would expect to have logs for last 15 minutes:  \n- **0:00**- first log arrives\n- **0:05**- compactor is running and because of retention 10 min, no logs are meant to be deleted\n- **0:10**- compactor is running again and because of retention 10 min, my first log should be assign for deletion. Because of retention delay 5min, it is still not deleted at this moment.\n- **0:15**- compactor is running for third time. From the second run, there has been log to be deleted and retention delay 5 min has expired, that means this log can be deleted now.  \nThis is just an example of ideal time schedule. I understand there is async processes and timing is not so precise so lets say, according to my logic, this could be done in 30-40 minutes.  \nI am running this configuration and I can see logs for last 2,5 hours:  \nFirst log at  \n**20:43**and last one ad  \n**23:00**. This is quite long according to my estimation. I do not want to use it for this short period of time but how can I estimate log retention for example for 3 days, 1 week or 2 months?  \n**Questions:**  \n1. How is retention calculated and processed?\n2. Do I understand Loki configuration parameters properly?\n3. How does retention differs from `chunk.period`and`index.period`under`schema_config.config`?",
        "favicon": "https://favicons.linkup.so?domain=community.grafana.com",
        "type": "text"
      },
      {
        "name": "Log and Metric Retention in Loki and Prometheus | by Faith Sodipe | DevOps.dev",
        "url": "https://blog.devops.dev/log-and-metric-retention-in-loki-and-prometheus-57e81bbad023?gi=780056f55dc2",
        "content": "Compactor: provides explicit ... specifies the frequency of operation, retention_enabled determines if the compactor is allowed to perform retention operations. retention_delete_delay tells the compactor to wait x amount ...\nConfiguring Retention Policies Loki supports retention only when using the boltdb-shipper or tsdb block storage engine alongside compactor. This compactor is responsible for enforcing the deletion of old logs and merges index files while the boltdb-shipper/tsdb manages the storage of log index efficiently. Here is a minimal loki-config.yaml setup to enable retention.\nCompactor: provides explicit instructions on how the compactor should go about its clearing and forwarding business. compaction_interval specifies the frequency of operation, retention_enabled determines if the compactor is allowed to perform retention operations. retention_delete_delay tells the compactor to wait x amount of time (sec, min , hrs, month, year etc) after the set retention period before swinging into action. Verification and Testing Since my application uses docker, all my services such as Prometheus, Grafana and Loki itself have been configured to run in the container.\nLog and Metric Retention in Loki and Prometheus When working with observability tools such as Grafana, Loki and Prometheus, storage management is crucial to performance, cost, and compliance. By \u2026\nWhen working with observability tools such as Grafana, Loki and Prometheus, storage management is crucial to performance, cost, and\u2026",
        "favicon": "https://favicons.linkup.so?domain=blog.devops.dev",
        "type": "text"
      }
    ]
  }
}