Implement hybrid mode for Store Gateway

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 * [FEATURE] Ruler: Add experimental support for federated rule groups. A rule group listing tenants in its `source_tenants` field is evaluated against those tenants while the resulting series and alerts are written to the tenant owning the rule group. Enabled with `-ruler.enable-federated-rules` (requires `-tenant-federation.enabled`), and restricted to selected tenants with `-ruler.allowed-federated-tenants` and `-ruler.disallowed-federated-tenants`. #7828
 * [FEATURE] Distributor: Add experimental OTLP metrics ingestion over gRPC (`opentelemetry.proto.collector.metrics.v1.MetricsService/Export`) on the gRPC server port. Enable it with `-distributor.otlp.grpc-enabled`. The tenant is read from the `X-Scope-OrgID` gRPC metadata, and the request size is limited by `-server.grpc-max-recv-msg-size-bytes`. #7873
+* [FEATURE] StoreGateway: Add a hybrid mode to the parquet bucket store (`-blocks-storage.bucket-store.bucket-store-type=parquet`), which serves blocks not yet converted to Parquet from TSDB blocks so they stay queryable. Hybrid mode is the default and requires the bucket index. Set `-blocks-storage.bucket-store.parquet-fallback-disabled=true` to keep the previous Parquet-only behavior. #7689
 * [ENHANCEMENT] Ruler: Add rule information to query parameters sent to the Query Frontend when `-ruler.frontend-address` is configured, so the Query Frontend query stats logs (`-frontend.enabled-ruler-query-stats`) can identify the evaluated rule. In this mode, the Ruler query stats log (`-ruler.query-stats-enabled`) omits the fetched statistics fields, which are always zero. #6539
 * [ENHANCEMENT] gRPC: When `-auth.enabled=true`, a gRPC request without the `X-Scope-OrgID` metadata is now rejected with the `UNAUTHENTICATED` status code instead of `UNKNOWN`. #7873
 * [ENHANCEMENT] Upgrade promql-engine and Thanos to latest. #7889

docs/blocks-storage/querier.md

@@ -2102,10 +2102,20 @@ blocks_storage:
     # CLI flag: -blocks-storage.bucket-store.block-discovery-strategy
     [block_discovery_strategy: <string> | default = "concurrent"]
 
-    # Type of bucket store to use (tsdb or parquet).
+    # Type of bucket store to use (tsdb or parquet). With parquet, blocks
+    # converted to Parquet are served from Parquet files and blocks not yet
+    # converted are served from TSDB blocks, unless
+    # -blocks-storage.bucket-store.parquet-fallback-disabled is set.
     # CLI flag: -blocks-storage.bucket-store.bucket-store-type
     [bucket_store_type: <string> | default = "tsdb"]
 
+    # [Experimental] Only used when the bucket store type is parquet. If true,
+    # the store-gateway serves only blocks converted to Parquet and does not
+    # fall back to TSDB blocks for blocks not converted yet, so it keeps no
+    # local state. If false, the bucket index must be enabled.
+    # CLI flag: -blocks-storage.bucket-store.parquet-fallback-disabled
+    [parquet_fallback_disabled: <boolean> | default = false]
+
     # Max size - in bytes - of a chunks pool, used to reduce memory allocations.
     # The pool is shared across all tenants. 0 to disable the limit.
     # CLI flag: -blocks-storage.bucket-store.max-chunk-pool-bytes

docs/blocks-storage/store-gateway.md

@@ -2155,10 +2155,20 @@ blocks_storage:
     # CLI flag: -blocks-storage.bucket-store.block-discovery-strategy
     [block_discovery_strategy: <string> | default = "concurrent"]
 
-    # Type of bucket store to use (tsdb or parquet).
+    # Type of bucket store to use (tsdb or parquet). With parquet, blocks
+    # converted to Parquet are served from Parquet files and blocks not yet
+    # converted are served from TSDB blocks, unless
+    # -blocks-storage.bucket-store.parquet-fallback-disabled is set.
     # CLI flag: -blocks-storage.bucket-store.bucket-store-type
     [bucket_store_type: <string> | default = "tsdb"]
 
+    # [Experimental] Only used when the bucket store type is parquet. If true,
+    # the store-gateway serves only blocks converted to Parquet and does not
+    # fall back to TSDB blocks for blocks not converted yet, so it keeps no
+    # local state. If false, the bucket index must be enabled.
+    # CLI flag: -blocks-storage.bucket-store.parquet-fallback-disabled
+    [parquet_fallback_disabled: <boolean> | default = false]
+
     # Max size - in bytes - of a chunks pool, used to reduce memory allocations.
     # The pool is shared across all tenants. 0 to disable the limit.
     # CLI flag: -blocks-storage.bucket-store.max-chunk-pool-bytes

docs/configuration/config-file-reference.md

@@ -2799,10 +2799,20 @@ bucket_store:
   # CLI flag: -blocks-storage.bucket-store.block-discovery-strategy
   [block_discovery_strategy: <string> | default = "concurrent"]
 
-  # Type of bucket store to use (tsdb or parquet).
+  # Type of bucket store to use (tsdb or parquet). With parquet, blocks
+  # converted to Parquet are served from Parquet files and blocks not yet
+  # converted are served from TSDB blocks, unless
+  # -blocks-storage.bucket-store.parquet-fallback-disabled is set.
   # CLI flag: -blocks-storage.bucket-store.bucket-store-type
   [bucket_store_type: <string> | default = "tsdb"]
 
+  # [Experimental] Only used when the bucket store type is parquet. If true, the
+  # store-gateway serves only blocks converted to Parquet and does not fall back
+  # to TSDB blocks for blocks not converted yet, so it keeps no local state. If
+  # false, the bucket index must be enabled.
+  # CLI flag: -blocks-storage.bucket-store.parquet-fallback-disabled
+  [parquet_fallback_disabled: <boolean> | default = false]
+
   # Max size - in bytes - of a chunks pool, used to reduce memory allocations.
   # The pool is shared across all tenants. 0 to disable the limit.
   # CLI flag: -blocks-storage.bucket-store.max-chunk-pool-bytes

docs/configuration/v1-guarantees.md

@@ -165,7 +165,8 @@ Currently experimental features are:
     ring configuration
   - Querier: `-querier.parquet-queryable-default-block-store`, `-querier.parquet-queryable-fallback-disabled`,
     the `-querier.parquet-queryable.max-fetched-*` limits and `-querier.parquet-shard-cache-*`
-  - Store Gateway: `-blocks-storage.bucket-store.parquet-query-concurrency`,
+  - Store Gateway: `-blocks-storage.bucket-store.parquet-fallback-disabled`,
+    `-blocks-storage.bucket-store.parquet-query-concurrency`,
     `-blocks-storage.bucket-store.parquet-shard-cache-*` and the
     `-blocks-storage.bucket-store.parquet-labels-cache.*` /
     `-blocks-storage.bucket-store.parquet-row-ranges-cache.*` cache configuration

docs/guides/parquet-mode.md

@@ -142,6 +142,26 @@ querier:
   parquet_queryable_fallback_disabled: false
 ```
 
+### Querying Parquet Through the Store Gateway
+
+As an alternative to the parquet queryable, the store-gateway can serve Parquet files while the querier keeps querying the store-gateway as usual:
+
+```yaml
+blocks_storage:
+  bucket_store:
+    bucket_store_type: parquet
+
+    # Disable fallback to TSDB blocks for blocks not converted to Parquet yet
+    parquet_fallback_disabled: false
+
+    bucket_index:
+      enabled: true
+```
+
+By default the store-gateway runs in hybrid mode: blocks converted to Parquet are served from Parquet files, and blocks not converted yet are served from TSDB blocks, so they stay queryable while the conversion is pending. The store-gateway syncs only the not-yet-converted blocks, so it keeps index-headers on local disk for those blocks only. Hybrid mode requires the bucket index to be enabled.
+
+When `parquet_fallback_disabled` is `true`, the store-gateway serves only Parquet files and keeps no local state. Queries fail for blocks that have not been converted yet, so enable it only when every queried block is available as Parquet.
+
 ### Query Limits for Parquet
 
 Configure query limits specific to parquet operations: