edsiper · Nov 1, 2026 · coderabbitai · Nov 1, 2026 · coderabbitai · Nov 1, 2026
@@ -139,6 +139,17 @@
   * [Content modifier](pipeline/processors/content-modifier.md)
   * [Cumulative to delta](pipeline/processors/cumulative-to-delta.md)
   * [Filters as processors](pipeline/processors/filters.md)
+  * [Fluent Typed Scripting (FTS)](pipeline/processors/fluent-typed-scripting.md)
+    * [Quickstart](pipeline/processors/fluent-typed-scripting/quickstart.md)
+    * [Configuration](pipeline/processors/fluent-typed-scripting/configuration.md)
+    * [Typed values](pipeline/processors/fluent-typed-scripting/typed-values.md)
+    * [Logs](pipeline/processors/fluent-typed-scripting/logs.md)
+    * [Metrics](pipeline/processors/fluent-typed-scripting/metrics.md)
+    * [Traces](pipeline/processors/fluent-typed-scripting/traces.md)
+    * [Common helpers](pipeline/processors/fluent-typed-scripting/common-helpers.md)
+    * [Errors and limits](pipeline/processors/fluent-typed-scripting/errors-and-limits.md)
+    * [FTS and the Lua filter](pipeline/processors/fluent-typed-scripting/lua-filter.md)
+    * [Recipes](pipeline/processors/fluent-typed-scripting/recipes.md)
   * [Labels](pipeline/processors/labels.md)
   * [Metrics selector](pipeline/processors/metrics-selector.md)
   * [OpenTelemetry envelope](pipeline/processors/opentelemetry-envelope.md)

@@ -8,6 +8,8 @@
 
 The _Lua_ filter lets you modify incoming records (or split one record into multiple records) using custom [Lua](https://www.lua.org/) scripts.
 
+For policies that need typed telemetry values, metric or span transformations, or per-item rollback, see [Fluent Typed Scripting (FTS)](../processors/fluent-typed-scripting.md) and the [FTS and Lua filter comparison](../processors/fluent-typed-scripting/lua-filter.md).
+
 A Lua-based filter requires two steps:
 
 1. Configure the filter in the main configuration.

@@ -15,6 +15,7 @@ Only [YAML configuration files](../administration/configuring-fluent-bit/yaml.md
 Fluent Bit offers the following processors:
 
 - [Content modifier](./processors/content-modifier.md): Manipulate the content, metadata, and attributes of logs and traces.
+- [Fluent Typed Scripting (FTS)](./processors/fluent-typed-scripting.md): Write policies in Lua with typed access to logs, metrics, and traces.
 - [Labels](./processors/labels.md): Add, update, or delete metric labels.
 - [Metrics selector](./processors/metrics-selector.md): Choose which metrics to keep or discard.
 - [OpenTelemetry envelope](./processors/opentelemetry-envelope.md): Transform logs into an OpenTelemetry-compatible format.

@@ -0,0 +1,34 @@
+# Fluent Typed Scripting (FTS)
+
+{% hint style="info" %}
+**Supported event types:** `logs`, `metrics`, `traces`
+{% endhint %}
+
+Fluent Typed Scripting (FTS) lets you write policies in Lua with typed access to telemetry. Its configuration name is `fts`. Use it to redact structured fields, normalize log content and OpenTelemetry context, transform metric families, or edit spans.
+
+_Typed_ refers to the telemetry values exposed to the policy, including maps, arrays, binary data, exact 64-bit integers, and nanosecond timestamps. FTS doesn't add static typing to Lua. Type checks happen at runtime.
+
+## Availability
+
+FTS is included in the development source for Fluent Bit 5.2.0 and hasn't shipped in a stable release yet. It isn't included in the 5.1 release series. Use a development build with FTS enabled; don't assume a container tagged `latest` includes it. These pages describe the current implementation on `master`. Check your binary with `fluent-bit --version` and run the [quickstart](fluent-typed-scripting/quickstart.md) to confirm that it contains `fts`.
+
+For a native source build, enable `FLB_PROCESSOR_FTS`, `FLB_LUAJIT`, and `FLB_YYJSON`. FTS defaults to interpreted execution even when built with LuaJIT. See [Build and install](../../installation/downloads/source/build-and-install.md) for general build instructions.
+
+The experimental browser WASM profile also includes FTS when `FLB_WASM_LUA=ON`, the profile default. It uses portable Lua 5.4 and rejects `jit: true`. Setting `FLB_WASM_LUA=OFF` disables both FTS and the Lua filter in that profile. This experimental profile is separate from native support and from the [Wasm filter](../filters/wasm.md).
+
+## Choose FTS
+
+Choose FTS when a transformation needs typed values, per-item rollback, or programmable access to logs, metrics, and traces. Prefer a dedicated processor when its existing operations meet your needs. For an existing Lua filter policy, review [FTS and the Lua filter](fluent-typed-scripting/lua-filter.md) before migrating: their callback and value models differ.
+
+Each signal uses its own processor entry and script under `processors.logs`, `processors.metrics`, or `processors.traces`. A callback receives one log record, one metric family, or one span, respectively. Each processor instance has its own Lua state. FTS doesn't join signals or share globals between those instances.
+
+Processors require [YAML configuration](../../administration/configuring-fluent-bit/yaml.md). Attach FTS to an input or output and place it in the order in which transformations should run. See [Processors](../processors.md) for pipeline placement.
+
+## Get started and reference
+
+1. Run your first transformation with the [Quickstart](fluent-typed-scripting/quickstart.md).
+2. Configure the policy and execution limits with [Configuration](fluent-typed-scripting/configuration.md).
+3. Learn the value model in [Typed values](fluent-typed-scripting/typed-values.md).
+4. Use the signal references for [Logs](fluent-typed-scripting/logs.md), [Metrics](fluent-typed-scripting/metrics.md), and [Traces](fluent-typed-scripting/traces.md).
+5. Look up [Common helpers](fluent-typed-scripting/common-helpers.md) and [Errors and limits](fluent-typed-scripting/errors-and-limits.md).
+6. Adapt the complete [Recipes](fluent-typed-scripting/recipes.md).
@@ -0,0 +1,57 @@
+# Common helpers
+
+Load the FTS module with `local fb = require("fluentbit")`. The following helpers operate on typed values. For signal-specific helpers, see [Logs](logs.md#log-helpers) and [Metrics](metrics.md#metric-helpers). See [Typed values](typed-values.md#constructors-and-conversions) for constructors.
+
+## Redact structured fields
+
+`fb.redact(value, keys, replacement)` recursively replaces values of matching keys in nested maps and arrays. `keys` is a Lua table of key names. Matching is case-insensitive. The default replacement is `[REDACTED]`.
+
+```lua
+fb.redact(log.body, {"authorization", "password", "access_token"})
+```
+
+For a custom replacement:
+
+```lua
+fb.redact(log.body, {"password"}, "removed")
+```
+
+This helper doesn't search inside strings. It won't remove a password embedded in `message: "password=example-secret"`. Write and test a separate string policy for known text formats, or remove the text field when appropriate. Keep `error_mode: drop` for redaction policies when forwarding an unmodified failed item would expose sensitive data.
+
+## Keep selected keys
+
+`fb.keep_keys(map, keys)` removes every key not listed in the Lua table `keys`. Matching is case-sensitive. This helper acts on the supplied map only; it isn't recursive.
+
+```lua
+fb.keep_keys(log.body, {"message", "level", "service.name"})
+```
+
+## Parse JSON
+
+`fb.parse_json(text)` returns a typed value, preserving integer widths. Input is limited to 1 MiB and nesting to 64 levels. Duplicate keys and invalid JSON raise errors. A valid JSON document can produce a scalar or array, so check the result before assigning a log body.
+
+```lua
+local text = log.body:get_string("message")
+if text then
+    local ok, parsed = pcall(fb.parse_json, text)
+    if ok and parsed:type() == "map" then
+        log.body = parsed
+    else
+        return fb.DROP
+    end
+end
+```
+
+This fragment belongs in `process(log)` with `fb` loaded. Catching a parse error with `pcall` lets the policy choose a result; it doesn't roll back earlier edits automatically. An uncaught error invokes the processor's rollback and `error_mode` behavior.
+
+## Parse time
+
+`fb.parse_time(text)` parses RFC 3339 text and returns a Lua table `{sec, nsec}` for `log.timestamp`. It accepts `Z` or numeric timezone offsets and up to nine fractional digits. Invalid dates, leap seconds, excess fractional precision, and values outside `0..4294967293` epoch seconds raise errors. The supported calendar-year input range is 1969 through 2107; the resulting epoch seconds must still fit the event range and the platform's timestamp range.
+
+```lua
+log.timestamp = fb.parse_time("2026-01-02T03:04:05.123456789Z")
+```
+
+## Callback constants
+
+`fb.KEEP` commits edits and `fb.DROP` removes the item. Returning nothing also commits edits. These constants aren't the numeric return codes of the Lua filter. See [Errors and limits](errors-and-limits.md#callback-results).
@@ -0,0 +1,67 @@
+# Configuration
+
+FTS uses the configuration name `fts`. Attach a processor entry under the signal it processes. See the [quickstart](quickstart.md) for a complete pipeline.
+
+## Configuration parameters
+
+| Key | Description | Default |
+| --- | ----------- | ------- |
+| `call` | Name of the Lua callback. The script must define this function before initialization finishes. | `process` |
+| `code` | Inline Lua policy. Set exactly one of `code` or `script`. | _none_ |
+| `error_mode` | Action after a policy failure: `drop` removes the item; `keep` forwards the original item without edits. Values are case-sensitive. | `drop` |
+| `instruction_limit` | Lua instruction budget for each callback and script initialization. With JIT disabled, the range is `1000` through `2147483647`. With JIT enabled, must be `0`. | `1000000` |
+| `jit` | Enable LuaJIT compilation. Requires a LuaJIT runtime and `instruction_limit: 0`. | `false` |
+| `memory_limit` | Maximum policy memory in bytes, including the Lua heap and retained typed values. Minimum: `262144` bytes. | `32M` |
+| `output_limit` | Maximum serialized size of a processed batch. Minimum: `65536` bytes. See the signal-specific failure behavior in [Errors and limits](errors-and-limits.md#output-limit). | `64M` |
+| `script` | Path to a Lua policy file. Set exactly one of `script` or `code`. | _none_ |
+
+Size suffixes use decimal multipliers: `64K` is 64,000 bytes, less than the minimum `output_limit`. Use `65536` or a larger value. `32M` is 32,000,000 bytes and `64M` is 64,000,000 bytes.
+
+FTS has no `workers` parameter. Its execution follows the input or output to which it is attached.
+
+## Inline policies
+
+Use a YAML block scalar for an inline policy:
+
+```yaml
+pipeline:
+  inputs:
+    - name: dummy
+      samples: 1
+      processors:
+        logs:
+          - name: fts
+            code: |
+              function process(log)
+                  log.body:set("environment", "production")
+              end
+  outputs:
+    - name: stdout
+      match: '*'
+      format: json_lines
+```
+
+## Separate signals
+
+For an input that receives multiple signals, use separate entries and scripts. The following fragment belongs inside an input or output configuration:
+
+```yaml
+processors:
+  logs:
+    - name: fts
+      script: logs.lua
+  metrics:
+    - name: fts
+      script: metrics.lua
+  traces:
+    - name: fts
+      script: traces.lua
+```
+
+Each script can define `process`, because each instance has its own Lua state. A logs script receives a log, a metrics script receives a family, and a traces script receives a span. There is no shared callback that receives all three signals.
+
+## Invalid combinations
+
+Startup fails when both `script` and `code` are set, neither is set, the callback is missing, a limit is invalid, or the script fails during initialization. `error_mode` handles processing failures; it doesn't make an invalid configuration usable.
+
+To enable JIT on a native LuaJIT build, set both `jit: true` and `instruction_limit: 0`. Setting only one fails validation. Portable Lua, including the experimental browser WASM runtime, rejects JIT. See [Runtime restrictions](errors-and-limits.md#runtime-restrictions) before choosing these settings.
@@ -0,0 +1,62 @@
+# Errors and limits
+
+FTS policies are trusted configuration. The restricted runtime reduces available operations, but its sandbox isn't a security boundary. Review policies before deploying them.
+
+## Callback results
+
+| Result | Effect |
+| --- | --- |
+| No return, `nil`, or `fb.KEEP` | Commit the item's edits. |
+| `fb.DROP` | Remove the item. Metric emissions remain. |
+| Uncaught Lua error, invalid return, or invalid telemetry edit | Roll back edits and apply `error_mode`. |
+
+Transactions cover one log record, metric family, or span, including its resource and scope edits. For logs, body, metadata, context, and timestamp edits commit or roll back together. Metric emissions are discarded on an error. A transaction doesn't include other callbacks or restore Lua globals modified by the policy.
+
+With `error_mode: drop`, the failed item is removed. With `error_mode: keep`, the original item is forwarded without the policy's edits. For a redaction policy, `keep` can forward secrets. Leave the default `drop` when that outcome is unacceptable.
+
+A caught error, such as a failed `pcall(fb.parse_json, text)`, lets the callback continue. Earlier edits remain unless the callback ultimately fails. Instruction budget exhaustion can't be caught to continue an endless loop.
+
+## Memory and instruction limits
+
+The [configuration reference](configuration.md#configuration-parameters) lists defaults and valid ranges.
+
+`memory_limit` accounts for the Lua heap and typed values held by the policy. It isn't a limit on the whole Fluent Bit process, input chunks, or output buffers. Retained globals, copies, and metric/span snapshots consume this budget.
+
+`instruction_limit` counts Lua instructions during script initialization and each callback, using a hook in steps of 1,000 instructions. It isn't an elapsed-time deadline. Lua string patterns execute in C and aren't interrupted by the instruction hook. Avoid patterns whose runtime can grow without a defined bound on untrusted input.
+
+`jit: true` requires `instruction_limit: 0`, removing the instruction budget. Use it only when you control policy execution time. JIT isn't available in the portable Lua runtime. No throughput or latency improvement is guaranteed by these settings.
+
+## Output limit
+
+`output_limit` bounds serialized output from a processed batch. Failure behavior depends on the signal:
+
+| Signal | Behavior when transformed output doesn't fit |
+| --- | --- |
+| Logs | The whole batch fails. `drop` discards the batch; `keep` forwards the original batch. |
+| Metrics and traces | The overflowing item follows `error_mode`. The whole batch fails if retained original items don't fit either. |
+
+Internal failures, such as a batch that can't be decoded, apply `error_mode` to the whole batch.
+
+A processing batch isn't the same as a callback invocation: one batch can invoke the callback many times. Small chunks are processed as one batch. Larger log buffers, including chunks passed to output processors, are processed in batches of about 64 KiB or 1,024 records. Groups aren't split across batches, so a group can exceed those targets. See [Logs](logs.md#opentelemetry-context) for context behavior when groups span appends.
+
+## Value and format limits
+
+- Typed values support at most 64 levels of nesting.
+- `parse_json` input is limited to 1 MiB and rejects duplicate keys.
+- OpenTelemetry strings and keys must be valid UTF-8 without NUL characters. Use typed bytes for binary data.
+- OpenTelemetry integer fields have their own signed or unsigned ranges. A successful typed constructor doesn't guarantee that an arbitrary signal field accepts that value.
+- Invalid metric layouts, duplicate series/windows introduced by edits, invalid IDs, and invalid timestamps can fail a policy. See [Metrics](metrics.md) and [Traces](traces.md).
+
+## Runtime restrictions
+
+Only `require("fluentbit")` is available. Filesystem, process, FFI, debug, coroutine, and general module-loading libraries aren't available. The runtime also removes `print`, `xpcall`, `getfenv`, `setfenv`, `module`, and dynamic code-loading functions such as `load`, `loadfile`, `loadstring`, and `dofile`.
+
+Policies can use the restricted base, `math`, `string`, and `table` functions. `collectgarbage` accepts only `collect`, `count`, and `step`. Each processor instance has its own Lua state; globals persist across callbacks within that instance.
+
+The experimental browser WASM profile uses the portable Lua 5.4 interpreter with the same typed-value model and policy limits. It rejects JIT. See [Availability](../fluent-typed-scripting.md#availability) for the build distinction.
+
+## Diagnostics
+
+Each batch logs at most five policy errors plus a summary count. A low number of error messages doesn't imply that only five records failed. The processor error metric counts batches with handled policy failures, and drop metrics count removed records.
+
+If output disappears, check policy errors, expected field types, `error_mode`, and all three budgets. A startup failure instead points to configuration, script loading, initialization limits, or an unavailable runtime feature.