Never drop a line, log CSRF errors where Rails raises them, and log in forked workers

.cspell/project-words.txt

@@ -83,3 +83,8 @@ otps
 crypted
 jwts
 Luhn
+byteslice
+unwritable
+FFFD
+signm
+SIGUSR

README.md

@@ -1,6 +1,6 @@
 # LogStruct
 
-Adds secure JSON structured logging to any Rails app (>= 7.1). Simply add the gem to your Gemfile and add an initializer to configure it. By default, your Rails app prints JSON logs to STDOUT (or to the configured destination when `RAILS_LOG_TO_STDOUT` is set). They're easy to search and filter, you can turn them into metrics and alerts, and they're great for building dashboards in CloudWatch, Grafana, or Datadog.
+Adds secure JSON structured logging to any Rails app (>= 7.1). Simply add the gem to your Gemfile and add an initializer to configure it. In production, your Rails app prints JSON logs to STDOUT. They're easy to search and filter, you can turn them into metrics and alerts, and they're great for building dashboards in CloudWatch, Grafana, or Datadog.
 
 We support all your other favorite gems too, like Sidekiq, Sentry, and Shrine. (And if not, please open a PR!)
 
@@ -53,10 +53,10 @@ Once initialized (and enabled), the gem automatically includes its modules into
 - `ActionMailer::Base` includes error handling and event logging modules
 - We configure `Lograge` for request logging
 - Lograge request logs include request metadata (request_id, source_ip, user_agent, referer without its query string, host, content_type, accept) and custom fields from `lograge_custom_options`
-- A Rack middleware is inserted to catch and log errors, including security violations (IP spoofing, CSRF, blocked hosts, etc.)
+- CSRF and IP spoofing errors are logged as security lines and reported (`error_handling_modes.security_errors`), and Rails still responds to them as usual (422 for a CSRF error). A Rack middleware logs errors that get past Rails' ShowExceptions. Blocked hosts are logged by the host authorization integration.
 - Structured logging is set up for ActiveJob, Sidekiq, Shrine, etc.
 - LogStruct also filters the keys in Rails' `config.filter_parameters`, matched the way Rails matches them (`:passw` filters `user_password`, `"card.number"` filters a nested key by its path, and filter blocks get the parameters), so the same keys are filtered in every log line. Rails' own list is left in place, so `request.filtered_parameters`, `ActiveRecord::Base.filter_attributes` and other Rails features keep filtering, and values Rails has already replaced are logged as `[FILTERED]`. Configure extra sensitive keys via `LogStruct.config.filters`.
-- When `RAILS_LOG_TO_STDOUT` is set, we log to STDOUT only. Otherwise, we log to STDOUT by default without adding a file appender to avoid duplicate logs.
+- In production (and any environment that isn't in `config.local_environments`), we log to STDOUT only, with no log file to rotate or fill the disk. In development we log to STDOUT and Rails' log file, and in test to the log file only. With `RAILS_LOG_TO_STDOUT` set, both log to STDOUT only.
 
 ### Default behavior by process type
 

docs/app/docs/configuration/page.tsx

@@ -175,15 +175,30 @@ end`}
         docs for more information.
       </Callout>
 
+      <HeadingWithAnchor id="limits">Limits</HeadingWithAnchor>
+      <p className="text-neutral-600 dark:text-neutral-300 mb-4">
+        LogStruct writes only part of very large values, so a huge request body or Hash can&apos;t
+        slow down the thread that writes your logs. Long strings are scrubbed before they&apos;re
+        cut.
+      </p>
+
+      <RubyCodeExample name="limits_configuration" />
+
       <HeadingWithAnchor id="error-handling-configuration">
         Error Handling Configuration
       </HeadingWithAnchor>
       <p className="text-neutral-600 dark:text-neutral-300 mb-4">
-        LogStruct provides customizable error handling modes to control how errors are processed.
-        You probably don&apos;t want type-checking errors or internal logging-related errors to
-        crash your application, so our default behavior is to log and report those errors without
-        crashing. We automatically detect which error reporting service you use (Sentry, Bugsnag,
-        Rollbar, etc.). If you use a service that we don&apos;t support yet, you can configure a{' '}
+        LogStruct sorts the errors it handles into four groups, each with its own mode (the example
+        below shows all four with their defaults). Type-checking errors and errors inside LogStruct
+        are raised in development and test, so you see them early. In production they don&apos;t
+        crash your app: type-checking errors are logged and reported, and LogStruct&apos;s own
+        errors are logged. The modes are <code>Ignore</code>, <code>Log</code>, <code>Report</code>{' '}
+        (log and report), <code>LogProduction</code> and <code>ReportProduction</code> (log, or log
+        and report, in production; raise in development and test) and <code>Raise</code>.
+      </p>
+      <p className="text-neutral-600 dark:text-neutral-300 mb-4">
+        We automatically detect which error reporting service you use (Sentry, Bugsnag, Rollbar,
+        Honeybadger). If you use a service that we don&apos;t support yet, you can configure a{' '}
         <Link href="#custom-error-reporting">custom error handler</Link>. (Or you can send a PR!)
       </p>
 
@@ -259,10 +274,10 @@ end`}
 
       <HeadingWithAnchor id="sorbet-integration">Sorbet Integration</HeadingWithAnchor>
       <p className="text-neutral-600 dark:text-neutral-300 mb-4">
-        LogStruct integrates with Sorbet to handle type checking errors based on the environment. We
-        raise type errors or logging-related errors in test/development so you can catch them early,
-        but we only log or report them in production. You can configure a different error handling
-        mode to change this behavior.
+        LogStruct sets Sorbet&apos;s runtime error handlers, so a type error anywhere in your app or
+        its gems is raised in development and test, and logged and reported in production while your
+        app carries on (<code>error_handling_modes.type_checking_errors</code>). Without LogStruct,
+        Sorbet raises them everywhere.
       </p>
 
       <CodeBlock language="ruby">

docs/app/docs/integrations/page.tsx

@@ -77,9 +77,8 @@ export default function IntegrationsPage() {
       <div className="mt-10">
         <HeadingWithAnchor id="sorbet-integration">Sorbet Integration</HeadingWithAnchor>
         <p className="text-neutral-600 dark:text-neutral-300 mb-4">
-          LogStruct integrates with Sorbet to handle type checking errors appropriately based on the
-          environment. We raise any logging-related errors in test/development and log or report
-          them in production to avoid crashing your application.
+          LogStruct sets Sorbet&apos;s runtime error handlers: type errors are raised in development
+          and test, and logged and reported in production so they don&apos;t crash your app.
         </p>
 
         <HeadingWithAnchor

docs/app/docs/logging/page.tsx

@@ -37,9 +37,11 @@ export default function LoggingDocsPage() {
           logs go to STDOUT.
         </li>
         <li>
-          <b>LogStruct:</b> when enabled, replaces the logger with SemanticLogger and emits JSON to
-          STDOUT in test/production by default. In development, you can opt‑in to the same JSON to
-          avoid surprises.
+          <b>LogStruct:</b> when enabled, replaces the logger with SemanticLogger and emits JSON. In
+          production it logs to STDOUT only, with no log file. In development it logs to STDOUT and
+          Rails&apos; log file, and in test to the log file only. With{' '}
+          <code>RAILS_LOG_TO_STDOUT</code> set, both log to STDOUT only. In development, you can
+          opt‑in to the same JSON to avoid surprises.
         </li>
       </ul>
 
@@ -74,7 +76,8 @@ config.logger = ActiveSupport::TaggedLogging.new(logger)`}
       </HeadingWithAnchor>
       <ul className="list-disc list-inside space-y-2 text-neutral-600 dark:text-neutral-300">
         <li>
-          Ensure <code>RAILS_LOG_TO_STDOUT=1</code> (many platforms set this by default).
+          LogStruct logs to STDOUT in production whether or not <code>RAILS_LOG_TO_STDOUT</code> is
+          set, so your platform collects the lines from the process&apos;s output.
         </li>
         <li>
           Keep LogStruct enabled in production (default) to emit structured JSON for all

docs/app/docs/sorbet-types/page.tsx

@@ -1,4 +1,5 @@
 import Image from 'next/image';
+import Link from 'next/link';
 import { CodeBlock } from '@/components/code-block';
 import { EditPageLink } from '@/components/edit-page-link';
 import { EnumsList } from '@/components/enums-list';
@@ -65,10 +66,14 @@ bundle exec srb init`}
 
       <HeadingWithAnchor id="error-handling">Error Handling for Type Errors</HeadingWithAnchor>
       <p className="text-neutral-600 dark:text-neutral-300 mb-4">
-        LogStruct configures Sorbet error handlers to log and report type-checking errors. If you
-        already use Sorbet and you want to keep using your own error handlers, set{' '}
-        <code>enable_sorbet_error_handlers</code> to <code>false</code>. This will prevent LogStruct
-        from overriding your handlers.
+        LogStruct sets Sorbet&apos;s runtime error handlers: type errors are raised in development
+        and test, and logged and reported in production. The{' '}
+        <Link href="/docs/configuration#error-handling-configuration">
+          <code>type_checking_errors</code> mode
+        </Link>{' '}
+        changes this. If you already use Sorbet and you want to keep using your own error handlers,
+        set <code>enable_sorbet_error_handlers</code> to <code>false</code>. This will prevent
+        LogStruct from overriding your handlers.
       </p>
       <RubyCodeExample name="test_disable_sorbet_error_handler" />
 

lib/log_struct.rb

@@ -55,8 +55,9 @@ def self.server_mode=(value)
   LogStruct::RailsBootBannerSilencer.install!
 
   # Patch Puma immediately for server runs so we can convert its lifecycle
-  # messages into structured logs reliably.
-  if ARGV.include?("server")
+  # messages into structured logs reliably. Not when LogStruct is disabled, and not
+  # when Puma isn't installed (install_patches! skips it).
+  if LogStruct.enabled? && ARGV.include?("server")
     begin
       require "log_struct/integrations/puma"
       LogStruct::Integrations::Puma.install_patches!
@@ -66,7 +67,7 @@ def self.server_mode=(value)
       if defined?(::Rails) && ::Rails.respond_to?(:env) && ::Rails.env.test?
         raise e
       else
-        LogStruct.handle_exception(e, source: LogStruct::Source::Puma)
+        LogStruct.handle_exception(e, source: LogStruct::Source::Internal, context: {integration: :puma})
       end
     end
   end

lib/log_struct/config_struct/error_handling_modes.rb

@@ -16,7 +16,9 @@ class ErrorHandlingModes < T::Struct
       # - Raise - Always raise the error
 
       # Configurable error handling categories
-      prop :type_checking_errors, ErrorHandlingMode, default: ErrorHandlingMode::LogProduction
+      # Sorbet type errors in your app and its gems: logged and reported in production, so they don't pass unseen
+      prop :type_checking_errors, ErrorHandlingMode, default: ErrorHandlingMode::ReportProduction
+      # LogStruct's own errors
       prop :logstruct_errors, ErrorHandlingMode, default: ErrorHandlingMode::LogProduction
       prop :security_errors, ErrorHandlingMode, default: ErrorHandlingMode::Report
       prop :standard_errors, ErrorHandlingMode, default: ErrorHandlingMode::Raise

lib/log_struct/config_struct/limits.rb

@@ -0,0 +1,28 @@
+# typed: strict
+# frozen_string_literal: true
+
+module LogStruct
+  module ConfigStruct
+    # How much of each logged value LogStruct writes. The limits keep a huge value (a request body,
+    # a large Hash) from slowing down the thread that writes the logs.
+    class Limits < T::Struct
+      include Sorbet::SerializeSymbolKeys
+
+      # The most keys written from a Hash. The rest are counted in _more_keys.
+      # Default: 100
+      prop :max_hash_keys, Integer, default: 100
+
+      # The most items written from an Array, followed by "... and N more items"
+      # Default: 10
+      prop :max_array_items, Integer, default: 10
+
+      # The most lines written from an Array that looks like a backtrace
+      # Default: 100
+      prop :max_backtrace_lines, Integer, default: 100
+
+      # Strings longer than this many bytes are cut, and say how long they were
+      # Default: 32768 (32 KB)
+      prop :max_string_bytes, Integer, default: 32_768
+    end
+  end
+end

lib/log_struct/configuration.rb

@@ -5,6 +5,7 @@
 require_relative "config_struct/error_handling_modes"
 require_relative "config_struct/integrations"
 require_relative "config_struct/filters"
+require_relative "config_struct/limits"
 
 module LogStruct
   # Core configuration class that provides a type-safe API
@@ -35,6 +36,7 @@ class Configuration < T::Struct
 
     const :integrations, ConfigStruct::Integrations, factory: -> { ConfigStruct::Integrations.new }
     const :filters, ConfigStruct::Filters, factory: -> { ConfigStruct::Filters.new }
+    const :limits, ConfigStruct::Limits, factory: -> { ConfigStruct::Limits.new }
 
     # Custom log scrubbing handler for any additional string scrubbing
     # Default: nil