[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"Kw8oafdRYM":3},"# LeanReducers\n\nFused sequential and parallel reducers for Lean 4.\n\n`LeanReducers` provides two explicit pipeline modes:\n\n- `ReducerSeq` is the lightweight sequential API. Its core terminal is\n  `reduce unit step`; it does not require algebraic proofs or parallel config.\n- `ReducerPar` is the parallel API. It uses Lean's `Task`, and its lawful\n  terminals accept a `MonoidSpec` so chunk results can be combined safely.\n\nBoth modes fuse transforms into the terminal reduction instead of allocating\nintermediate pipeline collections.\n\n## Quick Start\n\n```lean\nimport LeanReducers\n\nopen LeanReducers\n\n#eval\n  #[(1 : Nat), 2, 3, 4]\n    |> ReducerSeq.ofArray\n    |>.map (fun x => x + 1)\n    |>.filter (fun x => x % 2 == 0)\n    |>.sum\n-- 6\n```\n\nOpt into parallel reduction when the workload warrants it:\n\n```lean\n#eval\n  #[(1 : Nat), 2, 3, 4]\n    |> ReducerPar.ofArray\n    |>.reduceWithLaws (MonoidSpec.additive Nat) (fun x acc => x + acc)\n-- 10\n```\n\nBuild and run the test executable:\n\n```sh\nlake build\nlake exe lean_reducers_tests\n```\n\nThe test executable uses Plausible to generate arrays, file contents, and\nparallel configurations. Each test compares a reducer pipeline against a\nsequential Lean model of the same producer, transforms, and terminal.\n\n### Native Backend And Interpreted Runs\n\n`ReducerPar` line-oriented file producers use a small native `pread` bridge in\ncompiled code. The normal path is to run through a Lake executable, as above.\n\nIf you run a file through Lean's interpreter, load the generated dynamic\nlibraries explicitly:\n\n```sh\nlake build LeanReducers:shared\nlake env lean \\\n  --load-dynlib .lake/build/lib/libleanreducersio.dylib \\\n  --load-dynlib .lake/build/lib/liblean__reducers_LeanReducers.dylib \\\n  --run Test.lean\n```\n\nOn Linux, use the generated `.so` files instead of `.dylib`.\n\n## Pipeline Model\n\nReducers are built as a three-stage pipeline:\n\n```lean\nproducer |> intermediate |> intermediate |> terminal\n```\n\n- A producer chooses sequential or parallel execution and the effect.\n  `ReducerSeq.ofArray` creates a pure `ReducerSeq α`;\n  `ReducerPar.readLines` creates an effectful `ReducerParIO String`.\n- Intermediate operations such as `map`, `filter`, and `flatMap` transform the\n  reducer without running it. They are fused into the terminal reduction plan\n  instead of allocating intermediate collections.\n- A terminal runs the reduction. `ReducerSeq.reduce` only needs a unit and local\n  step. `ReducerPar.reduceWithLaws` additionally accepts a lawful chunk combiner.\n  Convenience terminals such as `toArray`, `length`, `sum`, `min?`, `max?`,\n  `avgFloat`, and `groupBy` are available in both modes.\n\nFor example:\n\n```lean\n#[(1 : Nat), 2, 3, 4] -- source data\n  |> ReducerSeq.ofArray  -- producer\n  |>.map (fun x => x + 1) -- intermediate\n  |>.filter (fun x => x % 2 == 0) -- intermediate\n  |>.sum -- terminal\n```\n\n## Core Types\n\n```lean\nstructure ReducerSeqM (m : Type → Type) (α : Type)\n\nabbrev ReducerSeq (α : Type) := ReducerSeqM Id α\nabbrev ReducerSeqIO (α : Type) := ReducerSeqM IO α\n\nstructure MonoidSpec (α : Type) where\n  unit : α\n  combine : α → α → α\n  assoc : ∀ a b c, combine (combine a b) c = combine a (combine b c)\n  left_unit : ∀ a, combine unit a = a\n  right_unit : ∀ a, combine a unit = a\n\nstructure ReducerParM (m : Type → Type) (α : Type)\n\nabbrev ReducerPar (α : Type) := ReducerParM Id α\nabbrev ReducerParIO (α : Type) := ReducerParM IO α\n```\n\nThe `M` types carry the producer effect. Pure aliases use `Id`; file producers\nreturn the `IO` aliases.\n\nInternally, `ReducerSeq` producers run the fused step directly. `ReducerPar`\narray producers use a pure `Task` tree. Parallel line producers use IO tasks that\nread byte ranges and repair newline boundaries before folding lines.\n\n## API Overview\n\nBoth modes expose `ofArray`, `ofArrayM`, `readFile`, `readLines`,\n`readLinesFromFiles`, `readLinesFromFilesWithPath`, and `readChars` producers.\nBoth also expose fused `map`, `filter`, and `flatMap` transforms.\n\n`flatMap` receives a pure sequential reducer for each input:\n\n```lean\n.flatMap : ReducerSeqM m α → (α → ReducerSeq β) → ReducerSeqM m β\n.flatMap : ReducerParM m α → (α → ReducerSeq β) → ReducerParM m β\n```\n\nThe inner `ReducerSeq` is an allocation-free expansion plan. It is consumed\ninside the outer reduction; it does not introduce nested effects or parallel\ntask trees. Build small expansions with `ReducerSeq.empty`, `one`, and `append`,\nor wrap an existing collection with `ReducerSeq.ofArray` or `ofList`.\n\n### Sequential Terminals\n\n```lean\n.reduce    : ρ → (α → ρ → ρ) → ReducerSeqM m α → m ρ\n.reduceMap : ρ → (ρ → ρ → ρ) → (α → ρ) → ReducerSeqM m α → m ρ\n.groupBy   : ν → (α → κ) → (α → ν → ν) → ReducerSeqM m α → m (Array (κ × ν))\n```\n\n`ReducerSeq` convenience terminals include `toArray`, `length`, `sum`,\n`sumFloat`, `min?`, `max?`, `avgFloat`, and `avg`. Sequential `.sum` only needs\n`Add α` and `OfNat α 0`; it can be used directly with `Float`.\n\n### Parallel Terminals\n\n```lean\n.reduceWithLaws    : MonoidSpec ρ → (α → ρ → ρ) → ReducerParM m α → m ρ\n.reduceMapWithLaws : MonoidSpec ρ → (α → ρ) → ReducerParM m α → m ρ\n.reduceWithoutLaws : ρ → (ρ → ρ → ρ) → (α → ρ → ρ) → ReducerParM m α → m ρ\n.groupBy           : MonoidSpec ν → (α → κ) → (α → ν → ν) → ReducerParM m α →\n  m (Array (κ × ν))\n```\n\n`ReducerPar` exposes the same convenience terminal names. Parallel `.sum`\nrequires `LawfulAddMonoid α`; practical floating-point reductions use\n`.sumFloat`.\n\nFor pure reducers, `m` is `Id`, so terminals return values directly. For file\nproducers, terminals return `IO`. `min?`, `max?`, and floating averages return\n`none` for empty reducers. Pure reducers in both modes also expose proof-bearing\n`.min` and `.max`.\n\nGrouped values start at the supplied unit, then the per-key step is applied for\neach element. Parallel grouping receives its unit and chunk combiner through\n`MonoidSpec`. Group output order is unspecified.\n\n## Examples\n\n### `flatMap`\n\n```lean\n#eval\n  #[1, 2, 3]\n    |> ReducerSeq.ofArray\n    |>.flatMap (fun x =>\n      ReducerSeq.one x\n        |>.append (ReducerSeq.one (x * 10)))\n    |>.sum\n-- 66\n```\n\n### Convenience Terminals\n\n```lean\n#eval\n  #[3, 1, 4, 1, 5]\n    |> ReducerSeq.ofArray\n    |>.min?\n-- some 1\n\n#eval\n  match (#[3.0, 1.0, 4.0] |> ReducerSeq.ofArray |>.avgFloat) with\n  | some avg => avg > 2.0 && avg \u003C 3.0\n  | none => false\n-- true\n```\n\n### `groupBy`\n\n```lean\n#eval\n  #[(\"a\", 1), (\"b\", 2), (\"a\", 3)]\n    |> ReducerPar.ofArray\n    |>.groupBy\n      (MonoidSpec.additive Nat)\n      (fun row => row.1)\n      (fun row acc => row.2 + acc)\n    |>.qsort (fun left right => compare left.1 right.1 == Ordering.lt)\n-- #[(\"a\", 4), (\"b\", 2)]\n```\n\n### File Producers\n\n`ReducerSeq.readLines` reads and reduces lines sequentially without exposing\nparallel tuning knobs. Multi-file sequential producers process one file at a\ntime.\n\n`ReducerPar.readLines` is the large-file path: it splits the file into byte ranges,\nreads those ranges in parallel, then adjusts each range to whole-line boundaries\nso every line is folded exactly once.\n\n`ReducerPar.readLinesFromFiles` and `ReducerPar.readLinesFromFilesWithPath` use the\nsame source byte-range scheduler as `ReducerPar.readLines`, so skewed file sizes can\nbe balanced by splitting large files into multiple ranges.\n\nBecause these producers use the native backend in compiled code, interpreted\n`lean --run` sessions need the dynamic libraries described above.\n\n```lean\ndef countLinesByLength (path : System.FilePath) : IO (Array (Nat × Nat)) := do\n  ReducerPar.readLines path\n    |>.filter (fun line => line != \"\")\n    |>.groupBy\n      (MonoidSpec.additive Nat)\n      String.length\n      (fun _ acc => 1 + acc)\n```\n\nUse `readLinesFromFilesWithPath` when the terminal needs to know which file\nproduced each line:\n\n```lean\ndef countNonemptyLinesByFile\n    (paths : Array System.FilePath) : IO (Array (System.FilePath × Nat)) := do\n  ReducerPar.readLinesFromFilesWithPath paths\n    |>.filter (fun row => row.2 != \"\")\n    |>.groupBy\n      (MonoidSpec.additive Nat)\n      (fun row => row.1)\n      (fun _ acc => 1 + acc)\n```\n\n## Lawfulness\n\n`ReducerPar` can regroup chunk results. That is why its lawful reductions use\n`MonoidSpec`: the result combiner must be associative and must have a lawful\nunit. `ReducerSeq` does not regroup work and therefore does not require these\nproofs.\n\nUse `reduceWithoutLaws` when you have a useful `unit` and `combine` but do not want\nto provide monoid proofs. The reduction still runs in parallel, so the supplied\ncombiner should be stable under regrouping if you need deterministic equality\nwith a sequential fold.\n\n`Float` is intentionally not a lawful `.sum` target. IEEE floating-point\naddition is not mathematically associative, so the library exposes:\n\n```lean\nsumFloat : ReducerPar Float → Float\navgFloat : ReducerPar Float → Option Float\n```\n\nand the monadic equivalents for practical floating-point reductions over the\nwithout-laws reduction path.\n\nLikewise, `min?` and `max?` use the `Min` and `Max` classes. For\nparallel-stable results, those operations should behave consistently under\nregrouping.\n\n## Parallel Configuration\n\n```lean\nstructure Config where\n  grain : Nat := 2048\n  maxDepth : Nat := 4\n  priority : Task.Priority := Task.Priority.default\n  diagnostics : DiagnosticsConfig := {}\n```\n\nUse `ReducerPar.reduceWithLawsWithConfig`, `reduceMapWithLawsWithConfig`,\n`reduceWithoutLawsWithConfig`, or `groupByWithConfig` to tune parallel splitting.\nFor parallel line readers, `grain` is interpreted as a target byte chunk size\nbefore newline-boundary repair.\nDiagnostics are disabled by default; line readers can emit a colorized,\ntop-anchored panel with progress, OS-sampled process IO throughput, OS-sampled\nper-CPU bars, process RSS, and system memory usage with available memory through\na parameterized output sink.\nThe default sink is `DiagnosticsOutput.console`. `cpuBars := 0` means\nauto-detect the CPU count.\n\n## Design Notes\n\n- `ReducerSeq` line producers read one file at a time. `ReducerPar` array\n  producers use index-based chunking, and parallel line readers use a small native\n  `pread` bridge for true parallel range reads.\n- Parallel line producers schedule over source byte ranges. Single-file reads\n  start from one range, while multi-file reads start from one range per file and\n  may split a large file into multiple ranges.\n- File line chunks are expanded or trimmed at newline boundaries. A line that\n  starts exactly at a split belongs to the right chunk, so boundary lines are not\n  duplicated.\n- Sequential line producers, `readFile`, and `readChars` read whole files before\n  reducing.\n- `map`, `filter`, and `flatMap` are fused by rewriting the terminal reduction plan;\n  they do not build intermediate pipeline collections.\n- `groupBy` uses a `Std.HashMap` accumulator internally and returns an\n  `Array (key × value)` at the API boundary.\n",1784670018755]