distributed systems
5 minEvent-Time Windows: Watermarks, Late Data, and Correctable Results
Design a minute-level dashboard that tolerates delayed events without treating a watermark as proof that every event arrived.
Loading page...
distributed systems
5 minDesign a minute-level dashboard that tolerates delayed events without treating a watermark as proof that every event arrived.
typescript
6 minParse unknown data and use discriminated unions to preserve operational meaning through callers.
Q&A
Q&AThey require different caller behavior and must not collapse into null.
Validate unknown runtime data, then preserve outcome meaning in a discriminated union. Exhaustive handling exposes new states during compilation; structural validation does not establish permissions or retry safety.
Q&A
Q&AAssociativity and an identity element make regrouping partial results meaningful.
A parallel reduction may group values differently, so its combine operation should be associative for equivalent results and have a suitable identity value. Floating-point addition can vary slightly with grouping. Avoid reducers with hidden side effects because execution order may not be stable.
Q&A
Q&ACount repeated statements per request, then batch-load without changing result cardinality.
Trace or log normalized SQL for one request and look for the same child lookup repeated once per parent row. Replace the loop with an eager/batch load or a query at the correct grain. Verify query count and returned rows; a broad join can fix round trips while accidentally multiplying results.
Q&A
Q&AWHERE filters input rows before grouping; HAVING filters groups after aggregation.
Use WHERE to remove rows before GROUP BY, such as excluding inactive employees. Use HAVING to filter aggregate results, such as departments with COUNT(*) greater than five. Moving a condition between them can change both meaning and query work.
Q&A
Q&AIn Python 3, map returns an iterator that computes values as it is consumed.
Python 3 map is lazy: it returns an iterator and evaluates each mapped value as the iterator advances. Convert it to a list only when you need materialized results or repeated traversal. Lazy evaluation saves memory but exceptions may occur later when a consumer iterates.
Q&A
Q&AAnalytic functions calculate across a partition while keeping each input row in the result.
GROUP BY collapses rows into one row per group. An analytic function uses OVER with partition, ordering, and frame rules to calculate a value while preserving row detail. Specify a deterministic ordering and explicit frame for running totals when peer-row behavior would matter.