Experiment: (suite), Suite: must-have Backend: clickhouse, Scale: 10 worlds, Environment: local-docker-compressed Cloud results writer disabled (no --cloud-connection-string or CLICKHOUSE_CLOUD_CONNECTION_STRING). === Running REQ-INFRA-2: Evaluate ClickHouse.Client and ClickHouse.Driver .NET libraries for connect/create/insert/read/drop on Array(Float64). === [REQ-INFRA-2] Run fa2a38dc649e4f2bbd63a25885860018 starting against Host=localhost;Port=8123;Database=experiment;Username=default;Password=clickhouse;UseCompression=true [REQ-INFRA-2] Rows: 10, Array length: 1000 [REQ-INFRA-2] Recommendation ------------------------------ Winner (by write throughput): ClickHouse.Client at 128 rows/sec - ClickHouse.Client: 128 rows/sec write - ClickHouse.Driver: 127 rows/sec write RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ----------------------------------------------------------------------------------------------------------------- REQ-INFRA-2 | ClickHouse.Client | 10 | write_time_ms | 78.21 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 10 | read_time_ms | 9.84 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 10 | rows_per_sec_write | 127.87 | rows/sec | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 10 | rows_per_sec_read | 1,015.99 | rows/sec | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | write_time_ms | 78.60 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | read_time_ms | 6.95 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | rows_per_sec_write | 127.23 | rows/sec | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | rows_per_sec_read | 1,438.93 | rows/sec | rows=10, array_length=1000 === Running REQ-OC-W1: Bulk write of complete game Outcome Context at 10K scale === [REQ-OC-W1] Run d300d4fc35ae4797a4f950ac8f3d97fe starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes -------------------------------------------------------------------------------------------------------------------- REQ-OC-W1 | clickhouse | 10w | single_game_insert_ms | 68.82 | ms | game=game_1, rows=774 REQ-OC-W1 | clickhouse | 10w | readable_after_write_bool | 1.00 | bool | expected=774, actual=774 REQ-OC-W1 | clickhouse | 10w | full_season_insert_ms | 18,563.21 | ms | games=288, total_rows=229496 REQ-OC-W1 | clickhouse | 10w | rows_per_sec | 12,362.95 | rows/sec | games=288, total_rows=229496 === Running REQ-OC-R1: Full game Outcome Context hydration, single node === [REQ-OC-R1] Run 51133a700f1044318df6713c4355413d starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------ REQ-OC-R1 | clickhouse | 10w | query_latency_ms_p50 | 17.99 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 10w | query_latency_ms_p95 | 29.62 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 10w | total_rows | 791.00 | rows | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 10w | total_values | 7,910.00 | values | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 10w | payload_bytes_approx | 63,280.00 | bytes | iterations=5, game=game_1, approximation = values * 8 (Float64) === Running REQ-OC-S1: Measure Outcome Context compression ratio === [REQ-OC-S1] Run 592566c46ad3426ca3017fc6b9d64320 starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ REQ-OC-S1 | clickhouse | 10w | compressed_bytes | 4,780,028.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 10w | uncompressed_bytes | 26,472,031.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 10w | total_rows | 228,520.00 | rows | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 10w | compression_ratio | 5.54 | ratio | games=288, table=game_outcome_context, uncompressed_bytes / compressed_bytes REQ-OC-S1 | clickhouse | 10w | bytes_per_row_compressed | 20.92 | bytes/row | games=288, table=game_outcome_context === Running REQ-PBP-W1: Bulk write of play-by-play for a single game, all worlds === [REQ-PBP-W1] Run d8a28fc6c17a4936818f5ea8bd9ae821 starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ---------------------------------------------------------------------------------------------------------- REQ-PBP-W1 | clickhouse | 10w | single_game_insert_ms | 22.31 | ms | game=game_1, worlds=10 REQ-PBP-W1 | clickhouse | 10w | rows_inserted | 1,470.00 | rows | game=game_1, worlds=10 REQ-PBP-W1 | clickhouse | 10w | rows_per_sec | 65,894.76 | rows/sec | game=game_1, worlds=10 === Running REQ-PBP-W2: Full season play-by-play write, all worlds === [REQ-PBP-W2] Run dd3d6bcfcb24439a900516b0488969e2 starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ----------------------------------------------------------------------------------------------------------------------------- REQ-PBP-W2 | clickhouse | 10w | total_rows | 420,613.00 | rows | games=288, worlds=10 REQ-PBP-W2 | clickhouse | 10w | total_insert_ms | 5,018.83 | ms | games=288, worlds=10 REQ-PBP-W2 | clickhouse | 10w | rows_per_sec | 83,806.99 | rows/sec | games=288, worlds=10 REQ-PBP-W2 | clickhouse | 10w | compressed_bytes | 10,487,958.00 | bytes | games=288, worlds=10, table=play_by_play REQ-PBP-W2 | clickhouse | 10w | uncompressed_bytes | 21,079,758.00 | bytes | games=288, worlds=10, table=play_by_play