Experiment: (suite), Suite: must-have Backend: clickhouse, Scale: 1000 worlds, Environment: local-docker Cloud results writer disabled (no --cloud-connection-string or CLICKHOUSE_CLOUD_CONNECTION_STRING). === Running REQ-INFRA-2: Evaluate ClickHouse.Client and ClickHouse.Driver .NET libraries for connect/create/insert/read/drop on Array(Float64). === [REQ-INFRA-2] Run 6a69ffa6563b469690d27638377dff95 starting against Host=localhost;Port=8123;Database=experiment;Username=default;Password=clickhouse [REQ-INFRA-2] Rows: 1000, Array length: 1000 [REQ-INFRA-2] Recommendation ------------------------------ Winner (by write throughput): ClickHouse.Client at 3,640 rows/sec - ClickHouse.Client: 3,640 rows/sec write - ClickHouse.Driver: 3,580 rows/sec write RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------- REQ-INFRA-2 | ClickHouse.Client | 1000 | write_time_ms | 274.76 | ms | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 1000 | read_time_ms | 230.90 | ms | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 1000 | rows_per_sec_write | 3,639.54 | rows/sec | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 1000 | rows_per_sec_read | 4,330.83 | rows/sec | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 1000 | write_time_ms | 279.31 | ms | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 1000 | read_time_ms | 246.46 | ms | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 1000 | rows_per_sec_write | 3,580.27 | rows/sec | rows=1000, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 1000 | rows_per_sec_read | 4,057.49 | rows/sec | rows=1000, array_length=1000 === Running REQ-OC-W1: Bulk write of complete game Outcome Context at 10K scale === [REQ-OC-W1] Run 29702c446164473bb34dce3e486fab3d starting (worlds=1000) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes -------------------------------------------------------------------------------------------------------------------- REQ-OC-W1 | clickhouse | 1000w | single_game_insert_ms | 374.52 | ms | game=game_1, rows=850 REQ-OC-W1 | clickhouse | 1000w | readable_after_write_bool | 1.00 | bool | expected=850, actual=850 REQ-OC-W1 | clickhouse | 1000w | full_season_insert_ms | 48,611.75 | ms | games=288, total_rows=244800 REQ-OC-W1 | clickhouse | 1000w | rows_per_sec | 5,035.82 | rows/sec | games=288, total_rows=244800 === Running REQ-OC-R1: Full game Outcome Context hydration, single node === [REQ-OC-R1] Run 7e946415996c4481a4dac290dfb466d4 starting (worlds=1000) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes --------------------------------------------------------------------------------------------------------------------------------------------------- REQ-OC-R1 | clickhouse | 1000w | query_latency_ms_p50 | 200.06 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 1000w | query_latency_ms_p95 | 322.41 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 1000w | total_rows | 850.00 | rows | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 1000w | total_values | 850,000.00 | values | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 1000w | payload_bytes_approx | 6,800,000.00 | bytes | iterations=5, game=game_1, approximation = values * 8 (Float64) === Running REQ-OC-S1: Measure Outcome Context compression ratio === [REQ-OC-S1] Run ad6a1b67117d4ad5b47b0a6b0ad914af starting (worlds=1000) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- REQ-OC-S1 | clickhouse | 1000w | compressed_bytes | 282,574,850.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 1000w | uncompressed_bytes | 1,968,075,656.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 1000w | total_rows | 244,800.00 | rows | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 1000w | compression_ratio | 6.96 | ratio | games=288, table=game_outcome_context, uncompressed_bytes / compressed_bytes REQ-OC-S1 | clickhouse | 1000w | bytes_per_row_compressed | 1,154.31 | bytes/row | games=288, table=game_outcome_context === Running REQ-PBP-W1: Bulk write of play-by-play for a single game, all worlds === [REQ-PBP-W1] Run a5654068886040989aaf674d69e7f9e6 starting (worlds=1000) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------- REQ-PBP-W1 | clickhouse | 1000w | single_game_insert_ms | 3,048.65 | ms | game=game_1, worlds=1000 REQ-PBP-W1 | clickhouse | 1000w | rows_inserted | 145,709.00 | rows | game=game_1, worlds=1000 REQ-PBP-W1 | clickhouse | 1000w | rows_per_sec | 47,794.64 | rows/sec | game=game_1, worlds=1000 === Running REQ-PBP-W2: Full season play-by-play write, all worlds === [REQ-PBP-W2] Run 711fa7719cfa4ccd91c47c8093141be3 starting (worlds=1000) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ---------------------------------------------------------------------------------------------------------------------------------- REQ-PBP-W2 | clickhouse | 1000w | total_rows | 41,980,124.00 | rows | games=288, worlds=1000 REQ-PBP-W2 | clickhouse | 1000w | total_insert_ms | 476,203.18 | ms | games=288, worlds=1000 REQ-PBP-W2 | clickhouse | 1000w | rows_per_sec | 88,155.91 | rows/sec | games=288, worlds=1000 REQ-PBP-W2 | clickhouse | 1000w | compressed_bytes | 881,937,764.00 | bytes | games=288, worlds=1000, table=play_by_play REQ-PBP-W2 | clickhouse | 1000w | uncompressed_bytes | 2,041,298,711.00 | bytes | games=288, worlds=1000, table=play_by_play