Experiment: (suite), Suite: must-have Backend: clickhouse, Scale: 100 worlds, Environment: local-docker-compressed Cloud results writer disabled (no --cloud-connection-string or CLICKHOUSE_CLOUD_CONNECTION_STRING). === Running REQ-INFRA-2: Evaluate ClickHouse.Client and ClickHouse.Driver .NET libraries for connect/create/insert/read/drop on Array(Float64). === [REQ-INFRA-2] Run acbf7950144f48aab1ce8007e11c6d20 starting against Host=localhost;Port=8123;Database=experiment;Username=default;Password=clickhouse;UseCompression=true [REQ-INFRA-2] Rows: 100, Array length: 1000 [REQ-INFRA-2] Recommendation ------------------------------ Winner (by write throughput): ClickHouse.Client at 1,167 rows/sec - ClickHouse.Client: 1,167 rows/sec write - ClickHouse.Driver: 1,121 rows/sec write RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------ REQ-INFRA-2 | ClickHouse.Client | 100 | write_time_ms | 85.72 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 100 | read_time_ms | 30.59 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 100 | rows_per_sec_write | 1,166.57 | rows/sec | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 100 | rows_per_sec_read | 3,269.31 | rows/sec | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | write_time_ms | 89.19 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | read_time_ms | 27.24 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | rows_per_sec_write | 1,121.26 | rows/sec | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | rows_per_sec_read | 3,670.99 | rows/sec | rows=100, array_length=1000 === Running REQ-OC-W1: Bulk write of complete game Outcome Context at 10K scale === [REQ-OC-W1] Run 976aa2be1c384a97aa00ff59e4554021 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes -------------------------------------------------------------------------------------------------------------------- REQ-OC-W1 | clickhouse | 100w | single_game_insert_ms | 37.13 | ms | game=game_1, rows=849 REQ-OC-W1 | clickhouse | 100w | readable_after_write_bool | 1.00 | bool | expected=849, actual=849 REQ-OC-W1 | clickhouse | 100w | full_season_insert_ms | 7,190.39 | ms | games=288, total_rows=243745 REQ-OC-W1 | clickhouse | 100w | rows_per_sec | 33,898.71 | rows/sec | games=288, total_rows=243745 === Running REQ-OC-R1: Full game Outcome Context hydration, single node === [REQ-OC-R1] Run 8e7269544b72455e99e91266505f5fe1 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------- REQ-OC-R1 | clickhouse | 100w | query_latency_ms_p50 | 28.34 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 100w | query_latency_ms_p95 | 30.00 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 100w | total_rows | 846.00 | rows | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 100w | total_values | 84,600.00 | values | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 100w | payload_bytes_approx | 676,800.00 | bytes | iterations=5, game=game_1, approximation = values * 8 (Float64) === Running REQ-OC-S1: Measure Outcome Context compression ratio === [REQ-OC-S1] Run 7b3a79a6c2b54b6c95bf37227cf2bf2d starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- REQ-OC-S1 | clickhouse | 100w | compressed_bytes | 31,927,545.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 100w | uncompressed_bytes | 203,907,502.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 100w | total_rows | 243,733.00 | rows | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 100w | compression_ratio | 6.39 | ratio | games=288, table=game_outcome_context, uncompressed_bytes / compressed_bytes REQ-OC-S1 | clickhouse | 100w | bytes_per_row_compressed | 130.99 | bytes/row | games=288, table=game_outcome_context === Running REQ-PBP-W1: Bulk write of play-by-play for a single game, all worlds === [REQ-PBP-W1] Run 3ccdb34b8ebb41ce8e82325f05adb667 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------ REQ-PBP-W1 | clickhouse | 100w | single_game_insert_ms | 111.76 | ms | game=game_1, worlds=100 REQ-PBP-W1 | clickhouse | 100w | rows_inserted | 14,574.00 | rows | game=game_1, worlds=100 REQ-PBP-W1 | clickhouse | 100w | rows_per_sec | 130,401.64 | rows/sec | game=game_1, worlds=100 === Running REQ-PBP-W2: Full season play-by-play write, all worlds === [REQ-PBP-W2] Run 8e6eff0753da4d0aa9a40615baacf423 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------- REQ-PBP-W2 | clickhouse | 100w | total_rows | 4,198,840.00 | rows | games=288, worlds=100 REQ-PBP-W2 | clickhouse | 100w | total_insert_ms | 35,757.79 | ms | games=288, worlds=100 REQ-PBP-W2 | clickhouse | 100w | rows_per_sec | 117,424.47 | rows/sec | games=288, worlds=100 REQ-PBP-W2 | clickhouse | 100w | compressed_bytes | 89,172,525.00 | bytes | games=288, worlds=100, table=play_by_play REQ-PBP-W2 | clickhouse | 100w | uncompressed_bytes | 204,852,141.00 | bytes | games=288, worlds=100, table=play_by_play