Experiment: (suite), Suite: must-have Backend: clickhouse, Scale: 100 worlds, Environment: local-docker Cloud results writer disabled (no --cloud-connection-string or CLICKHOUSE_CLOUD_CONNECTION_STRING). === Running REQ-INFRA-2: Evaluate ClickHouse.Client and ClickHouse.Driver .NET libraries for connect/create/insert/read/drop on Array(Float64). === [REQ-INFRA-2] Run 79cb27767f7d44e787fb5f010682af04 starting against Host=localhost;Port=8123;Database=experiment;Username=default;Password=clickhouse [REQ-INFRA-2] Rows: 100, Array length: 1000 [REQ-INFRA-2] Recommendation ------------------------------ Winner (by write throughput): ClickHouse.Client at 1,022 rows/sec - ClickHouse.Client: 1,022 rows/sec write - ClickHouse.Driver: 978 rows/sec write RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------ REQ-INFRA-2 | ClickHouse.Client | 100 | write_time_ms | 97.81 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 100 | read_time_ms | 24.46 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 100 | rows_per_sec_write | 1,022.34 | rows/sec | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 100 | rows_per_sec_read | 4,087.92 | rows/sec | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | write_time_ms | 102.25 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | read_time_ms | 27.68 | ms | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | rows_per_sec_write | 978.04 | rows/sec | rows=100, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 100 | rows_per_sec_read | 3,612.26 | rows/sec | rows=100, array_length=1000 === Running REQ-OC-W1: Bulk write of complete game Outcome Context at 10K scale === [REQ-OC-W1] Run 9f6e988fb7ab4371a164ed34f6e808f9 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes -------------------------------------------------------------------------------------------------------------------- REQ-OC-W1 | clickhouse | 100w | single_game_insert_ms | 52.33 | ms | game=game_1, rows=847 REQ-OC-W1 | clickhouse | 100w | readable_after_write_bool | 1.00 | bool | expected=847, actual=847 REQ-OC-W1 | clickhouse | 100w | full_season_insert_ms | 7,677.65 | ms | games=288, total_rows=243776 REQ-OC-W1 | clickhouse | 100w | rows_per_sec | 31,751.40 | rows/sec | games=288, total_rows=243776 === Running REQ-OC-R1: Full game Outcome Context hydration, single node === [REQ-OC-R1] Run b3ca7ed84352404b95d0363ba2719e3a starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------- REQ-OC-R1 | clickhouse | 100w | query_latency_ms_p50 | 25.36 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 100w | query_latency_ms_p95 | 27.38 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 100w | total_rows | 849.00 | rows | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 100w | total_values | 84,900.00 | values | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 100w | payload_bytes_approx | 679,200.00 | bytes | iterations=5, game=game_1, approximation = values * 8 (Float64) === Running REQ-OC-S1: Measure Outcome Context compression ratio === [REQ-OC-S1] Run 1fbd178f71414b4abc710b0faef035d5 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- REQ-OC-S1 | clickhouse | 100w | compressed_bytes | 31,937,438.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 100w | uncompressed_bytes | 203,900,384.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 100w | total_rows | 243,663.00 | rows | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 100w | compression_ratio | 6.38 | ratio | games=288, table=game_outcome_context, uncompressed_bytes / compressed_bytes REQ-OC-S1 | clickhouse | 100w | bytes_per_row_compressed | 131.07 | bytes/row | games=288, table=game_outcome_context === Running REQ-PBP-W1: Bulk write of play-by-play for a single game, all worlds === [REQ-PBP-W1] Run 0b3799c4e9554e4da77effb17e9b5512 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------ REQ-PBP-W1 | clickhouse | 100w | single_game_insert_ms | 105.75 | ms | game=game_1, worlds=100 REQ-PBP-W1 | clickhouse | 100w | rows_inserted | 14,504.00 | rows | game=game_1, worlds=100 REQ-PBP-W1 | clickhouse | 100w | rows_per_sec | 137,155.48 | rows/sec | game=game_1, worlds=100 === Running REQ-PBP-W2: Full season play-by-play write, all worlds === [REQ-PBP-W2] Run fc3f7f6557464075a38fadb9b9136948 starting (worlds=100) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------- REQ-PBP-W2 | clickhouse | 100w | total_rows | 4,198,561.00 | rows | games=288, worlds=100 REQ-PBP-W2 | clickhouse | 100w | total_insert_ms | 34,367.74 | ms | games=288, worlds=100 REQ-PBP-W2 | clickhouse | 100w | rows_per_sec | 122,165.74 | rows/sec | games=288, worlds=100 REQ-PBP-W2 | clickhouse | 100w | compressed_bytes | 89,191,297.00 | bytes | games=288, worlds=100, table=play_by_play REQ-PBP-W2 | clickhouse | 100w | uncompressed_bytes | 205,008,226.00 | bytes | games=288, worlds=100, table=play_by_play