Experiment: (suite), Suite: must-have Backend: clickhouse, Scale: 10 worlds, Environment: local-docker Cloud results writer disabled (no --cloud-connection-string or CLICKHOUSE_CLOUD_CONNECTION_STRING). === Running REQ-INFRA-2: Evaluate ClickHouse.Client and ClickHouse.Driver .NET libraries for connect/create/insert/read/drop on Array(Float64). === [REQ-INFRA-2] Run a1fa0cacb156438180905c0a8046221a starting against Host=localhost;Port=8123;Database=experiment;Username=default;Password=clickhouse [REQ-INFRA-2] Rows: 10, Array length: 1000 [REQ-INFRA-2] Recommendation ------------------------------ Winner (by write throughput): ClickHouse.Driver at 138 rows/sec - ClickHouse.Driver: 138 rows/sec write - ClickHouse.Client: 99 rows/sec write RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ----------------------------------------------------------------------------------------------------------------- REQ-INFRA-2 | ClickHouse.Client | 10 | write_time_ms | 100.68 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 10 | read_time_ms | 9.45 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 10 | rows_per_sec_write | 99.32 | rows/sec | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Client | 10 | rows_per_sec_read | 1,058.03 | rows/sec | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | write_time_ms | 72.61 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | read_time_ms | 7.08 | ms | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | rows_per_sec_write | 137.72 | rows/sec | rows=10, array_length=1000 REQ-INFRA-2 | ClickHouse.Driver | 10 | rows_per_sec_read | 1,413.35 | rows/sec | rows=10, array_length=1000 === Running REQ-OC-W1: Bulk write of complete game Outcome Context at 10K scale === [REQ-OC-W1] Run 0eca8b5bdb44460c81a5c6aee7f87e98 starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes -------------------------------------------------------------------------------------------------------------------- REQ-OC-W1 | clickhouse | 10w | single_game_insert_ms | 72.50 | ms | game=game_1, rows=799 REQ-OC-W1 | clickhouse | 10w | readable_after_write_bool | 1.00 | bool | expected=799, actual=799 REQ-OC-W1 | clickhouse | 10w | full_season_insert_ms | 18,708.25 | ms | games=288, total_rows=229274 REQ-OC-W1 | clickhouse | 10w | rows_per_sec | 12,255.23 | rows/sec | games=288, total_rows=229274 === Running REQ-OC-R1: Full game Outcome Context hydration, single node === [REQ-OC-R1] Run c7cd2853258e4ecf991d17ab6dfe7b8e starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------ REQ-OC-R1 | clickhouse | 10w | query_latency_ms_p50 | 17.29 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 10w | query_latency_ms_p95 | 28.14 | ms | iterations=5, game=game_1, percentiles approximate at N=5 REQ-OC-R1 | clickhouse | 10w | total_rows | 795.00 | rows | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 10w | total_values | 7,950.00 | values | iterations=5, game=game_1 REQ-OC-R1 | clickhouse | 10w | payload_bytes_approx | 63,600.00 | bytes | iterations=5, game=game_1, approximation = values * 8 (Float64) === Running REQ-OC-S1: Measure Outcome Context compression ratio === [REQ-OC-S1] Run 93f8ecc3b2804e5498816d9e0e4e6888 starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ REQ-OC-S1 | clickhouse | 10w | compressed_bytes | 4,761,448.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 10w | uncompressed_bytes | 26,471,551.00 | bytes | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 10w | total_rows | 229,394.00 | rows | games=288, table=game_outcome_context REQ-OC-S1 | clickhouse | 10w | compression_ratio | 5.56 | ratio | games=288, table=game_outcome_context, uncompressed_bytes / compressed_bytes REQ-OC-S1 | clickhouse | 10w | bytes_per_row_compressed | 20.76 | bytes/row | games=288, table=game_outcome_context === Running REQ-PBP-W1: Bulk write of play-by-play for a single game, all worlds === [REQ-PBP-W1] Run a999e6da402242bc88fc4691ea1830d3 starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ---------------------------------------------------------------------------------------------------------- REQ-PBP-W1 | clickhouse | 10w | single_game_insert_ms | 32.25 | ms | game=game_1, worlds=10 REQ-PBP-W1 | clickhouse | 10w | rows_inserted | 1,437.00 | rows | game=game_1, worlds=10 REQ-PBP-W1 | clickhouse | 10w | rows_per_sec | 44,558.83 | rows/sec | game=game_1, worlds=10 === Running REQ-PBP-W2: Full season play-by-play write, all worlds === [REQ-PBP-W2] Run 7c470e6e56184bd5a275debf4f54abfa starting (worlds=10) RequirementId | Backend | Scale | MetricName | Value | Unit | Notes ----------------------------------------------------------------------------------------------------------------------------- REQ-PBP-W2 | clickhouse | 10w | total_rows | 419,412.00 | rows | games=288, worlds=10 REQ-PBP-W2 | clickhouse | 10w | total_insert_ms | 5,123.94 | ms | games=288, worlds=10 REQ-PBP-W2 | clickhouse | 10w | rows_per_sec | 81,853.47 | rows/sec | games=288, worlds=10 REQ-PBP-W2 | clickhouse | 10w | compressed_bytes | 10,473,603.00 | bytes | games=288, worlds=10, table=play_by_play REQ-PBP-W2 | clickhouse | 10w | uncompressed_bytes | 21,038,404.00 | bytes | games=288, worlds=10, table=play_by_play