CustomerApi.Jobs.GenerateAccountMetrics (19820255) • Oban

19820255 CustomerApi.Jobs.GenerateAccountMetrics

Queue clickhouse_account_metrics

Attempt 4 of 10

Priority 0

Tags ...

Node customer_api@10.10.1.109

Queue Time 01:41.599

Run Time 03:38.865

Inserted 3h ago

Scheduled 1h ago

Completed 1h ago (03:39)

Cancelled —

Discarded —

Args

%{
  "account_id" => "46248",
  "date" => "2026-03-06",
  "query" => "users_reached",
  "window_days" => 30
}

Recorded Output

%{
  users_reached: 60,
  builder_event_users_reached: 0,
  nps_survey_users_reached: 0
}

Errors

Attempt 3—1h ago

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "E) (version 25.10.1.7375 (official build))\n"

    # 2
    44

    # 3
    67

    # 4
    ["ergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_OPEN_FIL", "or. It can mean that some retryable error happened, or data is corrupted. Please retry the query: While executing ", "311_12661295_36_22488012 (state Active) failed with er", "pressed data header: <uninitialized>): (while reading column attributes): (attempt to read data part all_1239", "5, typename: DB::AsynchronousBoundedReadBuffer, co", "sing ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/niy/jcoefmaqpniynhwkpjubyyrqfhqvn (position: 40118863", "1434, kind: Regular, unbound: 0: While reading or decompre", "-3a66128a8d19:1434, current write offset: 4009754624, caller id: 65dca7f1-9dff-47c1-aeee-3a66128a8d19", "CONTINUATION, downloaded size: 0, reserved size: 1048576, downloader id: 65dca7f1-9dff-47c1-aee", " info: File segment: [4009754624, 4043309055], key: 31211262ceb296d126dc7cad9682b2e5, state: PARTIALLY_DOWNLOADED_NO", "UT_IN_CACHE, last caller: 65dca7f1-9dff-47c1-aeee-3a66128a8d19:1434, file segmen", "_end: 4011886325, read_until_position: 4322070158, internal buffer end: 4010803200, read_type: REMOTE_FS_READ_AND_", "e/niy/jcoefmaqpniynhwkpjubyyrqfhqvn, hash key: 31211262ceb296d126dc7cad9682b2e5, file_offset_of_buffe", "uffer path: ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetr", "128a8d19:1434, kind: Regular, unbound: 0: Cache info: ", "09754624, caller id: 65dca7f1-9dff-47c1-aeee-3a6", "rved size: 1048576, downloader id: 65dca7f1-9dff-47c1-aeee-3a66128a8d19:1434, current write offset: 4", " state: DOWNLOADING, downloaded size: 0, res", "e system, current cache state: File segment: [4009754624, 4043309055], key: 31211262ceb296d126dc7cad9682b2e5", "e5/4009754624: , errno: 30, strerror: Read-only fi", "ile system: Cannot open file /mnt/clickhouse-cache/sharedS3DiskCache/312/31211262ceb296d126dc7cad9682b", "b2e5/4009754624: , errno: 30, strerror: Read-only ", "de: 76. DB::Exception: Cannot open file /mnt/clickhouse-cache/sharedS3DiskCache/312/31211262ceb296d126dc7cad968"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 2—1h ago

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "CEEDED) (version 25.10.1.7375 (official build))\n"

    # 2
    63

    # 3
    67

    # 4
    ["s reached: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (MEMORY_LIMIT_EX", "iB. OvercommitTracker decision: Waiting timeout for memory to be freed ", " MiB), current RSS: 303.84 GiB, maximum: 320.40 ", "de: 241. DB::Exception: (total) memory limit exceeded: would use 343.75 GiB (attempt to allocate chunk of 254.0"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 1—1h ago

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ial build))\n"

    # 2
    59

    # 3
    67

    # 4
    ["ct(pool: PrefetchedReadPool, algorithm: Thread). (MEMORY_LIMIT_EXCEEDED) (version 25.10.1.7375 (offic", "3WithKeeperDisk of type s3, from mark 26696 with max_rows_to_read = 4406, offset = 0): While executing MergeTreeSel", "0ce73-d792-4409-8626-e5e307da7fa9) located on disk ", "prod_events.event_history_v2 (5f", "ading from part data/5f30ce73-d792-4409-8626-e5e307da7fa9/all_14956512_14965587_21_22488012/ in table", "ision: Query was selected to stop by OvercommitTracker: (while reading column attributes): (while r", "iB), current RSS: 323.60 GiB, maximum: 320.40 GiB. OvercommitTracker de", "de: 241. DB::Exception: (total) memory limit exceeded: would use 325.60 GiB (attempt to allocate chunk of 2.00 "]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Keyboard Shortcuts

States

Nodes