19817591 CustomerApi.Jobs.GenerateAccountMetrics
Queue clickhouse_account_metrics
Attempt 3 of 10
Priority 0
Tags ...
Node customer_api@10.10.0.159
Queue Time 07:16.870
Run Time 00:01.736
Inserted 6h ago
Scheduled 5h ago
Completed 4h ago (00:02)
Cancelled
Discarded

Args

%{
  "account_id" => "84352",
  "date" => "2026-03-06",
  "query" => "users_reached",
  "window_days" => 30
}

Meta

%{
  "deps" => ["generate_event_counts"],
  "name" => "generate_users_reached",
  "on_hold" => false,
  "orig_scheduled_at" => 1772759298370326,
  "recorded" => true,
  "return" => "g1AAAABXeJwrYWBgYC7nLS1OLSqOL0pNTM5ITUliYOAVK5dOKs3MSUktik8tS80riUdRkchWLpFXUBxfXFpUlloZj66bMQsABxQecQ",
  "structured" => true,
  "workflow" => true,
  "workflow_id" => "019cc07f-789b-7953-844d-5bbfd72eff4c"
}

Recorded Output

%{
  users_reached: 3350,
  builder_event_users_reached: 6,
  nps_survey_users_reached: 362
}

Errors

Attempt 2—5h ago

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "cial build))\n"

    # 2
    43

    # 3
    67

    # 4
    ["g MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_OPEN_FILE) (version 25.10.1.7375 (offi", "n that some retryable error happened, or data is corrupted. Please retry the query: While executi", "488012 (state Active) failed with error. It can me", "lized>): (while reading column attributes): (attempt to read data part all_14570857_14583516_13_2", "compressed data header: <uniniti", " 18669544555, typename: DB::AsynchronousBoundedReadBuffer,", "h-s3-f65/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/eck/afaloifcxbkvgijzywwghtfxwsvmd (position", "f6cb8ca-9ca8-4027-a1e4-36333859cd4f:2372, kind: Regular, unbound: 0: While reading or decompressing ", "offset: 18668847104, caller id: ", "loaded size: 0, reserved size: 697451, downloader id: df6cb8ca-9ca8-4027-a1e4-36333859cd4f:2372, current write", "aba0b86, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, dow", " 18673041407], key: cf8d2a13237034b8cfa4685a", "372, file segment info: File segment: [18668847104", "EMOTE_FS_READ_AND_PUT_IN_CACHE, last caller: df6cb8ca-9ca8-4027-a1e4-36333859cd4f:", "10, internal buffer end: 18669895680, read_type: ", "13237034b8cfa4685a9aba0b86, file_offset_of_buffer_end: 18669544555, read_until_position: 18669606", " path: ch-s3-f65/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/eck/afaloifcxbkvgijzywwghtfxwsvmd, hash key: cf8d2", "et: 18668847104, caller id: df6cb8ca-9ca8-4027-a1e4-36333859cd4f:2372, kind: Regular, unbound: 0: Cache info: Buffe", "ze: 0, reserved size: 697451, downloader id: df6cb8ca-9ca8-4027-a1e4-36333859cd4f:2372, current write off", "b8cfa4685a9aba0b86, state: DOWNLOADING, downloaded s", "-only file system, current cache state: File segment: [18668847104, 18673041407], key: cf8d2a1323703", "ache/sharedS3DiskCache/cf8/cf8d2a13237034b8cfa4685a9aba0b86/18668847104: , errno: 30, strerror: Rea", "0b86/18668847104: , errno: 30, strerror: Read-only file system: Cannot open file /mnt/clickhouse-", "de: 76. DB::Exception: Cannot open file /mnt/clickhouse-cache/sharedS3DiskCache/cf8/cf8d2a13237034b8cfa4685a9ab"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 1—5h ago

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "5.10.1.7375 (official build))\n"

    # 2
    45

    # 3
    67

    # 4
    ["uery: While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_OPEN_FILE) (version 2", "e Active) failed with error. It can mean that some retryable error happened, or data is corrupted. Please retry the ", "ized>): (while reading column attributes): (attempt to read data part all_14421489_14432149_16_22488012 (sta", "bqusbhqlca (position: 18440058250, typename: DB::AsynchronousBoundedReadBuffer, compressed data header: <uninitia", "8441591d3836/mergetree/kct/tevrlyirvfxmeavxow", "621:2307, kind: Regular, unbound: 0: While reading or decompressing ch-s3-cb4/aa8129e2-f47e-4b75-bbcb", "5c87eb0a-1c1e-4669-8154-257ff1bf", "307, current write offset: 18438160384, caller id:", "6, downloader id: 5c87eb0a-1c1e-4669-8154-257ff1bfe621:", ": 8683fc40b0071d63d8da5b9b9d5edcb1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 10485", "b0a-1c1e-4669-8154-257ff1bfe621:2307, file segment info: File segment: [18438160384, 18442354687], ke", "ition: 18440112429, internal buffer end: 18439208960, read_type: REMOTE_FS_READ_AND_PUT_IN_CACHE, last caller: 5c87", "wqbqusbhqlca, hash key: 8683fc40b0071d63d8da5b9b9d5edcb1, file_offset_of_buffer_end: 18440058250, read_until_po", "he info: Buffer path: ch-s3-cb4/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/kct/tevrlyirvfxmeavx", "fset: 18438160384, caller id: 5c87eb0a-1c1e-4669-8154-257ff1bfe621:2307, kind: Regular, unbound: 0: Ca", "aded size: 0, reserved size: 1048576, downloader id: 5c87eb0a-1c1e-4669-8154-257ff1bfe621:2307, current write o", "71d63d8da5b9b9d5edcb1, state: DOWNLOADING, downl", " File segment: [18438160384, 18442354687], key: 8683fc40b0", "file system, current cache state", "/sharedS3DiskCache/868/8683fc40b0071d63d8da5b9b9d5edcb1/18438160384: , errno: 30, strerror: Read-only", "dcb1/18438160384: , errno: 30, strerror: Read-only file system: Cannot open file /mnt/clickhouse-cach", "de: 76. DB::Exception: Cannot open file /mnt/clickhouse-cache/sharedS3DiskCache/868/8683fc40b0071d63d8da5b9b9d5"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4