19818103 CustomerApi.Jobs.GenerateAccountMetrics
Queue clickhouse_account_metrics
Attempt 8 of 10
Priority 0
Tags ...
Node customer_api@10.10.0.158
Queue Time 00:00.568
Run Time 00:00.356
Inserted 6h ago
Scheduled 4h ago
Completed 4h ago (00:01)
Cancelled —
Discarded —

Args 

%{
  "account_id" => "47985",
  "date" => "2026-03-06",
  "query" => "users_reached",
  "window_days" => 7
}

Meta 

%{
  "deps" => ["generate_event_counts"],
  "name" => "generate_users_reached",
  "on_hold" => false,
  "orig_scheduled_at" => 1772759348159929,
  "recorded" => true,
  "return" => "g1AAAABUeJwrYWBgYC7nLS1OLSqOL0pNTM5ITUliYGA3LJdOKs3MSUktik8tS80riUdRkchQLpFXUBxfXFpUllqJLgcAsL0eFA",
  "structured" => true,
  "workflow" => true,
  "workflow_id" => "019cc07f-78b1-752e-83d3-e7e0a937740d"
}

Recorded Output 

%{
  users_reached: 1841,
  builder_event_users_reached: 0,
  nps_survey_users_reached: 0
}

Errors

Attempt 7—4h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ool, algorithm: Thread). (CANNOT_READ_FROM_FILE_DESCRIPTOR) (version 25.10.1.7375 (official build))\n"

    # 2
    55

    # 3
    67

    # 4
    ["r happened, or data is corrupted. Please retry the query: While executing MergeTreeSelect(pool: PrefetchedReadP", "can mean that some retryable err", "n user_id): (attempt to read data part all_21886005_22284903_36_22488012 (state Active) failed with error. It", "aller id: 86bea79a-3e47-468b-8ad5-ef68f585bc79:2109, kind: Regular, unbound: 0: (while reading colu", "ded size: 4194304, reserved size: 4194304, downloader id: None, current write offset: 780140544, ", "t info: File segment: [775946240, 780140543], key: 470c50b4c4774e4e5a68b19bd9bfc8d2, state: DOWNLOADED, downlo", "D, last caller: 86bea79a-3e47-468b-8ad5-ef68f585bc79:2109, file segme", "78461736, internal buffer end: 2333616, read_type: CACH", "wtiri, hash key: 470c50b4c4774e4e5a68b19bd9bfc8d2, file_offset_of_buffer_end: 778279856, read_until_position: ", "ror: Cache info: Buffer path: ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/dxa/fvnzpxwzupyjflqohmigxvn", "bd9bfc8d2/775946240: , errno: 5, strerror: Input/output e", "de: 74. DB::Exception: Cannot read from file /mnt/clickhouse-cache/sharedS3DiskCache/470/470c50b4c4774e4e5a68b1"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 6—4h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ial build))\n"

    # 2
    42

    # 3
    67

    # 4
    ["eSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_OPEN_FILE) (version 25.10.1.7375 (offic", "ean that some retryable error happened, or data is corrupted. Please retry the query: While executing MergeTr", "state Active) failed with error. It can ", ">): (while reading column attributes): (attempt to read data part all_21886005_22284903_36_22488012 ", "zxlwskoenbibskfb (position: 6914456917, typename: DB::AsynchronousBoundedReadBuffer, compressed data header: <uninitialize", "e2-f47e-4b75-bbcb-8441591d3836/mergetree/ral/buegwywkjlct", "or decompressing ch-s3-c6c/aa812", "e offset: 6912212992, caller id: 2beaac5e-1bc9-41eb-a78e-a79d2e75ec42:2373, kind: Regular, unbound: 0: While reading", "1bc9-41eb-a78e-a79d2e75ec42:2373, current wri", "1048576, downloader id: 2beaac5e", "487b01cc7ef2def9fca65, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size:", "78e-a79d2e75ec42:2373, file segment info: File segment: [6912212992, 6916407295], key: 8bdd623835", "last caller: 2beaac5e-1bc9-41eb-", "261568, read_type: REMOTE_FS_READ_AND_PUT_IN_CACHE,", "ca65, file_offset_of_buffer_end: 6914456917, read_until_position: 6915378339, internal buffer end: 691", "key: 8bdd623835f487b01cc7ef2def9", ": Buffer path: ch-s3-c6c/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/ral/buegwywkjlctzzxlwskoenbibskfb, hash", "e75ec42:2373, kind: Regular, unbound: 0: Cache inf", "-a78e-a79d2e75ec42:2373, current write offset: 6912212992, caller id: 2beaac5e-1bc9-41eb-a78e-a79d", "downloader id: 2beaac5e-1bc9-41e", " downloaded size: 0, reserved size: 1048576,", "ent cache state: File segment: [6912212992, 6916407295], key: 8bdd623835f487b01cc7ef2def9fca65, state: DOWNLOADING", "haredS3DiskCache/8bd/8bdd623835f487b01cc7ef2def9fca65/6912212992: , errno: 30, strerror: Read-only file system, cur", "ca65/6912212992: , errno: 30, strerror: Read-only file system: Cannot open file /mnt/clickhouse-cache/", "de: 76. DB::Exception: Cannot open file /mnt/clickhouse-cache/sharedS3DiskCache/8bd/8bdd623835f487b01cc7ef2def9"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 5—4h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ool, algorithm: Thread). (CANNOT_READ_FROM_FILE_DESCRIPTOR) (version 25.10.1.7375 (official build))\n"

    # 2
    55

    # 3
    67

    # 4
    ["r happened, or data is corrupted. Please retry the query: While executing MergeTreeSelect(pool: PrefetchedReadP", "can mean that some retryable err", "n user_id): (attempt to read data part all_21886005_22284903_36_22488012 (state Active) failed with error. It", "aller id: c7cff2b2-2ab1-4a20-9984-6567e08bdfee:1178, kind: Regular, unbound: 0: (while reading colu", "ded size: 4194304, reserved size: 4194304, downloader id: None, current write offset: 780140544, ", "t info: File segment: [775946240, 780140543], key: 470c50b4c4774e4e5a68b19bd9bfc8d2, state: DOWNLOADED, downlo", "D, last caller: c7cff2b2-2ab1-4a20-9984-6567e08bdfee:1178, file segme", "78461736, internal buffer end: 2333616, read_type: CACH", "wtiri, hash key: 470c50b4c4774e4e5a68b19bd9bfc8d2, file_offset_of_buffer_end: 778279856, read_until_position: ", "ror: Cache info: Buffer path: ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/dxa/fvnzpxwzupyjflqohmigxvn", "bd9bfc8d2/775946240: , errno: 5, strerror: Input/output e", "de: 74. DB::Exception: Cannot read from file /mnt/clickhouse-cache/sharedS3DiskCache/470/470c50b4c4774e4e5a68b1"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 4—4h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ial build))\n"

    # 2
    43

    # 3
    67

    # 4
    ["eSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_OPEN_FILE) (version 25.10.1.7375 (offic", "ean that some retryable error happened, or data is corrupted. Please retry the query: While executing MergeTr", "state Active) failed with error. It can ", ">): (while reading column attributes): (attempt to read data part all_21886005_22284903_36_22488012 ", "zxlwskoenbibskfb (position: 6914456917, typename: DB::AsynchronousBoundedReadBuffer, compressed data header: <uninitialize", "e2-f47e-4b75-bbcb-8441591d3836/mergetree/ral/buegwywkjlct", ": While reading or decompressing ch-s3-c6c/aa812", "06d-a5d1-d50013502435:1376, kind: Regular, unbound: ", "ader id: 1cd2a44f-ca6d-406d-a5d1-d50013502435:1376, current write offset: 6912212992, caller id: 1cd2a44f-ca6d-", "9fca65, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 1048576, downl", " 6916407295], key: 8bdd623835f487b01cc7ef2de", "376, file segment info: File segment: [6912212992", "TE_FS_READ_AND_PUT_IN_CACHE, last caller: 1cd2a44f-ca6d-406d-a5d1-d50013502435:", "15378339, internal buffer end: 6913261568, read_type: REM", "ash key: 8bdd623835f487b01cc7ef2def9fca65, file_offset_of_buffer_end: 6914456917, read_until_position: 6", "fer path: ch-s3-c6c/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/ral/buegwywkjlctzzxlwskoenbibskfb, ", "02435:1376, kind: Regular, unbound: 0: Cache info: Bu", "-a5d1-d50013502435:1376, current write offset: 6912212992, caller id: 1cd2a44f-ca6d-406d-a5d1-d50013", "downloader id: 1cd2a44f-ca6d-406", " downloaded size: 0, reserved size: 1048576,", "ent cache state: File segment: [6912212992, 6916407295], key: 8bdd623835f487b01cc7ef2def9fca65, state: DOWNLOADING", "haredS3DiskCache/8bd/8bdd623835f487b01cc7ef2def9fca65/6912212992: , errno: 30, strerror: Read-only file system, cur", "ca65/6912212992: , errno: 30, strerror: Read-only file system: Cannot open file /mnt/clickhouse-cache/", "de: 76. DB::Exception: Cannot open file /mnt/clickhouse-cache/sharedS3DiskCache/8bd/8bdd623835f487b01cc7ef2def9"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 3—4h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ool, algorithm: Thread). (CANNOT_READ_FROM_FILE_DESCRIPTOR) (version 25.10.1.7375 (official build))\n"

    # 2
    55

    # 3
    67

    # 4
    ["r happened, or data is corrupted. Please retry the query: While executing MergeTreeSelect(pool: PrefetchedReadP", "can mean that some retryable err", "n user_id): (attempt to read data part all_21886005_22284903_36_22488012 (state Active) failed with error. It", "aller id: 2cf1e411-c9b1-4e26-b92f-ae85f7af4ddc:1614, kind: Regular, unbound: 0: (while reading colu", "ded size: 4194304, reserved size: 4194304, downloader id: None, current write offset: 780140544, ", "t info: File segment: [775946240, 780140543], key: 470c50b4c4774e4e5a68b19bd9bfc8d2, state: DOWNLOADED, downlo", "D, last caller: 2cf1e411-c9b1-4e26-b92f-ae85f7af4ddc:1614, file segme", "78461736, internal buffer end: 2333616, read_type: CACH", "wtiri, hash key: 470c50b4c4774e4e5a68b19bd9bfc8d2, file_offset_of_buffer_end: 778279856, read_until_position: ", "ror: Cache info: Buffer path: ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/dxa/fvnzpxwzupyjflqohmigxvn", "bd9bfc8d2/775946240: , errno: 5, strerror: Input/output e", "de: 74. DB::Exception: Cannot read from file /mnt/clickhouse-cache/sharedS3DiskCache/470/470c50b4c4774e4e5a68b1"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 2—5h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ool, algorithm: Thread). (CANNOT_READ_FROM_FILE_DESCRIPTOR) (version 25.10.1.7375 (official build))\n"

    # 2
    55

    # 3
    67

    # 4
    ["r happened, or data is corrupted. Please retry the query: While executing MergeTreeSelect(pool: PrefetchedReadP", "can mean that some retryable err", "n user_id): (attempt to read data part all_21886005_22284903_36_22488012 (state Active) failed with error. It", "aller id: 3f5b7f6a-0962-4a6e-8e0c-ba5331137836:2118, kind: Regular, unbound: 0: (while reading colu", "ded size: 4194304, reserved size: 4194304, downloader id: None, current write offset: 780140544, ", "t info: File segment: [775946240, 780140543], key: 470c50b4c4774e4e5a68b19bd9bfc8d2, state: DOWNLOADED, downlo", "D, last caller: 3f5b7f6a-0962-4a6e-8e0c-ba5331137836:2118, file segme", "78461736, internal buffer end: 2333616, read_type: CACH", "wtiri, hash key: 470c50b4c4774e4e5a68b19bd9bfc8d2, file_offset_of_buffer_end: 778279856, read_until_position: ", "ror: Cache info: Buffer path: ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/dxa/fvnzpxwzupyjflqohmigxvn", "bd9bfc8d2/775946240: , errno: 5, strerror: Input/output e", "de: 74. DB::Exception: Cannot read from file /mnt/clickhouse-cache/sharedS3DiskCache/470/470c50b4c4774e4e5a68b1"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4

Attempt 1—5h ago 

** (FunctionClauseError) no function clause matching in Ch.RowBinary.decode_names/4

The following arguments were given to Ch.RowBinary.decode_names/4:

    # 1
    "ool, algorithm: Thread). (CANNOT_READ_FROM_FILE_DESCRIPTOR) (version 25.10.1.7375 (official build))\n"

    # 2
    55

    # 3
    67

    # 4
    ["r happened, or data is corrupted. Please retry the query: While executing MergeTreeSelect(pool: PrefetchedReadP", "can mean that some retryable err", "n user_id): (attempt to read data part all_21886005_22284903_36_22488012 (state Active) failed with error. It", "aller id: 1a790ded-f64b-435f-ad0a-9b9ff5c489d3:2114, kind: Regular, unbound: 0: (while reading colu", "ded size: 4194304, reserved size: 4194304, downloader id: None, current write offset: 780140544, ", "t info: File segment: [775946240, 780140543], key: 470c50b4c4774e4e5a68b19bd9bfc8d2, state: DOWNLOADED, downlo", "D, last caller: 1a790ded-f64b-435f-ad0a-9b9ff5c489d3:2114, file segme", "78461736, internal buffer end: 2333616, read_type: CACH", "wtiri, hash key: 470c50b4c4774e4e5a68b19bd9bfc8d2, file_offset_of_buffer_end: 778279856, read_until_position: ", "ror: Cache info: Buffer path: ch-s3-30f/aa8129e2-f47e-4b75-bbcb-8441591d3836/mergetree/dxa/fvnzpxwzupyjflqohmigxvn", "bd9bfc8d2/775946240: , errno: 5, strerror: Input/output e", "de: 74. DB::Exception: Cannot read from file /mnt/clickhouse-cache/sharedS3DiskCache/470/470c50b4c4774e4e5a68b1"]

    (ch 0.7.1) lib/ch/row_binary.ex:789: Ch.RowBinary.decode_names/4
    (ch 0.7.1) lib/ch/query.ex:290: DBConnection.Query.Ch.Query.decode/3
    (db_connection 2.9.0) lib/db_connection.ex:1470: DBConnection.decode/4
    (db_connection 2.9.0) lib/db_connection.ex:849: DBConnection.execute/4
    (ch 0.7.1) lib/ch.ex:94: Ch.query/4
    (ecto_sql 3.13.4) lib/ecto/adapters/sql.ex:620: Ecto.Adapters.SQL.query!/4
    (ecto_ch 0.8.6) lib/ecto/adapters/clickhouse.ex:323: Ecto.Adapters.ClickHouse.execute/5
    (ecto 3.13.5) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4