JUnit version 4.13.2
.12:38:12.486 [INFO - main] (RandomUtils.java:49) Using random seed: 1786451892481
12:38:12.538 [INFO - main] (RetryRule.java:138) Creating RetryStatement with result reporter and retry count of 0 (no retries)
12:38:12.608 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
12:38:12.627 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-task3TN6t_/test-tmp
12:38:12.630 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/asan/bin
12:38:12.635 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-task3TN6t_/java/build/dist-test/../../../build/asan/bin/kudu, test, mini_cluster, --serialization=pb]
12:38:12.886 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
  num_masters: 3
  num_tservers: 3
  enable_kerberos: false
  cluster_root: "/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414"
  hms_mode: NONE
  mini_kdc_options {
  }
  principal: "kudu"
  mini_oidc_options {
  }
}

12:38:13.083 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
12:38:13.085 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}

12:38:13.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.089968 19867 mini_oidc.cc:152] Starting JWKS server
12:38:13.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.094190 19867 webserver.cc:460] Webserver started at http://127.0.0.1:36603/ using document root <none> and password file <none>
12:38:13.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.094575 19867 mini_oidc.cc:189] Starting OIDC Discovery server
12:38:13.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.095589 19867 webserver.cc:460] Webserver started at http://127.0.0.1:41017/ using document root <none> and password file <none>
12:38:13.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-11T12:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
12:38:13.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-11T12:38:13Z Disabled control of system clock
12:38:13.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.135375 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:13.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:13.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:13.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/logs
12:38:13.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data/info.pb
12:38:13.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:13.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:13.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
12:38:13.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.254:39737
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.254
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:13.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
12:38:13.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36603/jwks.json with env {}
12:38:13.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.516115 19882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:13.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.516472 19882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:13.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.516570 19882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:13.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.525177 19882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
12:38:13.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.525327 19882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:13.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.525375 19882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
12:38:13.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.525427 19882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
12:38:13.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.525471 19882 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
12:38:13.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.525525 19882 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:36603/jwks.json
12:38:13.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.537820 19882 master_runner.cc:386] Master server non-default flags:
12:38:13.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:13.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:13.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:13.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:13.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data
12:38:13.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.254:39737
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:13.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36603/jwks.json
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data/info.pb
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.254
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:13.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/logs
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:13.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:13.543 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:13.543 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.540096 19882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:13.543 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.542793 19882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:13.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.557555 19895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:13.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.557540 19893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:13.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.557714 19882 server_base.cc:1034] running on GCE node
12:38:13.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.557976 19892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:13.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.559656 19882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:13.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.561430 19882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:13.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.562695 19882 hybrid_clock.cc:648] HybridClock initialized: now 1786451893562621 us; error 44 us; skew 500 ppm
12:38:13.563 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.563041 19882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:13.566 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.565920 19882 webserver.cc:460] Webserver started at http://127.19.102.254:36633/ using document root <none> and password file <none>
12:38:13.566 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.566663 19882 fs_manager.cc:359] Metadata directory not provided
12:38:13.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.566780 19882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:13.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.567067 19882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
12:38:13.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.569990 19882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data/instance:
12:38:13.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "989ce9c8a2d4455db0e751ea56e08c67"
12:38:13.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:13 on dist-test-slave-t7g5"
12:38:13.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.571226 19882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal/instance:
12:38:13.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "989ce9c8a2d4455db0e751ea56e08c67"
12:38:13.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:13 on dist-test-slave-t7g5"
12:38:13.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.578177 19882 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
12:38:13.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.582660 19902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:13.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.584692 19882 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
12:38:13.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.584892 19882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "989ce9c8a2d4455db0e751ea56e08c67"
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:13 on dist-test-slave-t7g5"
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.585127 19882 fs_report.cc:383] FS layout report
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data/data
12:38:13.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:13.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:13.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:13.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:13.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.603320 19882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:13.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.604473 19882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:13.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.604858 19882 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:13.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.626368 19882 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.254:39737
12:38:13.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.626446 19994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.254:39737 every 8 connection(s)
12:38:13.629 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.628878 19882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/data/info.pb
12:38:13.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.634326 19995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
12:38:13.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.638093 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 19882
12:38:13.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.638433 19867 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-0/wal/instance
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.640956 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:13.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/logs
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data/info.pb
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:13.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.253:33449
12:38:13.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.253
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:13.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
12:38:13.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36603/jwks.json with env {}
12:38:13.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.643657 19995 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
12:38:13.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.667467 19995 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } has no permanent_uuid. Determining permanent_uuid...
12:38:13.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.669337 19905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.253:33449: connect: Connection refused (error 111)
12:38:13.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.671591 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.253:33449: Network error: Client connection negotiation failed: client connection to 127.19.102.253:33449: connect: Connection refused (error 111)
12:38:13.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.721280 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } attempt: 1
12:38:13.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.724355 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.253:33449: Network error: Client connection negotiation failed: client connection to 127.19.102.253:33449: connect: Connection refused (error 111)
12:38:13.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.792725 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } attempt: 2
12:38:13.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.795256 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.253:33449: Network error: Client connection negotiation failed: client connection to 127.19.102.253:33449: connect: Connection refused (error 111)
12:38:13.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:13.886636 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } attempt: 3
12:38:13.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:13.891085 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.253:33449: Network error: Client connection negotiation failed: client connection to 127.19.102.253:33449: connect: Connection refused (error 111)
12:38:14.029 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.028795 19998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:14.030 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.029300 19998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:14.030 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.029429 19998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:14.035 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.034734 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } attempt: 4
12:38:14.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.037367 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.253:33449: Network error: Client connection negotiation failed: client connection to 127.19.102.253:33449: connect: Connection refused (error 111)
12:38:14.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.038666 19998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
12:38:14.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.038784 19998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:14.039 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.038827 19998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
12:38:14.039 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.038878 19998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
12:38:14.039 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.038919 19998 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
12:38:14.039 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.038970 19998 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:36603/jwks.json
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.048633 19998 master_runner.cc:386] Master server non-default flags:
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
12:38:14.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.253:33449
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36603/jwks.json
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data/info.pb
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.253
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:14.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:14.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:14.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:14.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:14.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/logs
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:14.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:14.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:14.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:14.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:14.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.050190 19998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:14.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.052162 19998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:14.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.065255 20010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:14.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.065287 20015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:14.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.065315 20013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:14.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.066608 19998 server_base.cc:1034] running on GCE node
12:38:14.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.067607 19998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:14.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.069439 19998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:14.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.071096 19998 hybrid_clock.cc:648] HybridClock initialized: now 1786451894071015 us; error 85 us; skew 500 ppm
12:38:14.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.071605 19998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:14.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.075276 19998 webserver.cc:460] Webserver started at http://127.19.102.253:35343/ using document root <none> and password file <none>
12:38:14.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.076228 19998 fs_manager.cc:359] Metadata directory not provided
12:38:14.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.076378 19998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:14.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.076828 19998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
12:38:14.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.079888 19998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data/instance:
12:38:14.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "1077711000954cb9bdb3bd2f02a38e28"
12:38:14.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:14 on dist-test-slave-t7g5"
12:38:14.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.080871 19998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal/instance:
12:38:14.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "1077711000954cb9bdb3bd2f02a38e28"
12:38:14.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:14 on dist-test-slave-t7g5"
12:38:14.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.087960 19998 fs_manager.cc:691] Time spent creating directory manager: real 0.007s	user 0.009s	sys 0.000s
12:38:14.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.092980 20022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:14.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.095028 19998 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
12:38:14.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.095247 19998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal
12:38:14.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "1077711000954cb9bdb3bd2f02a38e28"
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:14 on dist-test-slave-t7g5"
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.095705 19998 fs_report.cc:383] FS layout report
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data/data
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:14.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:14.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:14.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:14.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:14.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:14.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.136475 19998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:14.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.137562 19998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:14.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.137979 19998 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:14.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.159106 19998 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.253:33449
12:38:14.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.159129 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.253:33449 every 8 connection(s)
12:38:14.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.160778 19998 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/data/info.pb
12:38:14.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.163563 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 19998
12:38:14.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.163780 19867 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-1/wal/instance
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.165100 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:14.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/logs
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data/info.pb
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.252:36537
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.252
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:14.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:14.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:14.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
12:38:14.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:14.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
12:38:14.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36603/jwks.json with env {}
12:38:14.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.165869 20109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
12:38:14.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.174492 20109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.197081 20109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.212206 20109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.213721 20026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.216 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.215833 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.265 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.265326 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 1
12:38:14.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.268209 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.337 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.336553 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 2
12:38:14.337 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.336740 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } attempt: 5
12:38:14.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.339375 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.346 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.345477 19995 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.348150 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.399493 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 1
12:38:14.402 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.402308 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.430639 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 3
12:38:14.433 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.433282 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.470602 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 2
12:38:14.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.472654 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.568224 20111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:14.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.568559 20111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:14.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.568648 20111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:14.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.576565 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 4
12:38:14.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.577899 20111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
12:38:14.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.578088 20111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:14.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.578164 20111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
12:38:14.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.578248 20111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
12:38:14.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.578308 20111 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
12:38:14.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.578390 20111 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:36603/jwks.json
12:38:14.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.578956 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 3
12:38:14.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.579578 20109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.580976 19995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.102.252:36537: Network error: Client connection negotiation failed: client connection to 127.19.102.252:36537: connect: Connection refused (error 111)
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.589936 20111 master_runner.cc:386] Master server non-default flags:
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:14.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.252:36537
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36603/jwks.json
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:14.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data/info.pb
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.252
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/logs
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
12:38:14.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:14.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:14.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:14.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:14.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:14.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:14.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.591554 20111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:14.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.594101 20111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:14.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.607990 20127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:14.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.608073 20128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:14.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.608033 20131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:14.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.609045 20111 server_base.cc:1034] running on GCE node
12:38:14.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.610083 20111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:14.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.611749 20111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:14.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.613073 20111 hybrid_clock.cc:648] HybridClock initialized: now 1786451894612974 us; error 73 us; skew 500 ppm
12:38:14.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.613691 20111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:14.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.616797 20111 webserver.cc:460] Webserver started at http://127.19.102.252:36689/ using document root <none> and password file <none>
12:38:14.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.617754 20111 fs_manager.cc:359] Metadata directory not provided
12:38:14.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.617899 20111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:14.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.618294 20111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
12:38:14.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.622641 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data/instance:
12:38:14.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "46a51c8bac89417e84212270593211ff"
12:38:14.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:14 on dist-test-slave-t7g5"
12:38:14.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.624238 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal/instance:
12:38:14.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "46a51c8bac89417e84212270593211ff"
12:38:14.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:14 on dist-test-slave-t7g5"
12:38:14.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.631394 20111 fs_manager.cc:691] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
12:38:14.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.635258 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:14.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.637072 20111 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
12:38:14.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.637288 20111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal
12:38:14.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "46a51c8bac89417e84212270593211ff"
12:38:14.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:14 on dist-test-slave-t7g5"
12:38:14.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.637547 20111 fs_report.cc:383] FS layout report
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data/data
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:14.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:14.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:14.660 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.659797 20111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:14.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.661186 20111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:14.661 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.661595 20111 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:14.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.683462 20111 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.252:36537
12:38:14.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.683530 20216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.252:36537 every 8 connection(s)
12:38:14.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.685714 20111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/data/info.pb
12:38:14.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.690629 20217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
12:38:14.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.691254 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 20111
12:38:14.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.691465 19867 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/master-2/wal/instance
12:38:14.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.698175 20217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.716097 20217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.725065 20217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } has no permanent_uuid. Determining permanent_uuid...
12:38:14.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.739058 20217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff: Bootstrap starting.
12:38:14.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.743722 20217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff: Neither blocks nor log segments found. Creating new log.
12:38:14.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.745950 20217 log.cc:824] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff: Log is configured to *not* fsync() on all Append() calls
12:38:14.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.749435 20217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff: No bootstrap required, opened a new log
12:38:14.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.757984 20217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.758232 19995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 4
12:38:14.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.758409 20217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
12:38:14.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.758535 20217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a51c8bac89417e84212270593211ff, State: Initialized, Role: FOLLOWER
12:38:14.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.759398 20217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.760363 20232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:14.761 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.760821 20232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: This master's current role is: FOLLOWER
12:38:14.762 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.761806 20217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: configured and running, proceeding with master startup.
12:38:14.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.771831 19995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: Bootstrap starting.
12:38:14.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.774708 20249 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
12:38:14.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.776403 19995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: Neither blocks nor log segments found. Creating new log.
12:38:14.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.778431 19995 log.cc:824] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: Log is configured to *not* fsync() on all Append() calls
12:38:14.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.786172 20249 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
12:38:14.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.771755 20111 master_runner.cc:186] Error getting master registration for 127.19.102.254:39737: OK, instance_id { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" instance_seqno: 1786451893620522 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
12:38:14.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.788013 20111 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
12:38:14.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.788901 19995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: No bootstrap required, opened a new log
12:38:14.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.797154 19995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.797559 19995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
12:38:14.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.797731 19995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989ce9c8a2d4455db0e751ea56e08c67, State: Initialized, Role: FOLLOWER
12:38:14.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.798408 19995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.799404 20255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:14.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.799979 20255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: This master's current role is: FOLLOWER
12:38:14.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.800930 19995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: configured and running, proceeding with master startup.
12:38:14.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.811937 20269 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
12:38:14.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.812057 20269 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
12:38:14.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.816881 20232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:14.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.817265 20232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.818936 20232 leader_election.cc:288] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 989ce9c8a2d4455db0e751ea56e08c67 (127.19.102.254:39737), 1077711000954cb9bdb3bd2f02a38e28 (127.19.102.253:33449)
12:38:14.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.821952 19947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46a51c8bac89417e84212270593211ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989ce9c8a2d4455db0e751ea56e08c67" is_pre_election: true
12:38:14.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.822638 19947 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a51c8bac89417e84212270593211ff in term 0.
12:38:14.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.811601 19882 master_runner.cc:186] Error getting master registration for 127.19.102.253:33449: OK, instance_id { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" instance_seqno: 1786451894153108 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
12:38:14.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.823489 19882 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
12:38:14.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.823473 20146 leader_election.cc:302] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a51c8bac89417e84212270593211ff, 989ce9c8a2d4455db0e751ea56e08c67; no voters: 
12:38:14.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.823931 20232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Leader pre-election won for term 1
12:38:14.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.824312 20232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
12:38:14.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.824432 20232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 0 FOLLOWER]: Advancing to term 1
12:38:14.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.827941 20232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.829736 19947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46a51c8bac89417e84212270593211ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989ce9c8a2d4455db0e751ea56e08c67"
12:38:14.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.830000 19947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 0 FOLLOWER]: Advancing to term 1
12:38:14.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.829982 20232 leader_election.cc:288] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [CANDIDATE]: Term 1 election: Requested vote from peers 989ce9c8a2d4455db0e751ea56e08c67 (127.19.102.254:39737), 1077711000954cb9bdb3bd2f02a38e28 (127.19.102.253:33449)
12:38:14.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.833353 19947 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a51c8bac89417e84212270593211ff in term 1.
12:38:14.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.833889 20146 leader_election.cc:302] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a51c8bac89417e84212270593211ff, 989ce9c8a2d4455db0e751ea56e08c67; no voters: 
12:38:14.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.835800 20232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 1 FOLLOWER]: Leader election won for term 1
12:38:14.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.837476 20232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [term 1 LEADER]: Becoming Leader. State: Replica: 46a51c8bac89417e84212270593211ff, State: Running, Role: LEADER
12:38:14.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.838817 20232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.842808 20272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a51c8bac89417e84212270593211ff. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:14.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.843452 20272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: This master's current role is: LEADER
12:38:14.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.845039 20275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
12:38:14.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.850548 20275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
12:38:14.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:14.859328 19909 tablet.cc:2369] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: Can't schedule compaction. Clean time has not been advanced past its initial value.
12:38:14.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.867292 19947 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [term 1 FOLLOWER]: Refusing update from remote peer 46a51c8bac89417e84212270593211ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
12:38:14.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.868487 20272 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
12:38:14.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.878794 20109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } attempt: 5
12:38:14.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.896160 20255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a51c8bac89417e84212270593211ff. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:14.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.896507 20255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: This master's current role is: FOLLOWER
12:38:14.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.899425 20109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: Bootstrap starting.
12:38:14.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.911831 20109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: Neither blocks nor log segments found. Creating new log.
12:38:14.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.915129 20109 log.cc:824] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: Log is configured to *not* fsync() on all Append() calls
12:38:14.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.918834 20255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:14.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.919188 20255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67 [sys.catalog]: This master's current role is: FOLLOWER
12:38:14.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.921766 20276 mvcc.cc:205] Tried to move back new op lower bound from 7317306961356054528 to 7317306961266302976. Current Snapshot: MvccSnapshot[applied={T|T < 7317306961356054528}]
12:38:14.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.928449 20232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:14.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.929041 20232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: This master's current role is: LEADER
12:38:14.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.930599 20275 catalog_manager.cc:1347] Generated new cluster ID: b1f5400145ad494fa69759c2dc2558cd
12:38:14.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.930727 20275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
12:38:14.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.953367 20109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: No bootstrap required, opened a new log
12:38:14.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.981123 20109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.982097 20109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
12:38:14.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.982460 20109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1077711000954cb9bdb3bd2f02a38e28, State: Initialized, Role: FOLLOWER
12:38:14.985 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.985132 20109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:14.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.986976 20275 catalog_manager.cc:1370] Generated new certificate authority record
12:38:14.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.991895 20275 catalog_manager.cc:1504] Loading token signing keys...
12:38:14.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.998405 20284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:15.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:14.998967 20284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: This master's current role is: FOLLOWER
12:38:15.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.002871 20109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: configured and running, proceeding with master startup.
12:38:15.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.003574 20068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 0 FOLLOWER]: Advancing to term 1
12:38:15.010 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.009840 20068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 1 FOLLOWER]: Refusing update from remote peer 46a51c8bac89417e84212270593211ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
12:38:15.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.014346 20069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46a51c8bac89417e84212270593211ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1077711000954cb9bdb3bd2f02a38e28"
12:38:15.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.014720 20070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46a51c8bac89417e84212270593211ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1077711000954cb9bdb3bd2f02a38e28" is_pre_election: true
12:38:15.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.021991 20275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff: Generated new TSK 0
12:38:15.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.022300 20284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:15.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.022701 20284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } }
12:38:15.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.022924 20272 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
12:38:15.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.024426 20275 catalog_manager.cc:1514] Initializing in-progress tserver states...
12:38:15.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.025306 20284 leader_election.cc:288] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 989ce9c8a2d4455db0e751ea56e08c67 (127.19.102.254:39737), 46a51c8bac89417e84212270593211ff (127.19.102.252:36537)
12:38:15.029 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.027509 19947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1077711000954cb9bdb3bd2f02a38e28" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989ce9c8a2d4455db0e751ea56e08c67" is_pre_election: true
12:38:15.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.026320 20182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1077711000954cb9bdb3bd2f02a38e28" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a51c8bac89417e84212270593211ff" is_pre_election: true
12:38:15.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.039143 20026 leader_election.cc:302] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1077711000954cb9bdb3bd2f02a38e28; no voters: 46a51c8bac89417e84212270593211ff, 989ce9c8a2d4455db0e751ea56e08c67
12:38:15.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.047246 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:15.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:15.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:0
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:15.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:15.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.098026 20284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a51c8bac89417e84212270593211ff. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:15.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.098443 20284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: This master's current role is: FOLLOWER
12:38:15.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.098657 20288 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
12:38:15.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.099102 20298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:15.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.099622 20298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28 [sys.catalog]: This master's current role is: FOLLOWER
12:38:15.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.202442 20314 catalog_manager.cc:1259] Loaded cluster ID: b1f5400145ad494fa69759c2dc2558cd
12:38:15.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.202616 20314 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: loading cluster ID for follower catalog manager: success
12:38:15.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.215327 20314 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: acquiring CA information for follower catalog manager: success
12:38:15.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.235867 20314 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1077711000954cb9bdb3bd2f02a38e28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
12:38:15.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.453413 20272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46a51c8bac89417e84212270593211ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989ce9c8a2d4455db0e751ea56e08c67" member_type: VOTER last_known_addr { host: "127.19.102.254" port: 39737 } } peers { permanent_uuid: "1077711000954cb9bdb3bd2f02a38e28" member_type: VOTER last_known_addr { host: "127.19.102.253" port: 33449 } } peers { permanent_uuid: "46a51c8bac89417e84212270593211ff" member_type: VOTER last_known_addr { host: "127.19.102.252" port: 36537 } } }
12:38:15.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.453819 20272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff [sys.catalog]: This master's current role is: LEADER
12:38:15.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.566793 20294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:15.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.567126 20294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:15.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.567246 20294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:15.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.575371 20294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:15.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.575635 20294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.193
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.585774 20294 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:15.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:0
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:15.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:15.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:15.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:15.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:15.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:15.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:15.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.587656 20294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:15.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.589818 20294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:15.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.608656 20324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:15.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.609597 20325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:15.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:15.611532 20327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:15.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.612717 20294 server_base.cc:1034] running on GCE node
12:38:15.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.613709 20294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:15.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.615000 20294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:15.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.618494 20294 hybrid_clock.cc:648] HybridClock initialized: now 1786451895618355 us; error 90 us; skew 500 ppm
12:38:15.619 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.619071 20294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:15.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.622521 20294 webserver.cc:460] Webserver started at http://127.19.102.193:41923/ using document root <none> and password file <none>
12:38:15.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.623389 20294 fs_manager.cc:359] Metadata directory not provided
12:38:15.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.623533 20294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:15.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.623908 20294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
12:38:15.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.627171 20294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/instance:
12:38:15.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:15.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:15 on dist-test-slave-t7g5"
12:38:15.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.628159 20294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal/instance:
12:38:15.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:15.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:15 on dist-test-slave-t7g5"
12:38:15.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.636320 20294 fs_manager.cc:691] Time spent creating directory manager: real 0.008s	user 0.009s	sys 0.000s
12:38:15.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.640118 20334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:15.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.641736 20294 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.641949 20294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:15 on dist-test-slave-t7g5"
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.642180 20294 fs_report.cc:383] FS layout report
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:15.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/data
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:15.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:15.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.658217 20294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:15.659 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.659216 20294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:15.659 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.659556 20294 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:15.660 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.660724 20294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:15.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.662588 20294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
12:38:15.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.662671 20294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
12:38:15.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.662802 20294 ts_tablet_manager.cc:610] Registered 0 tablets
12:38:15.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.662861 20294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
12:38:15.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.710855 20294 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.193:34365
12:38:15.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.710878 20506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.193:34365 every 8 connection(s)
12:38:15.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.712687 20294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:15.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.717504 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 20294
12:38:15.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.717744 19867 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal/instance
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.724839 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:15.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:0
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:15.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:15.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:15.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:15.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:15.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.744576 20507 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:15.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.744596 20508 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:15.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.745012 20507 heartbeater.cc:464] Registering TS with master...
12:38:15.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.746055 20507 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:15.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.745020 20508 heartbeater.cc:464] Registering TS with master...
12:38:15.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.747565 20508 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:15.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.748814 20165 ts_manager.cc:194] Registered new tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:15.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.749477 19931 ts_manager.cc:194] Registered new tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:15.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.750653 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.193:47945
12:38:15.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.752251 20510 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:15.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.752460 20510 heartbeater.cc:464] Registering TS with master...
12:38:15.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.753051 20510 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:15.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.755577 20056 ts_manager.cc:194] Registered new tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:15.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.814615 20269 catalog_manager.cc:1259] Loaded cluster ID: b1f5400145ad494fa69759c2dc2558cd
12:38:15.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.814791 20269 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: loading cluster ID for follower catalog manager: success
12:38:15.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.818533 20269 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: acquiring CA information for follower catalog manager: success
12:38:15.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:15.820886 20269 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 989ce9c8a2d4455db0e751ea56e08c67: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
12:38:16.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.130162 20524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:16.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.130474 20524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:16.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.130561 20524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:16.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.138446 20524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:16.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.138656 20524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.194
12:38:16.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.148461 20524 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:16.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:16.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:0
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:16.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:16.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.150025 20524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:16.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.152254 20524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:16.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.171970 20534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:16.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.171939 20535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:16.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.172009 20537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:16.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.174434 20524 server_base.cc:1034] running on GCE node
12:38:16.175 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.175230 20524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:16.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.176301 20524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:16.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.177603 20524 hybrid_clock.cc:648] HybridClock initialized: now 1786451896177497 us; error 60 us; skew 500 ppm
12:38:16.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.178006 20524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:16.181 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.180873 20524 webserver.cc:460] Webserver started at http://127.19.102.194:44599/ using document root <none> and password file <none>
12:38:16.181 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.181624 20524 fs_manager.cc:359] Metadata directory not provided
12:38:16.182 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.181840 20524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:16.182 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.182158 20524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
12:38:16.185 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.185139 20524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/instance:
12:38:16.185 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:16.185 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:16.186 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.186144 20524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal/instance:
12:38:16.186 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:16.186 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:16.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.195432 20524 fs_manager.cc:691] Time spent creating directory manager: real 0.009s	user 0.007s	sys 0.001s
12:38:16.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.200439 20543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:16.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.203344 20524 fs_manager.cc:725] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.000s
12:38:16.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.203590 20524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:16.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:16.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.203868 20524 fs_report.cc:383] FS layout report
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/data
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:16.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:16.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.235436 20524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:16.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.236532 20524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:16.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.236919 20524 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:16.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.238183 20524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:16.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.240048 20524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
12:38:16.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.240139 20524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
12:38:16.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.240268 20524 ts_tablet_manager.cc:610] Registered 0 tablets
12:38:16.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.240329 20524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
12:38:16.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.277496 20524 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.194:44869
12:38:16.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.277601 20708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.194:44869 every 8 connection(s)
12:38:16.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.279181 20524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:16.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.289189 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 20524
12:38:16.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.289374 19867 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal/instance
12:38:16.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.294751 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:0
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:16.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:16.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:16.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:16.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.300022 20710 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:16.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.300051 20709 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:16.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.300431 20709 heartbeater.cc:464] Registering TS with master...
12:38:16.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.300776 20713 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:16.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.300909 20713 heartbeater.cc:464] Registering TS with master...
12:38:16.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.301600 20713 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:16.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.300427 20710 heartbeater.cc:464] Registering TS with master...
12:38:16.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.301646 20709 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:16.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.302217 20710 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:16.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.303515 20165 ts_manager.cc:194] Registered new tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:16.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.303673 19931 ts_manager.cc:194] Registered new tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:16.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.303661 20056 ts_manager.cc:194] Registered new tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:16.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.304559 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.194:44855
12:38:16.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.692745 20729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:16.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.693069 20729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:16.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.693154 20729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:16.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.702137 20729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:16.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.702426 20729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.195
12:38:16.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.716059 20729 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:16.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:16.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:16.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:0
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:16.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:16.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.717434 20729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:16.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.719668 20729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:16.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.734095 20743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:16.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.734071 20739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:16.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:16.734043 20740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:16.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.735164 20729 server_base.cc:1034] running on GCE node
12:38:16.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.736266 20729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:16.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.737785 20729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:16.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.739117 20729 hybrid_clock.cc:648] HybridClock initialized: now 1786451896739017 us; error 56 us; skew 500 ppm
12:38:16.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.739598 20729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:16.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.750653 20729 webserver.cc:460] Webserver started at http://127.19.102.195:43649/ using document root <none> and password file <none>
12:38:16.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.751820 20729 fs_manager.cc:359] Metadata directory not provided
12:38:16.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.751973 20729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:16.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.752378 20729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
12:38:16.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.755250 20507 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:16.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.755616 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/instance:
12:38:16.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:16.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:16.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.756541 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal/instance:
12:38:16.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:16.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:16.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.763235 20729 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.008s	sys 0.000s
12:38:16.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.767786 20755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:16.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.769737 20729 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.004s
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.769972 20729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.770222 20729 fs_report.cc:383] FS layout report
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/data
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:16.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:16.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:16.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:16.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:16.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.806846 20729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:16.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.807898 20729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:16.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.808238 20729 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:16.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.809453 20729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:16.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.811371 20729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
12:38:16.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.811460 20729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
12:38:16.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.811586 20729 ts_tablet_manager.cc:610] Registered 0 tablets
12:38:16.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.811645 20729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
12:38:16.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.856664 20729 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.195:36907
12:38:16.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.856752 20940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.195:36907 every 8 connection(s)
12:38:16.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.858143 20729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:16.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.862183 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 20729
12:38:16.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.862403 19867 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal/instance
12:38:16.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.892166 20943 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:16.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.892233 20941 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:16.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.892258 20944 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:16.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.892648 20944 heartbeater.cc:464] Registering TS with master...
12:38:16.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.892648 20943 heartbeater.cc:464] Registering TS with master...
12:38:16.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.892733 20941 heartbeater.cc:464] Registering TS with master...
12:38:16.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.894124 20944 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:16.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.894203 20943 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:16.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.894229 20941 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:16.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.896569 20165 ts_manager.cc:194] Registered new tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:16.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.896727 19931 ts_manager.cc:194] Registered new tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:16.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.897624 20056 ts_manager.cc:194] Registered new tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:16.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.897794 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.195:50711
12:38:16.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:16.903625 19867 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
12:38:16.906 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
12:38:16.907 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}

12:38:16.920 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
  masters {
    id {
      type: MASTER
      index: 0
    }
    bound_rpc_address {
      host: "127.19.102.254"
      port: 39737
    }
    bound_http_address {
      host: "127.19.102.254"
      port: 36633
    }
  }
  masters {
    id {
      type: MASTER
      index: 1
    }
    bound_rpc_address {
      host: "127.19.102.253"
      port: 33449
    }
    bound_http_address {
      host: "127.19.102.253"
      port: 35343
    }
  }
  masters {
    id {
      type: MASTER
      index: 2
    }
    bound_rpc_address {
      host: "127.19.102.252"
      port: 36537
    }
    bound_http_address {
      host: "127.19.102.252"
      port: 36689
    }
  }
}

12:38:16.929 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}

12:38:16.933 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
  tservers {
    id {
      type: TSERVER
      index: 0
    }
    bound_rpc_address {
      host: "127.19.102.193"
      port: 34365
    }
    bound_http_address {
      host: "127.19.102.193"
      port: 41923
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 1
    }
    bound_rpc_address {
      host: "127.19.102.194"
      port: 44869
    }
    bound_http_address {
      host: "127.19.102.194"
      port: 44599
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 2
    }
    bound_rpc_address {
      host: "127.19.102.195"
      port: 36907
    }
    bound_http_address {
      host: "127.19.102.195"
      port: 43649
    }
  }
}

12:38:16.934 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
12:38:17.141 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication 
12:38:17.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:17.307122 20709 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:17.343 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.19.102.253:33449(127.19.102.253:33449)] connecting to peer
12:38:17.412 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.19.102.252:36537(127.19.102.252:36537)] connecting to peer
12:38:17.414 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.19.102.254:39737(127.19.102.254:39737)] connecting to peer
12:38:17.453 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.19.102.253:33449(127.19.102.253:33449)] Successfully connected to peer
12:38:17.453 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.19.102.252:36537(127.19.102.252:36537)] Successfully connected to peer
12:38:17.538 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.19.102.254:39737(127.19.102.254:39737)] Successfully connected to peer
12:38:17.566 [DEBUG - kudu-nio-1] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.19.102.252
12:38:17.566 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.19.102.253
12:38:17.675 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.19.102.254
12:38:17.819 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x2d41a15f, L:/127.0.0.1:58814 - R:/127.19.102.253:33449] using SASL/PLAIN
12:38:17.819 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0xc21e36f7, L:/127.0.0.1:36766 - R:/127.19.102.252:36537] using SASL/PLAIN
12:38:17.823 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x9988708b, L:/127.0.0.1:57036 - R:/127.19.102.254:39737] using SASL/PLAIN
12:38:17.869 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet Kudu Master for table 'Kudu Master' with partition [<start>, <end>)
12:38:17.870 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=CreateTable, tablet=null, attempt=1, TimeoutTracker(timeout=50000, elapsed=571), Traces: [0ms] refreshing cache from master, [31ms] Sub RPC ConnectToMaster: sending RPC to server master-127.19.102.253:33449, [109ms] Sub RPC ConnectToMaster: sending RPC to server master-127.19.102.252:36537, [111ms] Sub RPC ConnectToMaster: sending RPC to server master-127.19.102.254:39737, [530ms] Sub RPC ConnectToMaster: received response from server master-127.19.102.253:33449: OK, [532ms] Sub RPC ConnectToMaster: received response from server master-127.19.102.252:36537: OK, [534ms] Sub RPC ConnectToMaster: received response from server master-127.19.102.254:39737: OK, deferred=null) after lookup
12:38:17.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:17.901273 20941 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:17.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:17.937107 20165 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36766:
12:38:17.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) name: "org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283"
12:38:17.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) schema {
12:38:17.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
12:38:17.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "key"
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: true
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column1_i"
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column2_i"
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column3_s"
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: STRING
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: true
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     encoding: DICT_ENCODING
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     compression: LZ4
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 4096
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column4_b"
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: BOOL
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
12:38:17.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) partition_schema {
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   range_schema {
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     columns {
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)       name: "key"
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     }
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
12:38:17.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
12:38:17.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:17.939695 20165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
12:38:17.994 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:894) Opened table 38787831ef4042aa89489d62d177324b
12:38:18.002 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.001534 20829 tablet_service.cc:1467] Processing CreateTablet for tablet 77cb47d5bfc441bd91b99ae31dc3eed3 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283 [id=38787831ef4042aa89489d62d177324b]), partition=RANGE (key) PARTITION UNBOUNDED
12:38:18.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.003194 20619 tablet_service.cc:1467] Processing CreateTablet for tablet 77cb47d5bfc441bd91b99ae31dc3eed3 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283 [id=38787831ef4042aa89489d62d177324b]), partition=RANGE (key) PARTITION UNBOUNDED
12:38:18.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.005720 20619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77cb47d5bfc441bd91b99ae31dc3eed3. 1 dirs total, 0 dirs full, 0 dirs failed
12:38:18.008 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.008816 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77cb47d5bfc441bd91b99ae31dc3eed3. 1 dirs total, 0 dirs full, 0 dirs failed
12:38:18.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.012928 20403 tablet_service.cc:1467] Processing CreateTablet for tablet 77cb47d5bfc441bd91b99ae31dc3eed3 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283 [id=38787831ef4042aa89489d62d177324b]), partition=RANGE (key) PARTITION UNBOUNDED
12:38:18.021 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.014605 20403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77cb47d5bfc441bd91b99ae31dc3eed3. 1 dirs total, 0 dirs full, 0 dirs failed
12:38:18.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.027031 20987 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap starting.
12:38:18.029 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.029206 20985 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap starting.
12:38:18.030 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.030223 20990 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap starting.
12:38:18.031 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.031083 20987 tablet_bootstrap.cc:654] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Neither blocks nor log segments found. Creating new log.
12:38:18.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.032418 20987 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Log is configured to *not* fsync() on all Append() calls
12:38:18.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.033236 20985 tablet_bootstrap.cc:654] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Neither blocks nor log segments found. Creating new log.
12:38:18.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.033607 20990 tablet_bootstrap.cc:654] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Neither blocks nor log segments found. Creating new log.
12:38:18.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.034615 20990 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Log is configured to *not* fsync() on all Append() calls
12:38:18.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.034598 20985 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Log is configured to *not* fsync() on all Append() calls
12:38:18.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.035985 20987 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: No bootstrap required, opened a new log
12:38:18.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.036343 20987 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent bootstrapping tablet: real 0.010s	user 0.007s	sys 0.000s
12:38:18.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.037705 20985 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: No bootstrap required, opened a new log
12:38:18.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.037845 20990 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: No bootstrap required, opened a new log
12:38:18.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.037994 20985 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent bootstrapping tablet: real 0.009s	user 0.008s	sys 0.000s
12:38:18.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.038087 20990 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.003s
12:38:18.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.049054 20987 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.049953 20987 raft_consensus.cc:374] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
12:38:18.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.050144 20987 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Initialized, Role: FOLLOWER
12:38:18.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.050215 20985 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.050961 20985 raft_consensus.cc:374] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
12:38:18.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.051151 20985 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Initialized, Role: FOLLOWER
12:38:18.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.051343 20987 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.052284 20985 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.056579 20990 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.057700 20990 raft_consensus.cc:374] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
12:38:18.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.058064 20990 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Initialized, Role: FOLLOWER
12:38:18.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.060686 20990 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.061287 20987 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.010s
12:38:18.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.062111 20985 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent starting tablet: real 0.024s	user 0.019s	sys 0.004s
12:38:18.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.073464 20990 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent starting tablet: real 0.035s	user 0.016s	sys 0.011s
12:38:18.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:18.112792 20946 tablet.cc:2369] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
12:38:18.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:18.230890 20522 tablet.cc:2369] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
12:38:18.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.254333 20996 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:18.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.254911 20996 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.257555 20996 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:18.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.274176 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:18.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.274336 20863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:18.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.275147 20433 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 0.
12:38:18.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.275252 20863 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 0.
12:38:18.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.276266 20549 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 
12:38:18.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.276821 20996 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Leader pre-election won for term 1
12:38:18.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.277866 20996 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
12:38:18.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.278093 20996 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 0 FOLLOWER]: Advancing to term 1
12:38:18.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.282270 20996 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:18.283742 20714 tablet.cc:2369] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Can't schedule compaction. Clean time has not been advanced past its initial value.
12:38:18.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.283806 20996 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 1 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:18.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.284024 20863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:18.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.284228 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:18.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.284360 20863 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 0 FOLLOWER]: Advancing to term 1
12:38:18.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.284518 20433 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 0 FOLLOWER]: Advancing to term 1
12:38:18.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.288784 20863 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 1.
12:38:18.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.288772 20433 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 1.
12:38:18.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.289567 20546 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: 
12:38:18.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.290143 20996 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 FOLLOWER]: Leader election won for term 1
12:38:18.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.291262 20996 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 LEADER]: Becoming Leader. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Running, Role: LEADER
12:38:18.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.292114 20996 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:18.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.300647 20165 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194). New cstate: current_term: 1 leader_uuid: "ee5818823eb4422d94f6c2c5957eb31d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:18.508 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:894) Opened table 38787831ef4042aa89489d62d177324b
12:38:18.533 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:18.541 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299992, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:18.543 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=null, attempt=1, sequence_id=-1, TimeoutTracker(timeout=30000, elapsed=25), Traces: [0ms] refreshing cache from master, [0ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=null) after lookup
12:38:18.556 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:18.558 [DEBUG - kudu-nio-1] (Connection.java:757) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] Successfully connected to peer
12:38:18.622 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0xf3765b58, L:/127.0.0.1:53882 - R:/127.19.102.194:44869] using TOKEN/null
12:38:18.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.636757 20433 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Refusing update from remote peer ee5818823eb4422d94f6c2c5957eb31d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
12:38:18.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.636775 20863 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 1 FOLLOWER]: Refusing update from remote peer ee5818823eb4422d94f6c2c5957eb31d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
12:38:18.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.638259 21000 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
12:38:18.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.639159 20996 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
12:38:18.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.662886 21005 mvcc.cc:205] Tried to move back new op lower bound from 7317306976801861632 to 7317306975411208192. Current Snapshot: MvccSnapshot[applied={T|T < 7317306976801861632}]
12:38:18.803 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 3 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:18.806 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.194:44869
12:38:18.808 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.809027 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 20524
12:38:18.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.821971 20702 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:18.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:18.931488 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 20524
12:38:18.958 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:18.960 [DEBUG - kudu-nio-1] (Connection.java:239) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] handling channelInactive
12:38:18.961 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state READY due to: connection closed
12:38:18.962 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: connection closed
12:38:18.984 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:18.985 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:18.986 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=26), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [24ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1375696939(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@d40f5a9 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@53aadb18 -> wakeup thread Time-limited test)) after lookup
12:38:18.987 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:18.989 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:18.989 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:18.990 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.004 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:19.004 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:19.005 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=46), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [24ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [27ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [30ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [31ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1375696939(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@d40f5a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@53aadb18 -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334))) after lookup
12:38:19.006 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:19.008 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.008 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.008 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.045 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:19.045 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:19.046 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=87), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [24ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [27ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [30ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [31ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1375696939(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@d40f5a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@53aadb18 -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071))) after lookup
12:38:19.047 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:19.049 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.050 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.050 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.144 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:19.145 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:19.146 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=186), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [24ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [27ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [30ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [31ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [88ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [91ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [91ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [181ms] refreshing cache from master, [181ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [184ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1375696939(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@d40f5a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071) -> (continuation of Deferred@630752279 after retry RPC@178818425), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@53aadb18 -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071) -> (continuation of Deferred@630752279 after retry RPC@178818425))) after lookup
12:38:19.147 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:19.148 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.148 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.148 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.345 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:19.345 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:19.346 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=387), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [24ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [27ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [30ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [31ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [88ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [91ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [91ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [181ms] refreshing cache from master, [181ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [184ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [187ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [189ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [189ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [381ms] refreshing cache from master, [381ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [385ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1375696939(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@d40f5a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071) -> (continuation of Deferred@630752279 after retry RPC@178818425) -> (continuation of Deferred@237138231 after retry RPC@2119587474), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@53aadb18 -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071) -> (continuation of Deferred@630752279 after retry RPC@178818425) -> (continuation of Deferred@237138231 after retry RPC@2119587474))) after lookup
12:38:19.347 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:19.349 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.349 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:19.349 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.236105 21036 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.236353 21036 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:20.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.237821 21037 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:20.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.238112 21037 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:20.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.238615 21036 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:20.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.240363 21037 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:20.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:20.240384 20758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:20.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:20.246742 20338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:20.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.256876 20863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:20.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.257234 20863 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78eb3b7a2de2414faa706d782f134af4 in term 1.
12:38:20.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:20.257869 20338 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:20.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:20.258256 20758 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:20.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.258327 20341 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:20.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.258775 21037 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Leader pre-election won for term 2
12:38:20.259 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.258940 21037 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:20.259 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.259181 21037 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 1 FOLLOWER]: Advancing to term 2
12:38:20.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.259640 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:20.262 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.262048 21037 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:20.262 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.262629 20433 raft_consensus.cc:2379] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a96bc5edb0f947ffb1ca060f8a31d9ff in current term 2: Already voted for candidate 78eb3b7a2de2414faa706d782f134af4 in this term.
12:38:20.263 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.262930 21037 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 2 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:20.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.263895 20863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:20.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.263958 20758 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d
12:38:20.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.264237 20863 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 1 FOLLOWER]: Advancing to term 2
12:38:20.265 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:20.265209 20338 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:20.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.266914 20863 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78eb3b7a2de2414faa706d782f134af4 in term 2.
12:38:20.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.267119 21036 raft_consensus.cc:2726] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
12:38:20.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.267408 20341 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:20.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.267720 21037 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Leader election won for term 2
12:38:20.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.268258 21037 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 LEADER]: Becoming Leader. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Running, Role: LEADER
12:38:20.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.268774 21037 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:20.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.272197 20165 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 reported cstate change: term changed from 1 to 2, leader changed from ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194) to 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193). New cstate: current_term: 2 leader_uuid: "78eb3b7a2de2414faa706d782f134af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: HEALTHY } } }
12:38:20.605 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:20.605 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:20.606 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=1646), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [24ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [27ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [30ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [31ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [88ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [91ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [91ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [181ms] refreshing cache from master, [181ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [184ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [187ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [189ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [189ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [381ms] refreshing cache from master, [381ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [385ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [388ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [390ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [390ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [1641ms] refreshing cache from master, [1641ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1645ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1375696939(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@d40f5a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071) -> (continuation of Deferred@630752279 after retry RPC@178818425) -> (continuation of Deferred@237138231 after retry RPC@2119587474) -> (continuation of Deferred@1106932366 after retry RPC@223629029), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@53aadb18 -> wakeup thread Time-limited test -> (continuation of Deferred@1845341578 after retry RPC@1745537334) -> (continuation of Deferred@1338716885 after retry RPC@1682533071) -> (continuation of Deferred@630752279 after retry RPC@178818425) -> (continuation of Deferred@237138231 after retry RPC@2119587474) -> (continuation of Deferred@1106932366 after retry RPC@223629029))) after lookup
12:38:20.607 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:20.608 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] Successfully connected to peer
12:38:20.629 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0xc2b8250c, L:/127.0.0.1:46866 - R:/127.19.102.193:34365] using TOKEN/null
12:38:20.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.635901 20863 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Refusing update from remote peer 78eb3b7a2de2414faa706d782f134af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
12:38:20.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:20.637255 20338 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 -> Peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Couldn't send request to peer ee5818823eb4422d94f6c2c5957eb31d. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:20.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.637521 21051 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
12:38:20.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.643900 21074 mvcc.cc:205] Tried to move back new op lower bound from 7317306984994500608 to 7317306983504977920. Current Snapshot: MvccSnapshot[applied={T|T < 7317306984994500608}]
12:38:20.673 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.194:44869
12:38:20.675 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:20.676 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:20.676642 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:20.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:21.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.054548 21082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:21.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.054885 21082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:21.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.055016 21082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:21.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.063717 21082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.063972 21082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.194
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.073753 21082 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:21.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:21.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.075547 21082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:21.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.077689 21082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:21.092 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.091938 21091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:21.092 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.091848 21095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:21.092 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:21.091920 21092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:21.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.093391 21082 server_base.cc:1034] running on GCE node
12:38:21.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.094245 21082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:21.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.095567 21082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:21.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.096835 21082 hybrid_clock.cc:648] HybridClock initialized: now 1786451901096756 us; error 48 us; skew 500 ppm
12:38:21.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.097203 21082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:21.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.100108 21082 webserver.cc:460] Webserver started at http://127.19.102.194:44599/ using document root <none> and password file <none>
12:38:21.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.100914 21082 fs_manager.cc:359] Metadata directory not provided
12:38:21.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.101083 21082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:21.107 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.107074 21082 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.000s	sys 0.004s
12:38:21.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.110565 21103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:21.112 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.112115 21082 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.112321 21082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.113461 21082 fs_report.cc:383] FS layout report
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:21.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/data
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:21.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:21.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.137985 21082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:21.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.139081 21082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:21.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.139400 21082 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:21.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.140522 21082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:21.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.143232 21112 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:21.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.152835 21082 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:21.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.153021 21082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s	user 0.001s	sys 0.000s
12:38:21.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.153156 21082 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:21.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.156508 21082 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:21.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.156687 21082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s	user 0.001s	sys 0.000s
12:38:21.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.157275 21112 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap starting.
12:38:21.187 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.186510 21112 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Log is configured to *not* fsync() on all Append() calls
12:38:21.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.222007 21112 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:21.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.223452 21112 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap complete.
12:38:21.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.225144 21112 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent bootstrapping tablet: real 0.068s	user 0.040s	sys 0.014s
12:38:21.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.245362 21112 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:21.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.247228 21112 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Initialized, Role: FOLLOWER
12:38:21.248 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.248231 21112 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:21.250 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.250730 21112 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent starting tablet: real 0.025s	user 0.019s	sys 0.003s
12:38:21.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.284915 21082 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.194:44869
12:38:21.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.285092 21292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.194:44869 every 8 connection(s)
12:38:21.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.287180 21082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:21.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.299127 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 21082
12:38:21.299 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:21.320 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 6 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:21.322 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.193:34365
12:38:21.323 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

12:38:21.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.323438 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 20294
12:38:21.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.341418 21295 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:21.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.342314 21295 heartbeater.cc:464] Registering TS with master...
12:38:21.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.343007 20501 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:21.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.344044 21295 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:21.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.344738 21293 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:21.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.345136 21293 heartbeater.cc:464] Registering TS with master...
12:38:21.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.346400 21293 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:21.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.347761 19931 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:21.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.348368 20165 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:21.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.350085 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.194:35755
12:38:21.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.356885 21296 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:21.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.357187 21296 heartbeater.cc:464] Registering TS with master...
12:38:21.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.357966 21296 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:21.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.360303 20056 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:21.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:21.454319 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 20294
12:38:21.477 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] handling channelInactive
12:38:21.478 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state READY due to: connection closed
12:38:21.478 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:21.479 [DEBUG - Time-limited test] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:21.482 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.482 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.482 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.504 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:21.505 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:21.505 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=27), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test)) after lookup
12:38:21.506 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:21.507 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.507 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.507 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.524 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:21.525 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:21.525 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=47), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [28ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [29ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [30ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479))) after lookup
12:38:21.526 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:21.528 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.528 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.528 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.545 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:21.545 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:21.546 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=67), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [28ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [29ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [30ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [48ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [50ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [50ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108))) after lookup
12:38:21.547 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:21.548 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.548 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.548 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.566 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:21.566 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:21.567 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=88), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [28ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [29ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [30ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [48ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [50ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [50ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [69ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [70ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [70ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105))) after lookup
12:38:21.568 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:21.570 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.570 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:21.570 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.065 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:22.065 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:22.066 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=587), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [28ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [29ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [30ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [48ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [50ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [50ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [69ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [70ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [70ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [90ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [92ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [92ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [582ms] refreshing cache from master, [582ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [586ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105) -> (continuation of Deferred@1114666078 after retry RPC@831259670), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105) -> (continuation of Deferred@1114666078 after retry RPC@831259670))) after lookup
12:38:22.067 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:22.069 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.069 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.069 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.355028 21293 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:22.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.718358 21310 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 78eb3b7a2de2414faa706d782f134af4)
12:38:22.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.718780 21310 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:22.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.719581 21310 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:22.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:22.723515 20758 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:22.725 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:22.726 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:22.727 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=1248), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [28ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [29ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [30ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [48ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [50ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [50ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [69ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [70ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [70ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [90ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [92ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [92ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [582ms] refreshing cache from master, [582ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [586ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [589ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [591ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [591ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1242ms] refreshing cache from master, [1242ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1246ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105) -> (continuation of Deferred@1114666078 after retry RPC@831259670) -> (continuation of Deferred@397792972 after retry RPC@2000432687), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105) -> (continuation of Deferred@1114666078 after retry RPC@831259670) -> (continuation of Deferred@397792972 after retry RPC@2000432687))) after lookup
12:38:22.728 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:22.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.729126 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d" is_pre_election: true
12:38:22.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.729817 21214 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 1.
12:38:22.730 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.730 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.730 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:22.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.730976 20758 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4
12:38:22.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.731348 21310 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Leader pre-election won for term 3
12:38:22.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.731504 21310 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Starting leader election (detected failure of leader 78eb3b7a2de2414faa706d782f134af4)
12:38:22.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.731601 21310 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 2 FOLLOWER]: Advancing to term 3
12:38:22.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.733779 21310 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:22.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.734488 21310 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 3 election: Requested vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:22.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.735208 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:22.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.735438 21214 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 1 FOLLOWER]: Advancing to term 3
12:38:22.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:22.739463 20758 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:22.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.740942 21214 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 3.
12:38:22.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.741532 20758 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4
12:38:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.741896 21310 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 3 FOLLOWER]: Leader election won for term 3
12:38:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.742442 21310 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 3 LEADER]: Becoming Leader. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Running, Role: LEADER
12:38:22.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.742982 21310 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:22.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:22.746554 20165 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff reported cstate change: term changed from 2 to 3, leader changed from 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193) to a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195). New cstate: current_term: 3 leader_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:23.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:23.162057 21214 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Refusing update from remote peer a96bc5edb0f947ffb1ca060f8a31d9ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
12:38:23.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:23.163439 21310 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.000s
12:38:23.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:23.182646 20758 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff -> Peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Couldn't send request to peer 78eb3b7a2de2414faa706d782f134af4. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:25.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:25.428084 20758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111) [suppressed 6 similar messages]
12:38:25.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:25.824028 20758 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff -> Peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Couldn't send request to peer 78eb3b7a2de2414faa706d782f134af4. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
12:38:26.784 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:26.785 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:26.787 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=14, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=5307), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [4ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [4ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [28ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [29ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [30ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [48ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [50ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [50ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [62ms] refreshing cache from master, [62ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [69ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [70ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [70ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [82ms] refreshing cache from master, [82ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [90ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [92ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [92ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [582ms] refreshing cache from master, [582ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [586ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [589ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [591ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [591ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1242ms] refreshing cache from master, [1242ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1246ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [1250ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [1252ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1253ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [5301ms] refreshing cache from master, [5301ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [5305ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@2053727908(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@6fe787fa -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105) -> (continuation of Deferred@1114666078 after retry RPC@831259670) -> (continuation of Deferred@397792972 after retry RPC@2000432687) -> (continuation of Deferred@1214868001 after retry RPC@1580361813), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37932c9f -> wakeup thread Time-limited test -> (continuation of Deferred@718672189 after retry RPC@1659338479) -> (continuation of Deferred@521394015 after retry RPC@1180353108) -> (continuation of Deferred@194873120 after retry RPC@1961947105) -> (continuation of Deferred@1114666078 after retry RPC@831259670) -> (continuation of Deferred@397792972 after retry RPC@2000432687) -> (continuation of Deferred@1214868001 after retry RPC@1580361813))) after lookup
12:38:26.788 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:26.789 [DEBUG - kudu-nio-1] (Connection.java:757) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] Successfully connected to peer
12:38:26.818 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x574bd975, L:/127.0.0.1:44954 - R:/127.19.102.195:36907] using TOKEN/null
12:38:26.856 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.193:34365
12:38:26.857 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:26.862408 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:26.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:34365
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41923
12:38:26.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:26.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:26.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:26.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:26.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:27.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.237728 21376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:27.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.238035 21376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:27.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.238118 21376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:27.245 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.245692 21376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:27.246 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.245887 21376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.193
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.255851 21376 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:34365
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41923
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:27.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:27.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.257246 21376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:27.259 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.259078 21376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:27.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.272833 21385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:27.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.272935 21391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:27.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.273154 21376 server_base.cc:1034] running on GCE node
12:38:27.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:27.272907 21386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:27.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.274544 21376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:27.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.275772 21376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:27.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.277026 21376 hybrid_clock.cc:648] HybridClock initialized: now 1786451907276954 us; error 32 us; skew 500 ppm
12:38:27.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.277379 21376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:27.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.286245 21376 webserver.cc:460] Webserver started at http://127.19.102.193:41923/ using document root <none> and password file <none>
12:38:27.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.287087 21376 fs_manager.cc:359] Metadata directory not provided
12:38:27.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.287258 21376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:27.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.293493 21376 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.003s
12:38:27.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.297056 21400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:27.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.299321 21376 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.299510 21376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:15 on dist-test-slave-t7g5"
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.300609 21376 fs_report.cc:383] FS layout report
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/data
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:27.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:27.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:27.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.326467 21376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:27.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.327795 21376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:27.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.328142 21376 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:27.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.329737 21376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:27.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.332145 21413 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:27.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.338555 21376 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:27.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.338717 21376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s	user 0.000s	sys 0.001s
12:38:27.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.338837 21376 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:27.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.342698 21376 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:27.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.342972 21376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s	user 0.003s	sys 0.001s
12:38:27.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.343544 21413 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap starting.
12:38:27.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.416780 21413 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Log is configured to *not* fsync() on all Append() calls
12:38:27.448 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.447999 21413 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=6 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:27.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.449633 21413 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap complete.
12:38:27.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.451951 21413 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent bootstrapping tablet: real 0.109s	user 0.042s	sys 0.061s
12:38:27.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.464944 21413 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:27.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.466652 21413 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Initialized, Role: FOLLOWER
12:38:27.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.467908 21413 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:27.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.470989 21413 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent starting tablet: real 0.019s	user 0.014s	sys 0.006s
12:38:27.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.471560 21376 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.193:34365
12:38:27.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.471838 21598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.193:34365 every 8 connection(s)
12:38:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.472955 21376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:27.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.479519 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 21376
12:38:27.480 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:27.503 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 9 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:27.504 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.195:36907
12:38:27.505 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

12:38:27.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.505911 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 20729
12:38:27.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.519043 21600 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:27.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.519578 21600 heartbeater.cc:464] Registering TS with master...
12:38:27.521 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.521416 21600 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:27.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.522070 21599 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:27.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.522370 21599 heartbeater.cc:464] Registering TS with master...
12:38:27.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.523293 21599 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.525897 20165 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.526309 19927 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.527819 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.193:41169
12:38:27.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.535773 21601 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:27.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.536036 21601 heartbeater.cc:464] Registering TS with master...
12:38:27.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.536844 21601 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:27.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.538552 20055 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:27.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.548188 20933 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:27.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:27.656621 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 20729
12:38:27.679 [DEBUG - kudu-nio-1] (Connection.java:239) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] handling channelInactive
12:38:27.679 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state READY due to: connection closed
12:38:27.679 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:27.681 [DEBUG - Time-limited test] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:27.683 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.683 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.684 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.704 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:27.705 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:27.705 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=25), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test)) after lookup
12:38:27.706 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:27.707 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.707 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.707 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.724 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:27.725 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:27.725 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=45), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [25ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [27ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [27ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392))) after lookup
12:38:27.726 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:27.728 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.728 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.728 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.743 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:27.743 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:27.744 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=63), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [25ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [27ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [27ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [46ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [48ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [48ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [62ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328))) after lookup
12:38:27.745 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:27.746 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.747 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.747 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.823 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:27.824 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:27.824 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=144), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [25ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [27ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [27ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [46ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [48ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [48ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [62ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [65ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [67ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [67ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [140ms] refreshing cache from master, [140ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [143ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923))) after lookup
12:38:27.825 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:27.828 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.828 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:27.828 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.164 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:28.164 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:28.165 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=485), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [25ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [27ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [27ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [46ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [48ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [48ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [62ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [65ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [67ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [67ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [140ms] refreshing cache from master, [140ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [143ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [145ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [148ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [148ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [479ms] refreshing cache from master, [479ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [483ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923) -> (continuation of Deferred@1286182986 after retry RPC@1780242454), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923) -> (continuation of Deferred@1286182986 after retry RPC@1780242454))) after lookup
12:38:28.167 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:28.168 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.168 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.168 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.304 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:28.304 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:28.305 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=624), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [25ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [27ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [27ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [46ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [48ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [48ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [62ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [65ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [67ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [67ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [140ms] refreshing cache from master, [140ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [143ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [145ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [148ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [148ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [479ms] refreshing cache from master, [479ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [483ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [486ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [488ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [488ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [620ms] refreshing cache from master, [620ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [623ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923) -> (continuation of Deferred@1286182986 after retry RPC@1780242454) -> (continuation of Deferred@287625289 after retry RPC@1489921681), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923) -> (continuation of Deferred@1286182986 after retry RPC@1780242454) -> (continuation of Deferred@287625289 after retry RPC@1489921681))) after lookup
12:38:28.306 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:28.308 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.308 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.308 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:28.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.532202 21599 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:28.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.646487 21622 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:28.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.646880 21622 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:28.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.648514 21622 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:28.650 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:28.650363 21407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:28.652 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:28.652025 21407 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:28.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.655735 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d" is_pre_election: true
12:38:28.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.656729 21402 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d
12:38:28.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.657155 21622 raft_consensus.cc:2726] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
12:38:28.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.785840 21630 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a96bc5edb0f947ffb1ca060f8a31d9ff)
12:38:28.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.786062 21630 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:28.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.787736 21630 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:28.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:28.789362 21107 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:28.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:28.791189 21107 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:28.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.797385 21516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:28.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.797955 21516 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 2.
12:38:28.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.798988 21105 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:28.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.799413 21630 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Leader pre-election won for term 4
12:38:28.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.799551 21630 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Starting leader election (detected failure of leader a96bc5edb0f947ffb1ca060f8a31d9ff)
12:38:28.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.799640 21630 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 3 FOLLOWER]: Advancing to term 4
12:38:28.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.801829 21630 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:28.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.802515 21630 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 4 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:28.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.803407 21516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:28.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.803793 21516 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 2 FOLLOWER]: Advancing to term 4
12:38:28.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:28.804939 21107 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:28.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.808358 21516 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 4.
12:38:28.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.808851 21105 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:28.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.809180 21630 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Leader election won for term 4
12:38:28.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.809638 21630 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 LEADER]: Becoming Leader. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Running, Role: LEADER
12:38:28.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.810159 21630 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:28.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:28.814864 20165 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d reported cstate change: term changed from 3 to 4, leader changed from a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195) to ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194). New cstate: current_term: 4 leader_uuid: "ee5818823eb4422d94f6c2c5957eb31d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:29.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:29.339788 21107 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d -> Peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Couldn't send request to peer a96bc5edb0f947ffb1ca060f8a31d9ff. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:29.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:29.372818 21516 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Refusing update from remote peer ee5818823eb4422d94f6c2c5957eb31d: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 4 index: 13. (index mismatch)
12:38:29.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:29.373832 21659 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.000s
12:38:31.663 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:31.663 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:31.664 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=14, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=3983), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [4ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [25ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [27ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [27ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [44ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [46ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [48ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [48ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [62ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [65ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [67ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [67ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [140ms] refreshing cache from master, [140ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [143ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [145ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [148ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [148ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [479ms] refreshing cache from master, [479ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [483ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [486ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [488ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [488ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [620ms] refreshing cache from master, [620ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [623ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [626ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [628ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [628ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [3980ms] refreshing cache from master, [3980ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [3982ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1752661897(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@166ff297 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923) -> (continuation of Deferred@1286182986 after retry RPC@1780242454) -> (continuation of Deferred@287625289 after retry RPC@1489921681) -> (continuation of Deferred@1177184659 after retry RPC@1836803369), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1cac4f89 -> wakeup thread Time-limited test -> (continuation of Deferred@327344967 after retry RPC@1159898392) -> (continuation of Deferred@1225381852 after retry RPC@424915328) -> (continuation of Deferred@168869084 after retry RPC@470241923) -> (continuation of Deferred@1286182986 after retry RPC@1780242454) -> (continuation of Deferred@287625289 after retry RPC@1489921681) -> (continuation of Deferred@1177184659 after retry RPC@1836803369))) after lookup
12:38:31.665 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:31.666 [DEBUG - kudu-nio-1] (Connection.java:757) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] Successfully connected to peer
12:38:31.690 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x4c60a29e, L:/127.0.0.1:34372 - R:/127.19.102.194:44869] using TOKEN/null
12:38:31.734 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.195:36907
12:38:31.734 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

12:38:31.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:31.735509 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:31.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:31.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:36907
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43649
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:31.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:31.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:31.914922 21107 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d -> Peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Couldn't send request to peer a96bc5edb0f947ffb1ca060f8a31d9ff. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
12:38:32.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.133526 21710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:32.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.133926 21710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:32.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.134049 21710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:32.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.143208 21710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:32.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.143464 21710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.195
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.153091 21710 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:36907
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43649
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:32.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:32.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.154937 21710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:32.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.156869 21710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:32.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.171368 21717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:32.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.171301 21722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:32.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.171347 21718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:32.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.172590 21710 server_base.cc:1034] running on GCE node
12:38:32.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.173251 21710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:32.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.174357 21710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:32.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.175765 21710 hybrid_clock.cc:648] HybridClock initialized: now 1786451912175618 us; error 48 us; skew 500 ppm
12:38:32.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.176275 21710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:32.179 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.179456 21710 webserver.cc:460] Webserver started at http://127.19.102.195:43649/ using document root <none> and password file <none>
12:38:32.180 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.180176 21710 fs_manager.cc:359] Metadata directory not provided
12:38:32.180 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.180303 21710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:32.187 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.187229 21710 fs_manager.cc:709] Time spent opening directory manager: real 0.005s	user 0.000s	sys 0.004s
12:38:32.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.192799 21732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.194796 21710 fs_manager.cc:725] Time spent opening block manager: real 0.006s	user 0.004s	sys 0.002s
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.195032 21710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.196159 21710 fs_report.cc:383] FS layout report
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/data
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:32.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.235327 21710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:32.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.236331 21710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:32.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.237100 21710 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:32.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.238775 21710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:32.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.241369 21746 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:32.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.249142 21710 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:32.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.249313 21710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s	user 0.001s	sys 0.000s
12:38:32.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.249461 21710 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:32.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.252882 21710 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:32.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.252982 21710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s	user 0.001s	sys 0.000s
12:38:32.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.253376 21746 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap starting.
12:38:32.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.295338 21746 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Log is configured to *not* fsync() on all Append() calls
12:38:32.337 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.337235 21746 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:32.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.340001 21746 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap complete.
12:38:32.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.342252 21746 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent bootstrapping tablet: real 0.089s	user 0.052s	sys 0.032s
12:38:32.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.348912 21710 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.195:36907
12:38:32.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.349117 21910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.195:36907 every 8 connection(s)
12:38:32.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.350777 21710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:32.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.355621 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 21710
12:38:32.356 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:32.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.353935 21746 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:32.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.363960 21746 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Initialized, Role: FOLLOWER
12:38:32.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.365062 21746 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 3.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:32.372 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 12 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:32.392 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.194:44869
12:38:32.392 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:32.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.392977 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 21082
12:38:32.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.414002 21912 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:32.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.414462 21912 heartbeater.cc:464] Registering TS with master...
12:38:32.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.415570 21746 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent starting tablet: real 0.073s	user 0.023s	sys 0.011s
12:38:32.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.416208 21912 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:32.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.416708 21911 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:32.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.416925 21911 heartbeater.cc:464] Registering TS with master...
12:38:32.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.417874 21911 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:32.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.420429 19927 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:32.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.420712 20165 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:32.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.423084 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.195:60079
12:38:32.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.426693 21913 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:32.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.426977 21913 heartbeater.cc:464] Registering TS with master...
12:38:32.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.428915 21913 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:32.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.430436 21911 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:32.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.431435 20056 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:32.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.445730 21283 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:32.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.553797 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 21082
12:38:32.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:32.558636 21839 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 3 FOLLOWER]: Advancing to term 4
12:38:32.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:32.586321 21739 connection.cc:535] server connection from 127.19.102.194:41275 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
12:38:32.587 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:32.587 [DEBUG - kudu-nio-1] (Connection.java:239) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] handling channelInactive
12:38:32.587 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state READY due to: connection closed
12:38:32.588 [DEBUG - Time-limited test] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:32.590 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.590 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.590 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.604 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:32.604 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:32.605 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=17), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [15ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1595723720(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@7487a91b -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1151ac6d -> wakeup thread Time-limited test)) after lookup
12:38:32.605 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:32.607 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.607 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.607 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.624 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:32.624 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:32.624 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=36), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [15ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [17ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [19ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [19ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1595723720(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@7487a91b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1151ac6d -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651))) after lookup
12:38:32.625 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:32.627 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.627 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.627 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.643 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:32.644 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:32.644 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=56), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [15ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [17ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [19ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [19ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [39ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [55ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1595723720(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@7487a91b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1151ac6d -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961))) after lookup
12:38:32.645 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:32.646 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.646 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.646 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.723 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:32.723 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:32.724 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=136), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [15ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [17ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [19ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [19ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [39ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [55ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [57ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [58ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [58ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [131ms] refreshing cache from master, [131ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1595723720(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@7487a91b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961) -> (continuation of Deferred@1320911120 after retry RPC@1025160750), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1151ac6d -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961) -> (continuation of Deferred@1320911120 after retry RPC@1025160750))) after lookup
12:38:32.725 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:32.727 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.727 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.727 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.763 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:32.763 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:32.763 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=175), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [15ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [17ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [19ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [19ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [39ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [55ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [57ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [58ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [58ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [131ms] refreshing cache from master, [131ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [137ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [139ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [139ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [171ms] refreshing cache from master, [171ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [174ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1595723720(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@7487a91b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961) -> (continuation of Deferred@1320911120 after retry RPC@1025160750) -> (continuation of Deferred@1900279046 after retry RPC@427857606), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1151ac6d -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961) -> (continuation of Deferred@1320911120 after retry RPC@1025160750) -> (continuation of Deferred@1900279046 after retry RPC@427857606))) after lookup
12:38:32.764 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:32.765 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.765 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:32.765 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:33.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:33.714102 21945 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:33.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:33.714427 21945 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:33.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:33.715140 21945 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:33.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:33.716796 21402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:33.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:33.719215 21402 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:33.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:33.723383 21839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:33.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:33.724406 21407 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d
12:38:33.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:33.724721 21945 raft_consensus.cc:2726] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
12:38:34.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.069408 21952 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:34.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.069737 21952 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:34.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.071225 21952 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:34.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.072502 21735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:34.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.074841 21735 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:34.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.078459 21516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:34.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.078696 21516 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 4.
12:38:34.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.079526 21735 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:34.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.080138 21952 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 4 FOLLOWER]: Leader pre-election won for term 5
12:38:34.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.080437 21952 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 4 FOLLOWER]: Starting leader election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:34.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.080650 21952 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 4 FOLLOWER]: Advancing to term 5
12:38:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.083206 21952 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:34.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.084000 21952 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 5 election: Requested vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.085392 21516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.085695 21516 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 4 FOLLOWER]: Advancing to term 5
12:38:34.086 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.086216 21735 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:34.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.087826 21516 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 5.
12:38:34.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.088433 21735 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:34.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.088977 21952 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 FOLLOWER]: Leader election won for term 5
12:38:34.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.089515 21952 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 LEADER]: Becoming Leader. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Running, Role: LEADER
12:38:34.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.090081 21952 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:34.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.093734 20165 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff reported cstate change: term changed from 4 to 5, leader changed from ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194) to a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195). New cstate: current_term: 5 leader_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:34.463 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:34.463 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:34.463 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=1875), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [15ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [17ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [19ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [19ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [39ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [40ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [55ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [57ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [58ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [58ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [131ms] refreshing cache from master, [131ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [137ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [139ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [139ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [171ms] refreshing cache from master, [171ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [174ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [176ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [177ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [177ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [1871ms] refreshing cache from master, [1871ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1874ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1595723720(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@7487a91b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961) -> (continuation of Deferred@1320911120 after retry RPC@1025160750) -> (continuation of Deferred@1900279046 after retry RPC@427857606) -> (continuation of Deferred@1627965713 after retry RPC@1336730695), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1151ac6d -> wakeup thread Time-limited test -> (continuation of Deferred@835863271 after retry RPC@508268651) -> (continuation of Deferred@1454982789 after retry RPC@347294961) -> (continuation of Deferred@1320911120 after retry RPC@1025160750) -> (continuation of Deferred@1900279046 after retry RPC@427857606) -> (continuation of Deferred@1627965713 after retry RPC@1336730695))) after lookup
12:38:34.465 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:34.467 [DEBUG - kudu-nio-0] (Connection.java:757) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] Successfully connected to peer
12:38:34.488 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x3165842c, L:/127.0.0.1:47778 - R:/127.19.102.195:36907] using TOKEN/null
12:38:34.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.493619 21516 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Refusing update from remote peer a96bc5edb0f947ffb1ca060f8a31d9ff: Log matching property violated. Preceding OpId in replica: term: 4 index: 16. Preceding OpId from leader: term: 5 index: 18. (index mismatch)
12:38:34.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.494541 21735 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff -> Peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Couldn't send request to peer ee5818823eb4422d94f6c2c5957eb31d. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:34.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.494586 21958 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
12:38:34.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.501264 21981 mvcc.cc:205] Tried to move back new op lower bound from 7317307041758281728 to 7317307040116838400. Current Snapshot: MvccSnapshot[applied={T|T < 7317307041758281728}]
12:38:34.523 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.194:44869
12:38:34.524 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:34.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.524731 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:34.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:34.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:34.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:34.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.899470 21991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:34.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.900125 21991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:34.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.900326 21991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:34.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.909349 21991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:34.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.909597 21991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.194
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.919827 21991 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:34.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:34.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.921612 21991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:34.923 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.923594 21991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:34.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.938688 22001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:34.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.938784 21999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:34.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:34.938709 22005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:34.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.940579 21991 server_base.cc:1034] running on GCE node
12:38:34.941 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.941545 21991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:34.942 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.942783 21991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:34.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.944161 21991 hybrid_clock.cc:648] HybridClock initialized: now 1786451914944036 us; error 77 us; skew 500 ppm
12:38:34.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.944633 21991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:34.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.948035 21991 webserver.cc:460] Webserver started at http://127.19.102.194:44599/ using document root <none> and password file <none>
12:38:34.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.948817 21991 fs_manager.cc:359] Metadata directory not provided
12:38:34.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.948946 21991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:34.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.955111 21991 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.000s	sys 0.004s
12:38:34.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.958443 22011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:34.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.959939 21991 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.960124 21991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.961267 21991 fs_report.cc:383] FS layout report
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/data
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:34.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:34.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.987491 21991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:34.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.988524 21991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:34.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.988902 21991 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:34.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.990242 21991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:34.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.992705 22023 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:34.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.998950 21991 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:34.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.999117 21991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
12:38:34.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:34.999269 21991 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:35.003 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.003031 21991 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:35.003 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.003139 21991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s	user 0.003s	sys 0.000s
12:38:35.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.003958 22023 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap starting.
12:38:35.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.033372 22023 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Log is configured to *not* fsync() on all Append() calls
12:38:35.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.067920 22023 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:35.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.069083 22023 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap complete.
12:38:35.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.070508 22023 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent bootstrapping tablet: real 0.067s	user 0.048s	sys 0.016s
12:38:35.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.079550 21991 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.194:44869
12:38:35.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.079656 22189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.194:44869 every 8 connection(s)
12:38:35.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.079679 22023 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:35.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.081225 21991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:35.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.081235 22023 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Initialized, Role: FOLLOWER
12:38:35.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.082386 22023 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 4.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:35.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.087299 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 21991
12:38:35.087 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:35.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.103485 22023 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent starting tablet: real 0.033s	user 0.016s	sys 0.005s
12:38:35.112 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 15 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:35.114 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.195:36907
12:38:35.115 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

12:38:35.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.115437 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 21710
12:38:35.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.126823 22191 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:35.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.127269 22191 heartbeater.cc:464] Registering TS with master...
12:38:35.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.128152 22191 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:35.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.129912 22190 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:35.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.130074 22190 heartbeater.cc:464] Registering TS with master...
12:38:35.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.130551 22190 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:35.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.131577 19927 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:35.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.132647 20165 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:35.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.134111 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.194:49065
12:38:35.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.135658 22193 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:35.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.135802 22193 heartbeater.cc:464] Registering TS with master...
12:38:35.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.136240 22193 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:35.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.138823 22190 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:35.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.138921 20056 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:35.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.148512 21906 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:35.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:35.250960 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 21710
12:38:35.289 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:35.289 [DEBUG - kudu-nio-0] (Connection.java:239) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] handling channelInactive
12:38:35.290 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state READY due to: connection closed
12:38:35.290 [DEBUG - Time-limited test] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:35.292 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.292 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.292 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.304 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:35.304 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:35.305 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=15), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test)) after lookup
12:38:35.306 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:35.308 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.308 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.308 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.323 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:35.323 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:35.324 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=34), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [15ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [29ms] refreshing cache from master, [29ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [33ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703))) after lookup
12:38:35.325 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:35.326 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.326 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.326 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.364 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:35.365 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:35.365 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=75), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [15ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [29ms] refreshing cache from master, [29ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [33ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [34ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [36ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [36ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [69ms] refreshing cache from master, [70ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [73ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323))) after lookup
12:38:35.366 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:35.368 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.368 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.368 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.485 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:35.486 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:35.487 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=196), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [15ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [29ms] refreshing cache from master, [29ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [33ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [34ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [36ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [36ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [69ms] refreshing cache from master, [70ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [73ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [76ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [78ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [78ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [170ms] refreshing cache from master, [170ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [173ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979))) after lookup
12:38:35.488 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:35.489 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.489 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.490 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.924 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:35.924 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:35.924 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=634), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [15ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [29ms] refreshing cache from master, [29ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [33ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [34ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [36ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [36ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [69ms] refreshing cache from master, [70ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [73ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [76ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [78ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [78ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [170ms] refreshing cache from master, [170ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [173ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [197ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [199ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [200ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [630ms] refreshing cache from master, [630ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [633ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979) -> (continuation of Deferred@1155407881 after retry RPC@1907020753), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979) -> (continuation of Deferred@1155407881 after retry RPC@1907020753))) after lookup
12:38:35.926 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:35.927 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.927 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:35.927 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:36.164 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:36.164 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:36.165 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=875), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [15ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [29ms] refreshing cache from master, [29ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [33ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [34ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [36ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [36ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [69ms] refreshing cache from master, [70ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [73ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [76ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [78ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [78ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [170ms] refreshing cache from master, [170ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [173ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [197ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [199ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [200ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [630ms] refreshing cache from master, [630ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [633ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [635ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [637ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [638ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [870ms] refreshing cache from master, [870ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [873ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979) -> (continuation of Deferred@1155407881 after retry RPC@1907020753) -> (continuation of Deferred@306772861 after retry RPC@903583422), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979) -> (continuation of Deferred@1155407881 after retry RPC@1907020753) -> (continuation of Deferred@306772861 after retry RPC@903583422))) after lookup
12:38:36.167 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:36.168 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:36.168 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:36.169 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:36.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.378582 22215 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:36.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.378986 22215 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:36.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.381297 22215 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:36.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:36.383685 22019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:36.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:36.386133 22019 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:36.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.390336 21516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:36.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.391271 22016 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:36.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.391778 22215 raft_consensus.cc:2726] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
12:38:36.437 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.436882 22221 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a96bc5edb0f947ffb1ca060f8a31d9ff)
12:38:36.437 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.437150 22221 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:36.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.437947 22221 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:36.441 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:36.440994 21407 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:36.448 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.448377 22119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d" is_pre_election: true
12:38:36.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.448853 22119 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78eb3b7a2de2414faa706d782f134af4 in term 4.
12:38:36.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.449972 21402 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:36.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.450367 22221 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Leader pre-election won for term 6
12:38:36.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.450484 22221 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a96bc5edb0f947ffb1ca060f8a31d9ff)
12:38:36.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.450546 22221 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 5 FOLLOWER]: Advancing to term 6
12:38:36.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.452564 22221 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:36.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.453181 22221 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 6 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:36.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.453819 22119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 6 candidate_status { last_received { term: 5 index: 20 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:36.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.454061 22119 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 4 FOLLOWER]: Advancing to term 6
12:38:36.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:36.454898 21407 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:36.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.460098 22119 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78eb3b7a2de2414faa706d782f134af4 in term 6.
12:38:36.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.460796 21402 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:36.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.461325 22221 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Leader election won for term 6
12:38:36.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.462186 22221 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 LEADER]: Becoming Leader. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Running, Role: LEADER
12:38:36.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.462769 22221 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 5.20, Last appended by leader: 20, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:36.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.466944 20165 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 reported cstate change: term changed from 5 to 6, leader changed from a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195) to 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193). New cstate: current_term: 6 leader_uuid: "78eb3b7a2de2414faa706d782f134af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: HEALTHY } } }
12:38:36.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.890388 22119 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Refusing update from remote peer 78eb3b7a2de2414faa706d782f134af4: Log matching property violated. Preceding OpId in replica: term: 4 index: 16. Preceding OpId from leader: term: 6 index: 21. (index mismatch)
12:38:36.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:36.891364 22221 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 16, Time since last communication: 0.000s
12:38:36.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:36.906481 21407 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 -> Peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Couldn't send request to peer a96bc5edb0f947ffb1ca060f8a31d9ff. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:37.103 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:37.104 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:37.105 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=14, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=1814), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [2ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [9ms] refreshing cache from master, [10ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [13ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [15ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [29ms] refreshing cache from master, [29ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [33ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [34ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [36ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [36ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [69ms] refreshing cache from master, [70ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [73ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [76ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [78ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [78ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [170ms] refreshing cache from master, [170ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [173ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [197ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [199ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [200ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [630ms] refreshing cache from master, [630ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [633ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [635ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [637ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [638ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [870ms] refreshing cache from master, [870ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [873ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [876ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [879ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [879ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [1810ms] refreshing cache from master, [1810ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1813ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@22381546(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@10bb8739 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979) -> (continuation of Deferred@1155407881 after retry RPC@1907020753) -> (continuation of Deferred@306772861 after retry RPC@903583422) -> (continuation of Deferred@152792693 after retry RPC@888066595), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1ea9c263 -> wakeup thread Time-limited test -> (continuation of Deferred@1804829540 after retry RPC@997846703) -> (continuation of Deferred@395554935 after retry RPC@1549382323) -> (continuation of Deferred@1493449328 after retry RPC@1554207979) -> (continuation of Deferred@1155407881 after retry RPC@1907020753) -> (continuation of Deferred@306772861 after retry RPC@903583422) -> (continuation of Deferred@152792693 after retry RPC@888066595))) after lookup
12:38:37.106 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:37.107 [DEBUG - kudu-nio-0] (Connection.java:757) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] Successfully connected to peer
12:38:37.128 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0xf8e2829b, L:/127.0.0.1:40776 - R:/127.19.102.193:34365] using TOKEN/null
12:38:37.156 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.195:36907
12:38:37.157 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.158732 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:36907
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43649
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:37.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:37.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:37.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:37.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.532688 22261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:37.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.533037 22261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:37.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.533160 22261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:37.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.540685 22261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:37.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.540932 22261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.195
12:38:37.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.550531 22261 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:37.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:36907
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43649
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:37.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:37.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.552260 22261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:37.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.554344 22261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:37.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.567854 22271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:37.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.567811 22274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:37.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:37.567885 22276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:37.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.568812 22261 server_base.cc:1034] running on GCE node
12:38:37.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.569686 22261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:37.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.570973 22261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:37.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.572383 22261 hybrid_clock.cc:648] HybridClock initialized: now 1786451917572237 us; error 59 us; skew 500 ppm
12:38:37.573 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.572949 22261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:37.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.580163 22261 webserver.cc:460] Webserver started at http://127.19.102.195:43649/ using document root <none> and password file <none>
12:38:37.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.581084 22261 fs_manager.cc:359] Metadata directory not provided
12:38:37.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.581214 22261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:37.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.588217 22261 fs_manager.cc:709] Time spent opening directory manager: real 0.005s	user 0.005s	sys 0.000s
12:38:37.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.592046 22283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:37.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.593804 22261 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.593998 22261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.595023 22261 fs_report.cc:383] FS layout report
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/data
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:37.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:37.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.619879 22261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:37.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.621006 22261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:37.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.621342 22261 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:37.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.622531 22261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:37.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.624867 22293 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:37.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.630873 22261 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:37.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.631048 22261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
12:38:37.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.631181 22261 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:37.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.635919 22261 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:37.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.636052 22261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s	user 0.003s	sys 0.000s
12:38:37.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.636979 22293 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap starting.
12:38:37.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.666082 22293 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Log is configured to *not* fsync() on all Append() calls
12:38:37.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.696455 22261 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.195:36907
12:38:37.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.696434 22464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.195:36907 every 8 connection(s)
12:38:37.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.697875 22261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:37.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.700414 22293 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=0 applied=20 ignored=0} inserts{seen=15 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:37.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.701001 22293 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap complete.
12:38:37.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.703145 22293 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent bootstrapping tablet: real 0.067s	user 0.053s	sys 0.012s
12:38:37.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.703395 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 22261
12:38:37.704 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:37.710 [ERROR - Time-limited test] (ResourceLeakDetector.java:327) LEAK: HashedWheelTimer.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:312)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:251)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:224)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:203)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:185)
	org.apache.kudu.client.AsyncKuduScanner.<init>(AsyncKuduScanner.java:296)
	org.apache.kudu.client.AsyncKuduScanner.<init>(AsyncKuduScanner.java:431)
	org.apache.kudu.client.AsyncKuduScanner$AsyncKuduScannerBuilder.build(AsyncKuduScanner.java:1417)
	org.apache.kudu.client.TestMultipleLeaderFailover$1.get(TestMultipleLeaderFailover.java:48)
	org.apache.kudu.test.junit.AssertHelpers.assertEventuallyTrue(AssertHelpers.java:40)
	org.apache.kudu.client.TestMultipleLeaderFailover.waitUntilRowCount(TestMultipleLeaderFailover.java:44)
	org.apache.kudu.client.TestMultipleLeaderFailover.testMultipleFailover(TestMultipleLeaderFailover.java:97)
	sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke(Method.java:498)
	org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
	org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
	java.util.concurrent.FutureTask.run(FutureTask.java:266)
	java.lang.Thread.run(Thread.java:750)
12:38:37.710 [ERROR - Time-limited test] (ResourceLeakDetector.java:327) LEAK: HashedWheelTimer.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:312)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:251)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:224)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:203)
	io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:185)
	org.apache.kudu.client.AsyncKuduScanner.<init>(AsyncKuduScanner.java:296)
	org.apache.kudu.client.AsyncKuduScanner.<init>(AsyncKuduScanner.java:431)
	org.apache.kudu.client.AsyncKuduScanner$AsyncKuduScannerBuilder.build(AsyncKuduScanner.java:1417)
	org.apache.kudu.client.TestMultipleLeaderFailover$1.get(TestMultipleLeaderFailover.java:48)
	org.apache.kudu.test.junit.AssertHelpers.assertEventuallyTrue(AssertHelpers.java:40)
	org.apache.kudu.client.TestMultipleLeaderFailover.waitUntilRowCount(TestMultipleLeaderFailover.java:44)
	org.apache.kudu.client.TestMultipleLeaderFailover.testMultipleFailover(TestMultipleLeaderFailover.java:79)
	sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke(Method.java:498)
	org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
	org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
	java.util.concurrent.FutureTask.run(FutureTask.java:266)
	java.lang.Thread.run(Thread.java:750)
12:38:37.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.723995 22293 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:37.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.725616 22293 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Initialized, Role: FOLLOWER
12:38:37.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.726565 22293 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 5.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:37.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.744339 22293 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent starting tablet: real 0.041s	user 0.016s	sys 0.015s
12:38:37.747 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 18 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:37.747 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.193:34365
12:38:37.748 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

12:38:37.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.748487 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 21376
12:38:37.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.759294 22466 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:37.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.759639 22466 heartbeater.cc:464] Registering TS with master...
12:38:37.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.760910 22466 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:37.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.762657 22465 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:37.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.762985 22465 heartbeater.cc:464] Registering TS with master...
12:38:37.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.763548 22465 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:37.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.765424 20162 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:37.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.767068 20162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.195:32789
12:38:37.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.769016 19927 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:37.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.770552 22465 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:37.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.771292 22469 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:37.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.771430 22469 heartbeater.cc:464] Registering TS with master...
12:38:37.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.771751 22469 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:37.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.773379 20056 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:37.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.776854 21590 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:37.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:37.878019 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 21376
12:38:37.906 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:37.906 [DEBUG - kudu-nio-0] (Connection.java:239) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] handling channelInactive
12:38:37.907 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state READY due to: connection closed
12:38:37.907 [DEBUG - Time-limited test] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:37.909 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.909 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.909 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.923 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:37.923 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:37.924 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=17), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] refreshing cache from master, [13ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@309447509(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@73b9b7b6 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@8029c8 -> wakeup thread Time-limited test)) after lookup
12:38:37.925 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:37.927 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.927 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.927 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.943 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:37.943 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:37.944 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=37), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] refreshing cache from master, [13ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [18ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [20ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [20ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] refreshing cache from master, [33ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [36ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@309447509(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@73b9b7b6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@8029c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710))) after lookup
12:38:37.945 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:37.946 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.946 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.946 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.963 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:37.964 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:37.965 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=57), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] refreshing cache from master, [13ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [18ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [20ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [20ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] refreshing cache from master, [33ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [36ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [38ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [39ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [39ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [52ms] refreshing cache from master, [53ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [56ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@309447509(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@73b9b7b6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@8029c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608))) after lookup
12:38:37.965 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:37.967 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.967 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:37.967 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:38.044 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:38.044 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:38.045 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=137), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] refreshing cache from master, [13ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [18ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [20ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [20ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] refreshing cache from master, [33ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [36ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [38ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [39ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [39ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [52ms] refreshing cache from master, [53ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [56ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [58ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [60ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [61ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [133ms] refreshing cache from master, [133ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [136ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@309447509(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@73b9b7b6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608) -> (continuation of Deferred@1548006601 after retry RPC@548140882), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@8029c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608) -> (continuation of Deferred@1548006601 after retry RPC@548140882))) after lookup
12:38:38.046 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:38.046 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:38.046 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:38.047 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:38.223 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:38.224 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:38.224 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=317), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] refreshing cache from master, [13ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [18ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [20ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [20ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] refreshing cache from master, [33ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [36ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [38ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [39ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [39ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [52ms] refreshing cache from master, [53ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [56ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [58ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [60ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [61ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [133ms] refreshing cache from master, [133ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [136ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [138ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [140ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [140ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [312ms] refreshing cache from master, [312ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [316ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@309447509(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@73b9b7b6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608) -> (continuation of Deferred@1548006601 after retry RPC@548140882) -> (continuation of Deferred@1255159347 after retry RPC@987913054), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@8029c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608) -> (continuation of Deferred@1548006601 after retry RPC@548140882) -> (continuation of Deferred@1255159347 after retry RPC@987913054))) after lookup
12:38:38.225 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:38.227 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:38.227 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:38.227 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:39.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.204058 22483 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 78eb3b7a2de2414faa706d782f134af4)
12:38:39.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.204253 22483 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:39.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.204919 22483 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:39.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.207515 22016 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:39.213 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.213335 22393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 7 candidate_status { last_received { term: 6 index: 24 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:39.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.213864 22393 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 5.
12:38:39.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.214885 22019 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4
12:38:39.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.215287 22483 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Leader pre-election won for term 7
12:38:39.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.215444 22483 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 78eb3b7a2de2414faa706d782f134af4)
12:38:39.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.215540 22483 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 6 FOLLOWER]: Advancing to term 7
12:38:39.217 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.217485 22483 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:39.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.218120 22483 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 7 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:39.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.218618 22393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 7 candidate_status { last_received { term: 6 index: 24 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:39.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.218866 22393 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 5 FOLLOWER]: Advancing to term 7
12:38:39.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.219848 22016 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:39.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.222770 22393 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 7.
12:38:39.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.223176 22019 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4
12:38:39.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.223507 22483 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 FOLLOWER]: Leader election won for term 7
12:38:39.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.223968 22483 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 LEADER]: Becoming Leader. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Running, Role: LEADER
12:38:39.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.224489 22483 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 6.24, Last appended by leader: 24, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:39.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.227905 20162 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d reported cstate change: term changed from 6 to 7, leader changed from 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193) to ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194). New cstate: current_term: 7 leader_uuid: "ee5818823eb4422d94f6c2c5957eb31d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:39.244 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:39.244 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:39.245 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=1337), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] refreshing cache from master, [13ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [18ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [20ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [20ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] refreshing cache from master, [33ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [36ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [38ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [39ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [39ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [52ms] refreshing cache from master, [53ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [56ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [58ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [60ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [61ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [133ms] refreshing cache from master, [133ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [136ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [138ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [140ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [140ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [312ms] refreshing cache from master, [312ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [316ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [318ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [320ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [321ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1333ms] refreshing cache from master, [1333ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1336ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@309447509(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@73b9b7b6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608) -> (continuation of Deferred@1548006601 after retry RPC@548140882) -> (continuation of Deferred@1255159347 after retry RPC@987913054) -> (continuation of Deferred@2105752288 after retry RPC@1427355301), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@8029c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1681770408 after retry RPC@1701997710) -> (continuation of Deferred@2011802150 after retry RPC@273521608) -> (continuation of Deferred@1548006601 after retry RPC@548140882) -> (continuation of Deferred@1255159347 after retry RPC@987913054) -> (continuation of Deferred@2105752288 after retry RPC@1427355301))) after lookup
12:38:39.245 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:39.246 [DEBUG - kudu-nio-1] (Connection.java:757) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] Successfully connected to peer
12:38:39.267 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x349ab4fe, L:/127.0.0.1:47030 - R:/127.19.102.194:44869] using TOKEN/null
12:38:39.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.273216 22393 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Refusing update from remote peer ee5818823eb4422d94f6c2c5957eb31d: Log matching property violated. Preceding OpId in replica: term: 5 index: 20. Preceding OpId from leader: term: 7 index: 26. (index mismatch)
12:38:39.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.273447 22016 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d -> Peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Couldn't send request to peer 78eb3b7a2de2414faa706d782f134af4. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:39.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.274120 22490 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 20, Time since last communication: 0.000s
12:38:39.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.287894 22521 mvcc.cc:205] Tried to move back new op lower bound from 7317307061333049344 to 7317307061146972160. Current Snapshot: MvccSnapshot[applied={T|T < 7317307061333049344}]
12:38:39.310 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.193:34365
12:38:39.310 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.311107 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:34365
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41923
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:39.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.664213 22530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:39.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.664575 22530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:39.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.664672 22530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:39.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.672636 22530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:39.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.672820 22530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.193
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.682279 22530 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:34365
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:39.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41923
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:39.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.683597 22530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:39.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.685357 22530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:39.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.701617 22537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:39.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.701814 22538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:39.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:39.702862 22542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:39.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.703836 22530 server_base.cc:1034] running on GCE node
12:38:39.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.704615 22530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:39.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.705840 22530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:39.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.707096 22530 hybrid_clock.cc:648] HybridClock initialized: now 1786451919707002 us; error 54 us; skew 500 ppm
12:38:39.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.707465 22530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:39.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.710506 22530 webserver.cc:460] Webserver started at http://127.19.102.193:41923/ using document root <none> and password file <none>
12:38:39.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.711409 22530 fs_manager.cc:359] Metadata directory not provided
12:38:39.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.711541 22530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:39.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.717991 22530 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.002s	sys 0.001s
12:38:39.721 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.721688 22551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:39.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.723510 22530 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.000s	sys 0.003s
12:38:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.723701 22530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:15 on dist-test-slave-t7g5"
12:38:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.724728 22530 fs_report.cc:383] FS layout report
12:38:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/data
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:39.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:39.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.764585 22530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:39.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.765554 22530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:39.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.765854 22530 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:39.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.766913 22530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:39.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.769210 22560 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:39.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.775143 22530 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:39.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.775296 22530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
12:38:39.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.775475 22530 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:39.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.778977 22530 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:39.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.779114 22530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s	user 0.003s	sys 0.000s
12:38:39.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.779533 22560 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap starting.
12:38:39.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.808528 22560 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Log is configured to *not* fsync() on all Append() calls
12:38:39.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.840665 22560 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=24 ignored=0} inserts{seen=18 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:39.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.842535 22560 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap complete.
12:38:39.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.844364 22560 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent bootstrapping tablet: real 0.065s	user 0.041s	sys 0.021s
12:38:39.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.849488 22530 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.193:34365
12:38:39.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.849542 22727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.193:34365 every 8 connection(s)
12:38:39.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.850890 22530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:39.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.854226 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 22530
12:38:39.855 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:39.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.854586 22560 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:39.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.856081 22560 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Initialized, Role: FOLLOWER
12:38:39.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.856856 22560 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 24, Last appended: 6.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:39.872 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 21 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:39.873 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.194:44869
12:38:39.874 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:39.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.864379 22560 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.012s
12:38:39.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.875315 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 21991
12:38:39.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.889890 22728 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:39.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.890198 22728 heartbeater.cc:464] Registering TS with master...
12:38:39.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.891191 22728 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:39.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.892506 22729 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:39.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.892663 22729 heartbeater.cc:464] Registering TS with master...
12:38:39.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.894577 22729 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:39.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.896570 20162 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:39.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.897632 20162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.193:41089
12:38:39.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.898092 19927 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:39.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.900985 22728 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:39.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.905444 22730 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:39.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.905588 22730 heartbeater.cc:464] Registering TS with master...
12:38:39.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.906059 22730 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:39.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.907574 20056 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:39.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:39.908200 22181 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:40.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:40.003726 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 21991
12:38:40.030 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:40.030246 22284 connection.cc:535] server connection from 127.19.102.194:47789 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
12:38:40.030 [DEBUG - kudu-nio-1] (Connection.java:239) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] handling channelInactive
12:38:40.030 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state READY due to: connection closed
12:38:40.031 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:40.032 [DEBUG - Time-limited test] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:40.034 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.034 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.034 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.043 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:40.043 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:40.044 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=12), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1141528563(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@16e28be1 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7e437e9b -> wakeup thread Time-limited test)) after lookup
12:38:40.044 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:40.045 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.046 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.046 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.064 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:40.064 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:40.065 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=33), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [12ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [14ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [14ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1141528563(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@16e28be1 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7e437e9b -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221))) after lookup
12:38:40.066 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:40.068 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.068 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.068 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.083 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:40.084 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:40.084 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=52), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [12ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [14ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [14ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [33ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [36ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [36ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1141528563(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@16e28be1 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7e437e9b -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769))) after lookup
12:38:40.085 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:40.086 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.086 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.086 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.122 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:40.123 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:40.123 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=91), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [12ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [14ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [14ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [33ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [36ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [36ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [53ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [54ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [54ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [87ms] refreshing cache from master, [87ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [90ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1141528563(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@16e28be1 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769) -> (continuation of Deferred@1234931584 after retry RPC@88518243), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7e437e9b -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769) -> (continuation of Deferred@1234931584 after retry RPC@88518243))) after lookup
12:38:40.124 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:40.126 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.126 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.126 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.604 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:40.604 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:40.605 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=572), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [12ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [14ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [14ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [33ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [36ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [36ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [53ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [54ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [54ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [87ms] refreshing cache from master, [87ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [90ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [92ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [94ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [94ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [568ms] refreshing cache from master, [568ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [571ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1141528563(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@16e28be1 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769) -> (continuation of Deferred@1234931584 after retry RPC@88518243) -> (continuation of Deferred@1040593721 after retry RPC@243509036), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7e437e9b -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769) -> (continuation of Deferred@1234931584 after retry RPC@88518243) -> (continuation of Deferred@1040593721 after retry RPC@243509036))) after lookup
12:38:40.605 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:40.606 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.606 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:40.607 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:41.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.206199 22749 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:41.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.206543 22749 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:41.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.208250 22749 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:41.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:41.209688 22553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:41.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:41.211421 22553 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:41.217 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.216710 22393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 7 candidate_status { last_received { term: 6 index: 24 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:41.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.217974 22555 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d
12:38:41.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.218431 22749 raft_consensus.cc:2726] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
12:38:41.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.505887 22756 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:41.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.506177 22756 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:41.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.508371 22756 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:41.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:41.510182 22285 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:41.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:41.512032 22285 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:41.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.517436 22658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 8 candidate_status { last_received { term: 7 index: 28 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:41.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.517940 22658 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 6.
12:38:41.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.519021 22285 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:41.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.519644 22756 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Leader pre-election won for term 8
12:38:41.520 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.519838 22756 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Starting leader election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:41.520 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.519940 22756 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 7 FOLLOWER]: Advancing to term 8
12:38:41.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.521996 22756 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:41.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.522608 22756 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 8 election: Requested vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:41.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.523722 22658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 8 candidate_status { last_received { term: 7 index: 28 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:41.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.523942 22658 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 6 FOLLOWER]: Advancing to term 8
12:38:41.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:41.524544 22285 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:41.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.529196 22658 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 8.
12:38:41.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.529858 22285 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:41.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.530200 22756 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 FOLLOWER]: Leader election won for term 8
12:38:41.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.530807 22756 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 LEADER]: Becoming Leader. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Running, Role: LEADER
12:38:41.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.531396 22756 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 7.28, Last appended by leader: 28, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:41.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.535367 20162 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff reported cstate change: term changed from 7 to 8, leader changed from ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194) to a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195). New cstate: current_term: 8 leader_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:41.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.987550 22658 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Refusing update from remote peer a96bc5edb0f947ffb1ca060f8a31d9ff: Log matching property violated. Preceding OpId in replica: term: 6 index: 24. Preceding OpId from leader: term: 8 index: 29. (index mismatch)
12:38:41.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:41.988858 22756 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 24, Time since last communication: 0.000s
12:38:42.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.007361 22285 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff -> Peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Couldn't send request to peer ee5818823eb4422d94f6c2c5957eb31d. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:42.184 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:42.185 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:42.185 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=2153), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [2ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [12ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [14ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [14ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [33ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [36ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [36ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [47ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [53ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [54ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [54ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [87ms] refreshing cache from master, [87ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [90ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [92ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [94ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [94ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [568ms] refreshing cache from master, [568ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [571ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [573ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [575ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [575ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [2148ms] refreshing cache from master, [2148ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [2152ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1141528563(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@16e28be1 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769) -> (continuation of Deferred@1234931584 after retry RPC@88518243) -> (continuation of Deferred@1040593721 after retry RPC@243509036) -> (continuation of Deferred@1874293847 after retry RPC@1608603419), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7e437e9b -> wakeup thread Time-limited test -> (continuation of Deferred@1847535988 after retry RPC@990208221) -> (continuation of Deferred@1497084651 after retry RPC@1235759769) -> (continuation of Deferred@1234931584 after retry RPC@88518243) -> (continuation of Deferred@1040593721 after retry RPC@243509036) -> (continuation of Deferred@1874293847 after retry RPC@1608603419))) after lookup
12:38:42.186 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:42.188 [DEBUG - kudu-nio-0] (Connection.java:757) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] Successfully connected to peer
12:38:42.212 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x01ee9825, L:/127.0.0.1:47844 - R:/127.19.102.195:36907] using TOKEN/null
12:38:42.240 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.194:44869
12:38:42.240 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.241861 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:42.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:42.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:42.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:42.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:42.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:42.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.625813 22799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:42.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.626134 22799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:42.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.626221 22799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:42.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.634338 22799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:42.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.634594 22799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.194
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.645699 22799 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:42.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:42.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:42.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.648058 22799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:42.651 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.650713 22799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:42.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.665458 22810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:42.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.665407 22819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:42.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:42.665789 22812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:42.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.666574 22799 server_base.cc:1034] running on GCE node
12:38:42.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.667281 22799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:42.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.668344 22799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:42.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.669611 22799 hybrid_clock.cc:648] HybridClock initialized: now 1786451922669529 us; error 44 us; skew 500 ppm
12:38:42.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.670008 22799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:42.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.675756 22799 webserver.cc:460] Webserver started at http://127.19.102.194:44599/ using document root <none> and password file <none>
12:38:42.676 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.676581 22799 fs_manager.cc:359] Metadata directory not provided
12:38:42.676 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.676723 22799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:42.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.682928 22799 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.000s
12:38:42.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.686604 22825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:42.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.688313 22799 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
12:38:42.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.688500 22799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.689906 22799 fs_report.cc:383] FS layout report
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/data
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:42.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:42.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.709318 22799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:42.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.710261 22799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:42.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.710515 22799 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:42.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.711540 22799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:42.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.713835 22838 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:42.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.719628 22799 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:42.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.719753 22799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s	user 0.001s	sys 0.000s
12:38:42.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.719849 22799 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:42.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.722940 22799 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:42.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.723031 22799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s	user 0.002s	sys 0.000s
12:38:42.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.723454 22838 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap starting.
12:38:42.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.750353 22838 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Log is configured to *not* fsync() on all Append() calls
12:38:42.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.776492 23007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.194:44869 every 8 connection(s)
12:38:42.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.776647 22799 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.194:44869
12:38:42.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.778807 22799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:42.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.780820 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 22799
12:38:42.781 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:42.808 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 24 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:42.809 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.195:36907
12:38:42.809 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

12:38:42.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.811014 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 22261
12:38:42.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.821401 23010 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:42.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.821842 23010 heartbeater.cc:464] Registering TS with master...
12:38:42.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.823019 23010 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:42.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.828758 19927 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:42.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.832777 22456 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:42.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.841912 23009 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:42.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.842132 23009 heartbeater.cc:464] Registering TS with master...
12:38:42.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.842732 23009 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:42.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.844259 20162 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:42.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.845875 20162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.194:36603
12:38:42.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.853994 23019 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:42.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.854207 23019 heartbeater.cc:464] Registering TS with master...
12:38:42.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.854719 23019 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:42.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.856463 20056 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:42.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.858917 22838 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=21 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:42.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.859675 22838 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap complete.
12:38:42.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.861155 22838 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent bootstrapping tablet: real 0.138s	user 0.069s	sys 0.013s
12:38:42.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.867789 22838 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:42.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.868765 22838 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Initialized, Role: FOLLOWER
12:38:42.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.869493 22838 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 7.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:42.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.874425 23009 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:42.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.875319 22838 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.003s
12:38:42.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:42.963042 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 22261
12:38:42.987 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:42.987 [DEBUG - kudu-nio-0] (Connection.java:239) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] handling channelInactive
12:38:42.987 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state READY due to: connection closed
12:38:42.988 [DEBUG - Time-limited test] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:42.989 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:42.989 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:42.989 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.003 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:43.003 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:43.003 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=16), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test)) after lookup
12:38:43.004 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:43.006 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.006 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.006 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.024 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:43.024 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:43.025 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=37), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [16ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549))) after lookup
12:38:43.026 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:43.027 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.027 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.028 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.064 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:43.064 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:43.065 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=77), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [16ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [39ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [71ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [75ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236))) after lookup
12:38:43.066 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:43.068 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.068 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.069 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.123 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:43.124 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:43.124 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=136), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [16ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [39ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [71ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [75ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [77ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [80ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [81ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270))) after lookup
12:38:43.125 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:43.127 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.127 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.127 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.363 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:43.363 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:43.364 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=376), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [16ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [39ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [71ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [75ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [77ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [80ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [81ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [137ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [139ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [139ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [371ms] refreshing cache from master, [371ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [374ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270) -> (continuation of Deferred@632743065 after retry RPC@243153077), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270) -> (continuation of Deferred@632743065 after retry RPC@243153077))) after lookup
12:38:43.365 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:43.367 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.367 [DEBUG - kudu-nio-0] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:43.367 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:44.064 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:44.064 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:44.065 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=1077), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [16ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [39ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [71ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [75ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [77ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [80ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [81ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [137ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [139ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [139ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [371ms] refreshing cache from master, [371ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [374ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [377ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [379ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [379ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [1072ms] refreshing cache from master, [1072ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1075ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270) -> (continuation of Deferred@632743065 after retry RPC@243153077) -> (continuation of Deferred@595955276 after retry RPC@1547809049), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270) -> (continuation of Deferred@632743065 after retry RPC@243153077) -> (continuation of Deferred@595955276 after retry RPC@1547809049))) after lookup
12:38:44.065 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:44.066 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:44.067 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state CONNECTING due to: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:44.067 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907
12:38:44.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.269812 23031 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a96bc5edb0f947ffb1ca060f8a31d9ff)
12:38:44.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.270030 23031 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:44.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.270761 23031 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:44.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:44.272886 22555 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:44.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.279295 22940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 9 candidate_status { last_received { term: 8 index: 32 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d" is_pre_election: true
12:38:44.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.279800 22940 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78eb3b7a2de2414faa706d782f134af4 in term 7.
12:38:44.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.280812 22553 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:44.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.281188 23031 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Leader pre-election won for term 9
12:38:44.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.281319 23031 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a96bc5edb0f947ffb1ca060f8a31d9ff)
12:38:44.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.281440 23031 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 8 FOLLOWER]: Advancing to term 9
12:38:44.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.284829 23031 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:44.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.285527 23031 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 9 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:44.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.286093 22940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 9 candidate_status { last_received { term: 8 index: 32 } } ignore_live_leader: false dest_uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:44.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.286300 22940 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 7 FOLLOWER]: Advancing to term 9
12:38:44.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:44.287035 22555 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111)
12:38:44.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.290537 22940 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78eb3b7a2de2414faa706d782f134af4 in term 9.
12:38:44.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.291049 22553 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, ee5818823eb4422d94f6c2c5957eb31d; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff
12:38:44.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.291388 23031 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Leader election won for term 9
12:38:44.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.291852 23031 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 LEADER]: Becoming Leader. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Running, Role: LEADER
12:38:44.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.292378 23031 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 8.32, Last appended by leader: 32, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:44.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.295912 20162 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 reported cstate change: term changed from 8 to 9, leader changed from a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195) to 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193). New cstate: current_term: 9 leader_uuid: "78eb3b7a2de2414faa706d782f134af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: HEALTHY } } }
12:38:44.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.811403 22940 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Refusing update from remote peer 78eb3b7a2de2414faa706d782f134af4: Log matching property violated. Preceding OpId in replica: term: 7 index: 28. Preceding OpId from leader: term: 9 index: 33. (index mismatch)
12:38:44.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:44.812314 23060 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 28, Time since last communication: 0.000s
12:38:44.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:44.817248 22555 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 -> Peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Couldn't send request to peer a96bc5edb0f947ffb1ca060f8a31d9ff. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:46.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:46.706629 22555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111) [suppressed 6 similar messages]
12:38:47.123 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:47.123 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:47.124 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=14, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=4136), Traces: [0ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [2ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [11ms] refreshing cache from master, [11ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [16ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [18ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [19ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [35ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [37ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [39ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [40ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [71ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [75ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [77ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [80ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [81ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [137ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [139ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [139ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [371ms] refreshing cache from master, [371ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [374ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [377ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [379ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [379ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [1072ms] refreshing cache from master, [1072ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1075ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [1077ms] sending RPC to server a96bc5edb0f947ffb1ca060f8a31d9ff, [1079ms] received response from server a96bc5edb0f947ffb1ca060f8a31d9ff: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [1079ms] delaying RPC due to: Network error: Failed to connect to peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907): Connection refused: /127.19.102.195:36907, [4132ms] refreshing cache from master, [4132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [4135ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1545960093(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@498dff90 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270) -> (continuation of Deferred@632743065 after retry RPC@243153077) -> (continuation of Deferred@595955276 after retry RPC@1547809049) -> (continuation of Deferred@16787705 after retry RPC@432945826), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6406050 -> wakeup thread Time-limited test -> (continuation of Deferred@724681003 after retry RPC@779511549) -> (continuation of Deferred@1085984273 after retry RPC@494430236) -> (continuation of Deferred@1882686516 after retry RPC@777071270) -> (continuation of Deferred@632743065 after retry RPC@243153077) -> (continuation of Deferred@595955276 after retry RPC@1547809049) -> (continuation of Deferred@16787705 after retry RPC@432945826))) after lookup
12:38:47.125 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:47.127 [DEBUG - kudu-nio-0] (Connection.java:757) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] Successfully connected to peer
12:38:47.143 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x3694a109, L:/127.0.0.1:53696 - R:/127.19.102.193:34365] using TOKEN/null
12:38:47.168 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.195:36907
12:38:47.168 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.169144 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:36907
12:38:47.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43649
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:47.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:47.248 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.247572 22555 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 -> Peer a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907): Couldn't send request to peer a96bc5edb0f947ffb1ca060f8a31d9ff. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.195:36907: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
12:38:47.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.540203 23102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:47.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.540634 23102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:47.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.540783 23102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.548576 23102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:47.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.548800 23102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.195
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.559105 23102 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:47.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.195:36907
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.195
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=43649
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.195
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/logs
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:47.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:47.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.561252 23102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:47.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.563938 23102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:47.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.578025 23118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:47.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.578207 23121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:47.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.579259 23116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:47.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.579692 23102 server_base.cc:1034] running on GCE node
12:38:47.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.580668 23102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:47.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.581897 23102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:47.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.583257 23102 hybrid_clock.cc:648] HybridClock initialized: now 1786451927583158 us; error 50 us; skew 500 ppm
12:38:47.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.583643 23102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:47.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.586186 23102 webserver.cc:460] Webserver started at http://127.19.102.195:43649/ using document root <none> and password file <none>
12:38:47.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.586936 23102 fs_manager.cc:359] Metadata directory not provided
12:38:47.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.587066 23102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:47.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.592854 23102 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
12:38:47.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.596035 23130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:47.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.597829 23102 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.598012 23102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.599169 23102 fs_report.cc:383] FS layout report
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/wal
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/data
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:47.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:47.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:47.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.613574 23102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:47.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.614684 23102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:47.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.615012 23102 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:47.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.616098 23102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:47.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.618777 23140 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:47.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.624931 23102 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:47.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.625034 23102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
12:38:47.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.625128 23102 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:47.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.628279 23102 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:47.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.628377 23102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s	user 0.001s	sys 0.000s
12:38:47.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.628777 23140 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap starting.
12:38:47.660 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.660341 23140 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Log is configured to *not* fsync() on all Append() calls
12:38:47.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.674319 23102 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.195:36907
12:38:47.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.674340 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.195:36907 every 8 connection(s)
12:38:47.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.675563 23102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-2/data/info.pb
12:38:47.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.680927 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 23102
12:38:47.681 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:47.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.704316 23140 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=24 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:47.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.705217 23140 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Bootstrap complete.
12:38:47.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.706576 23140 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent bootstrapping tablet: real 0.078s	user 0.056s	sys 0.016s
12:38:47.706 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 27 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:47.707 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.193:34365
12:38:47.707 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

12:38:47.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.708092 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 22530
12:38:47.720 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.719689 23140 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:47.720 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.720674 23140 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Initialized, Role: FOLLOWER
12:38:47.721 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.721412 23140 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 8.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:47.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.726241 22723 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:47.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.727072 23294 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:47.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.727243 23295 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:47.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.727445 23295 heartbeater.cc:464] Registering TS with master...
12:38:47.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.727450 23294 heartbeater.cc:464] Registering TS with master...
12:38:47.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.728591 23295 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:47.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.730589 23294 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:47.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.731534 19927 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:47.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.732291 23296 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:47.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.732555 23296 heartbeater.cc:464] Registering TS with master...
12:38:47.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.733016 23296 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:47.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.733031 20162 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:47.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.735144 20162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.195:38787
12:38:47.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.736676 20056 ts_manager.cc:194] Re-registered known tserver with Master: a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907)
12:38:47.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.739302 23294 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:47.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.740193 23140 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff: Time spent starting tablet: real 0.033s	user 0.013s	sys 0.009s
12:38:47.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:47.843612 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 22530
12:38:47.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.871706 20026 connection.cc:535] server connection from 127.19.102.193:56059 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
12:38:47.872 [DEBUG - kudu-nio-0] (Connection.java:239) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] handling channelInactive
12:38:47.872 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state READY due to: connection closed
12:38:47.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.872370 20145 connection.cc:535] server connection from 127.19.102.193:41089 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
12:38:47.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:47.872378 19905 connection.cc:535] server connection from 127.19.102.193:52089 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
12:38:47.872 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:47.873 [DEBUG - Time-limited test] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:47.875 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.875 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.875 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.882 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:47.883 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:47.883 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=2, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=10), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test)) after lookup
12:38:47.883 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:47.886 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.886 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.886 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.903 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:47.903 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:47.903 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=4, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=30), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [10ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [13ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [26ms] refreshing cache from master, [26ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [29ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073))) after lookup
12:38:47.904 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:47.905 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.905 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.905 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.943 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:47.943 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:47.943 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=6, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=70), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [10ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [13ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [26ms] refreshing cache from master, [26ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [29ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [31ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [32ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [66ms] refreshing cache from master, [66ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [69ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937))) after lookup
12:38:47.944 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:47.945 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.946 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.946 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.983 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:47.983 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:47.983 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=8, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=110), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [10ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [13ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [26ms] refreshing cache from master, [26ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [29ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [31ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [32ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [66ms] refreshing cache from master, [66ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [69ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [70ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [73ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [73ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [106ms] refreshing cache from master, [106ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [109ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223))) after lookup
12:38:47.984 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:47.985 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.985 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:47.985 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:48.002 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:48.002 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:48.003 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=10, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=129), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [10ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [13ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [26ms] refreshing cache from master, [26ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [29ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [31ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [32ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [66ms] refreshing cache from master, [66ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [69ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [70ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [73ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [73ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [106ms] refreshing cache from master, [106ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [109ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [111ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [112ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [112ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [126ms] refreshing cache from master, [126ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [129ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223) -> (continuation of Deferred@1664303288 after retry RPC@137286716), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223) -> (continuation of Deferred@1664303288 after retry RPC@137286716))) after lookup
12:38:48.003 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:48.005 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:48.006 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:48.006 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:49.084 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:49.084 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)[L],a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:49.084 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=12, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=1211), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [10ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [13ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [26ms] refreshing cache from master, [26ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [29ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [31ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [32ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [66ms] refreshing cache from master, [66ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [69ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [70ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [73ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [73ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [106ms] refreshing cache from master, [106ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [109ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [111ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [112ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [112ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [126ms] refreshing cache from master, [126ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [129ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [130ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [133ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [133ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1207ms] refreshing cache from master, [1207ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1210ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223) -> (continuation of Deferred@1664303288 after retry RPC@137286716) -> (continuation of Deferred@673237439 after retry RPC@1448179635), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223) -> (continuation of Deferred@1664303288 after retry RPC@137286716) -> (continuation of Deferred@673237439 after retry RPC@1448179635))) after lookup
12:38:49.085 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] connecting to peer
12:38:49.086 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:49.086 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] cleaning up while in state CONNECTING due to: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:49.086 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365
12:38:49.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.093922 23318 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 78eb3b7a2de2414faa706d782f134af4)
12:38:49.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.094137 23318 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:49.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.096035 23318 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:49.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:49.100364 22828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:49.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:49.102283 22828 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:49.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.108698 23230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 10 candidate_status { last_received { term: 9 index: 36 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:49.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.109258 23230 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 8.
12:38:49.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.110532 22830 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4
12:38:49.111 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.110976 23318 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Leader pre-election won for term 10
12:38:49.111 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.111124 23318 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 78eb3b7a2de2414faa706d782f134af4)
12:38:49.111 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.111218 23318 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 9 FOLLOWER]: Advancing to term 10
12:38:49.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.113365 23318 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:49.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.114028 23318 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 10 election: Requested vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:49.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.114697 23230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "ee5818823eb4422d94f6c2c5957eb31d" candidate_term: 10 candidate_status { last_received { term: 9 index: 36 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff"
12:38:49.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.114950 23230 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 8 FOLLOWER]: Advancing to term 10
12:38:49.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:49.115676 22828 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111)
12:38:49.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.120262 23230 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5818823eb4422d94f6c2c5957eb31d in term 10.
12:38:49.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.120985 22830 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d; no voters: 78eb3b7a2de2414faa706d782f134af4
12:38:49.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.121369 23318 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 10 FOLLOWER]: Leader election won for term 10
12:38:49.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.121873 23318 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 10 LEADER]: Becoming Leader. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Running, Role: LEADER
12:38:49.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.122391 23318 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 9.36, Last appended by leader: 36, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:49.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.127210 20162 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d reported cstate change: term changed from 9 to 10, leader changed from 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193) to ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194). New cstate: current_term: 10 leader_uuid: "ee5818823eb4422d94f6c2c5957eb31d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:49.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:49.524963 22828 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d -> Peer 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365): Couldn't send request to peer 78eb3b7a2de2414faa706d782f134af4. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.193:34365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:49.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.706259 23230 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Refusing update from remote peer ee5818823eb4422d94f6c2c5957eb31d: Log matching property violated. Preceding OpId in replica: term: 8 index: 32. Preceding OpId from leader: term: 10 index: 37. (index mismatch)
12:38:49.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:49.707535 23318 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 32, Time since last communication: 0.000s
12:38:50.885 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:50.885 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:50.886 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=14, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=3012), Traces: [0ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [2ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [2ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [9ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [10ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [13ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [13ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [26ms] refreshing cache from master, [26ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [29ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [31ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [32ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [32ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [66ms] refreshing cache from master, [66ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [69ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [70ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [73ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [73ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [106ms] refreshing cache from master, [106ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [109ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [111ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [112ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [112ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [126ms] refreshing cache from master, [126ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [129ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [130ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [133ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [133ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1207ms] refreshing cache from master, [1207ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [1210ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [1212ms] sending RPC to server 78eb3b7a2de2414faa706d782f134af4, [1213ms] received response from server 78eb3b7a2de2414faa706d782f134af4: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [1213ms] delaying RPC due to: Network error: Failed to connect to peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365): Connection refused: /127.19.102.193:34365, [3006ms] refreshing cache from master, [3006ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [3010ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@1915048225(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5fd9c1d -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223) -> (continuation of Deferred@1664303288 after retry RPC@137286716) -> (continuation of Deferred@673237439 after retry RPC@1448179635) -> (continuation of Deferred@1227525026 after retry RPC@1932945216), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4f6a0b4c -> wakeup thread Time-limited test -> (continuation of Deferred@1174312961 after retry RPC@1515314073) -> (continuation of Deferred@126795270 after retry RPC@253425937) -> (continuation of Deferred@1698164385 after retry RPC@1035813223) -> (continuation of Deferred@1664303288 after retry RPC@137286716) -> (continuation of Deferred@673237439 after retry RPC@1448179635) -> (continuation of Deferred@1227525026 after retry RPC@1932945216))) after lookup
12:38:50.887 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:50.888 [DEBUG - kudu-nio-0] (Connection.java:757) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] Successfully connected to peer
12:38:50.908 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x28f37c28, L:/127.0.0.1:43974 - R:/127.19.102.194:44869] using TOKEN/null
12:38:50.938 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.193:34365
12:38:50.938 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:50.939138 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:34365
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41923
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:50.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:51.316 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.316282 23373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:51.316 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.316603 23373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:51.316 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.316824 23373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:51.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.325378 23373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:51.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.325604 23373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.193
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.337913 23373 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.193:34365
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.193
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41923
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.193
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/logs
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:51.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:51.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.339908 23373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:51.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.342384 23373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:51.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.359248 23382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:51.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.359215 23387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:51.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:51.359181 23383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:51.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.360033 23373 server_base.cc:1034] running on GCE node
12:38:51.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.360669 23373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:51.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.361855 23373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:51.363 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.363139 23373 hybrid_clock.cc:648] HybridClock initialized: now 1786451931363046 us; error 47 us; skew 500 ppm
12:38:51.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.363509 23373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:51.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.367002 23373 webserver.cc:460] Webserver started at http://127.19.102.193:41923/ using document root <none> and password file <none>
12:38:51.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.367753 23373 fs_manager.cc:359] Metadata directory not provided
12:38:51.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.367883 23373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:51.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.373631 23373 fs_manager.cc:709] Time spent opening directory manager: real 0.004s	user 0.002s	sys 0.000s
12:38:51.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.376921 23401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:51.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.378523 23373 fs_manager.cc:725] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.378715 23373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:15 on dist-test-slave-t7g5"
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.380082 23373 fs_report.cc:383] FS layout report
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/wal
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/data
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:51.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:51.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:51.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.411525 23373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:51.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.412411 23373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:51.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.412670 23373 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:51.413 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.413730 23373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:51.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.415973 23416 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:51.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.423105 23373 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:51.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.423291 23373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s	user 0.001s	sys 0.000s
12:38:51.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.423405 23373 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:51.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.427235 23373 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:51.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.427345 23373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s	user 0.003s	sys 0.000s
12:38:51.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.427788 23416 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap starting.
12:38:51.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.458442 23416 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Log is configured to *not* fsync() on all Append() calls
12:38:51.477 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.477089 23373 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.193:34365
12:38:51.477 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.477123 23575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.193:34365 every 8 connection(s)
12:38:51.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.478497 23373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-0/data/info.pb
12:38:51.482 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.482640 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 23373
12:38:51.483 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:51.508 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 30 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:51.509 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.102.194:44869
12:38:51.509 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:51.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.510028 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 22799
12:38:51.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.511356 23416 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=27 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:51.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.512038 23416 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Bootstrap complete.
12:38:51.513 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.513446 23416 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent bootstrapping tablet: real 0.086s	user 0.044s	sys 0.029s
12:38:51.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.522523 23416 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:51.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.524014 23416 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eb3b7a2de2414faa706d782f134af4, State: Initialized, Role: FOLLOWER
12:38:51.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.525031 23416 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 9.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:51.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.528743 23579 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:51.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.529059 23579 heartbeater.cc:464] Registering TS with master...
12:38:51.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.530236 23579 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:51.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.533900 19931 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:51.546 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.545994 23577 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:51.546 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.546183 23577 heartbeater.cc:464] Registering TS with master...
12:38:51.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.546643 23577 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:51.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.547128 23580 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:51.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.547238 23580 heartbeater.cc:464] Registering TS with master...
12:38:51.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.547626 23580 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:51.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.548770 20162 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:51.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.549067 20056 ts_manager.cc:194] Re-registered known tserver with Master: 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:51.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.549178 23416 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4: Time spent starting tablet: real 0.036s	user 0.018s	sys 0.016s
12:38:51.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.550660 20162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.193:33125
12:38:51.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.554486 23577 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:51.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.555382 23001 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:51.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:51.664973 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 22799
12:38:51.698 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:51.698 [DEBUG - kudu-nio-0] (Connection.java:239) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] handling channelInactive
12:38:51.699 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state READY due to: connection closed
12:38:51.720 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:51.722 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.722 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.722 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.744 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:51.744 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:51.744 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=3, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=46), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [1ms] delaying RPC due to: Illegal state: connection is terminated, [21ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [24ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [24ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@618195948(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5a4470a6 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6e3fe1ff -> wakeup thread Time-limited test)) after lookup
12:38:51.745 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:51.746 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.746 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.747 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.763 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:51.763 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:51.764 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=5, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=65), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [1ms] delaying RPC due to: Illegal state: connection is terminated, [21ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [24ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [24ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [61ms] refreshing cache from master, [61ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [65ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@618195948(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5a4470a6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6e3fe1ff -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591))) after lookup
12:38:51.765 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:51.766 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.766 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.766 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.783 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:51.783 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:51.783 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=7, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=84), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [1ms] delaying RPC due to: Illegal state: connection is terminated, [21ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [24ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [24ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [61ms] refreshing cache from master, [61ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [65ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [66ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [68ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [68ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [84ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@618195948(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5a4470a6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6e3fe1ff -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435))) after lookup
12:38:51.784 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:51.785 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.785 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.786 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.963 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:51.964 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:51.964 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=9, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=265), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [1ms] delaying RPC due to: Illegal state: connection is terminated, [21ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [24ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [24ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [61ms] refreshing cache from master, [61ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [65ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [66ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [68ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [68ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [84ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [86ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [87ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [88ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [262ms] refreshing cache from master, [262ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [265ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@618195948(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5a4470a6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435) -> (continuation of Deferred@507285017 after retry RPC@474271573), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6e3fe1ff -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435) -> (continuation of Deferred@507285017 after retry RPC@474271573))) after lookup
12:38:51.965 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:51.966 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.966 [DEBUG - kudu-nio-1] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:51.967 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:52.623 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:52.623 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907),ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)[L]]}]
12:38:52.624 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=11, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=925), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [1ms] delaying RPC due to: Illegal state: connection is terminated, [21ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [24ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [24ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [61ms] refreshing cache from master, [61ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [65ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [66ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [68ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [68ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [84ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [86ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [87ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [88ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [262ms] refreshing cache from master, [262ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [265ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [267ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [268ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [269ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [921ms] refreshing cache from master, [921ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [924ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@618195948(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5a4470a6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435) -> (continuation of Deferred@507285017 after retry RPC@474271573) -> (continuation of Deferred@1235400405 after retry RPC@1720348940), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6e3fe1ff -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435) -> (continuation of Deferred@507285017 after retry RPC@474271573) -> (continuation of Deferred@1235400405 after retry RPC@1720348940))) after lookup
12:38:52.625 [DEBUG - kudu-nio-1] (Connection.java:748) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] connecting to peer
12:38:52.626 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:52.626 [DEBUG - kudu-nio-0] (Connection.java:709) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] cleaning up while in state CONNECTING due to: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:52.626 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869) for tablet 77cb47d5bfc441bd91b99ae31dc3eed3: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869
12:38:52.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.746559 23605 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
12:38:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.746893 23605 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:52.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.748595 23605 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195:36907), ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:52.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:52.750078 23403 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:52.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:52.752314 23403 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:52.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.756845 23230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "78eb3b7a2de2414faa706d782f134af4" candidate_term: 10 candidate_status { last_received { term: 9 index: 36 } } ignore_live_leader: false dest_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" is_pre_election: true
12:38:52.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.758008 23405 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4; no voters: a96bc5edb0f947ffb1ca060f8a31d9ff, ee5818823eb4422d94f6c2c5957eb31d
12:38:52.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.758601 23605 raft_consensus.cc:2726] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
12:38:52.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.829793 23612 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:52.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.830044 23612 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:52.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.831972 23612 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:52.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:52.845167 23133 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:52.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:52.847237 23133 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:52.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.851625 23506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 11 candidate_status { last_received { term: 10 index: 40 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4" is_pre_election: true
12:38:52.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.852051 23506 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 9.
12:38:52.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.853073 23133 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:52.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.853545 23612 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Leader pre-election won for term 11
12:38:52.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.853698 23612 raft_consensus.cc:482] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Starting leader election (detected failure of leader ee5818823eb4422d94f6c2c5957eb31d)
12:38:52.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.853793 23612 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 10 FOLLOWER]: Advancing to term 11
12:38:52.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.855865 23612 raft_consensus.cc:504] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:52.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.856467 23612 leader_election.cc:288] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 11 election: Requested vote from peers ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869), 78eb3b7a2de2414faa706d782f134af4 (127.19.102.193:34365)
12:38:52.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.857878 23506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cb47d5bfc441bd91b99ae31dc3eed3" candidate_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" candidate_term: 11 candidate_status { last_received { term: 10 index: 40 } } ignore_live_leader: false dest_uuid: "78eb3b7a2de2414faa706d782f134af4"
12:38:52.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.858108 23506 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 9 FOLLOWER]: Advancing to term 11
12:38:52.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:52.858587 23133 leader_election.cc:334] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111)
12:38:52.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.862888 23506 raft_consensus.cc:2454] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bc5edb0f947ffb1ca060f8a31d9ff in term 11.
12:38:52.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.863669 23133 leader_election.cc:302] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78eb3b7a2de2414faa706d782f134af4, a96bc5edb0f947ffb1ca060f8a31d9ff; no voters: ee5818823eb4422d94f6c2c5957eb31d
12:38:52.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.864148 23612 raft_consensus.cc:2781] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 11 FOLLOWER]: Leader election won for term 11
12:38:52.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.864761 23612 raft_consensus.cc:686] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [term 11 LEADER]: Becoming Leader. State: Replica: a96bc5edb0f947ffb1ca060f8a31d9ff, State: Running, Role: LEADER
12:38:52.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.865317 23612 consensus_queue.cc:237] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 10.40, Last appended by leader: 40, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:52.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:52.869234 20162 catalog_manager.cc:5495] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff reported cstate change: term changed from 10 to 11, leader changed from ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194) to a96bc5edb0f947ffb1ca060f8a31d9ff (127.19.102.195). New cstate: current_term: 11 leader_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } health_report { overall_health: UNKNOWN } } }
12:38:53.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:53.254411 23506 raft_consensus.cc:1261] T 77cb47d5bfc441bd91b99ae31dc3eed3 P 78eb3b7a2de2414faa706d782f134af4 [term 11 FOLLOWER]: Refusing update from remote peer a96bc5edb0f947ffb1ca060f8a31d9ff: Log matching property violated. Preceding OpId in replica: term: 9 index: 36. Preceding OpId from leader: term: 11 index: 41. (index mismatch)
12:38:53.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:53.255328 23612 consensus_queue.cc:1035] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 36, Time since last communication: 0.000s
12:38:53.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:53.269316 23133 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff -> Peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Couldn't send request to peer ee5818823eb4422d94f6c2c5957eb31d. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:55.683 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 77cb47d5bfc441bd91b99ae31dc3eed3 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283' with partition [<start>, <end>)
12:38:55.683 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=77cb47d5bfc441bd91b99ae31dc3eed3@[78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365),a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)[L],ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)]}]
12:38:55.683 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, attempt=13, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=3984), Traces: [0ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [1ms] delaying RPC due to: Illegal state: connection is terminated, [21ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [24ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [24ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [42ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [45ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [47ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [49ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [49ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [61ms] refreshing cache from master, [61ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [65ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [66ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [68ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [68ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [84ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [86ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [87ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [88ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [262ms] refreshing cache from master, [262ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [265ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [267ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [268ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [269ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [921ms] refreshing cache from master, [921ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [924ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, [927ms] sending RPC to server ee5818823eb4422d94f6c2c5957eb31d, [928ms] received response from server ee5818823eb4422d94f6c2c5957eb31d: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [928ms] delaying RPC due to: Network error: Failed to connect to peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869): Connection refused: /127.19.102.194:44869, [3981ms] refreshing cache from master, [3982ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.102.252:36537, [3984ms] Sub RPC GetTableLocations: received response from server master-127.19.102.252:36537: OK, deferred=Deferred@618195948(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$83/118600881@5a4470a6 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435) -> (continuation of Deferred@507285017 after retry RPC@474271573) -> (continuation of Deferred@1235400405 after retry RPC@1720348940) -> (continuation of Deferred@580191999 after retry RPC@396497328), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6e3fe1ff -> wakeup thread Time-limited test -> (continuation of Deferred@1662381371 after retry RPC@1924454591) -> (continuation of Deferred@613673147 after retry RPC@1543184435) -> (continuation of Deferred@507285017 after retry RPC@474271573) -> (continuation of Deferred@1235400405 after retry RPC@1720348940) -> (continuation of Deferred@580191999 after retry RPC@396497328))) after lookup
12:38:55.684 [DEBUG - kudu-nio-1] (Connection.java:748) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] connecting to peer
12:38:55.685 [DEBUG - kudu-nio-1] (Connection.java:757) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] Successfully connected to peer
12:38:55.706 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0xf6bed1ca, L:/127.0.0.1:57162 - R:/127.19.102.195:36907] using TOKEN/null
12:38:55.731 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.102.194:44869
12:38:55.732 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

12:38:55.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:55.732725 19867 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task3TN6t_/build/asan/bin/kudu
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
12:38:55.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
12:38:55.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:55.733160 23133 consensus_peers.cc:487] T 77cb47d5bfc441bd91b99ae31dc3eed3 P a96bc5edb0f947ffb1ca060f8a31d9ff -> Peer ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869): Couldn't send request to peer ee5818823eb4422d94f6c2c5957eb31d. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.194:44869: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
12:38:56.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.097932 23671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
12:38:56.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.098233 23671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
12:38:56.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.098325 23671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
12:38:56.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.106305 23671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
12:38:56.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.106480 23671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.102.194
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.115810 23671 tablet_server_runner.cc:78] Tablet server non-default flags:
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.102.212:39011
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.102.194:44869
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.102.194
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44599
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.102.254:39737,127.19.102.253:33449,127.19.102.252:36537
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.102.194
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/logs
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4580142
12:38:56.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
12:38:56.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.117125 23671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
12:38:56.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.118949 23671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
12:38:56.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.132365 23681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:56.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.132310 23687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:56.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:56.132405 23685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
12:38:56.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.133180 23671 server_base.cc:1034] running on GCE node
12:38:56.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.133901 23671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
12:38:56.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.134826 23671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
12:38:56.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.136070 23671 hybrid_clock.cc:648] HybridClock initialized: now 1786451936136003 us; error 39 us; skew 500 ppm
12:38:56.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.136350 23671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
12:38:56.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.146142 23671 webserver.cc:460] Webserver started at http://127.19.102.194:44599/ using document root <none> and password file <none>
12:38:56.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.146842 23671 fs_manager.cc:359] Metadata directory not provided
12:38:56.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.146978 23671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
12:38:56.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.152235 23671 fs_manager.cc:709] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.001s
12:38:56.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.155774 23699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
12:38:56.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.157294 23671 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.000s
12:38:56.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.157480 23671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data,/tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:56.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "ee5818823eb4422d94f6c2c5957eb31d"
12:38:56.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 12:38:16 on dist-test-slave-t7g5"
12:38:56.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.158623 23671 fs_report.cc:383] FS layout report
12:38:56.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
12:38:56.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/wal
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/data
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
12:38:56.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
12:38:56.181 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.180963 23671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips  26 Jan 2017
12:38:56.182 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.181942 23671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
12:38:56.182 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.182243 23671 kserver.cc:163] Server-wide thread pool size limit: 3276
12:38:56.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.183358 23671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
12:38:56.185 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.185551 23707 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
12:38:56.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.192130 23671 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
12:38:56.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.192258 23671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
12:38:56.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.192396 23671 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
12:38:56.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.196810 23671 ts_tablet_manager.cc:610] Registered 1 tablets
12:38:56.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.196916 23671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s	user 0.002s	sys 0.000s
12:38:56.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.198550 23707 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap starting.
12:38:56.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.228724 23707 log.cc:824] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Log is configured to *not* fsync() on all Append() calls
12:38:56.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.247509 23671 rpc_server.cc:307] RPC server started. Bound to: 127.19.102.194:44869
12:38:56.248 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.247524 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.102.194:44869 every 8 connection(s)
12:38:56.248 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.248849 23671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3TN6t_/test-tmp/mini-kudu-cluster2688843485335226414/ts-1/data/info.pb
12:38:56.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.249549 19867 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3TN6t_/build/asan/bin/kudu as pid 23671
12:38:56.250 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
12:38:56.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.279556 23707 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=30 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
12:38:56.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.280436 23707 tablet_bootstrap.cc:492] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Bootstrap complete.
12:38:56.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.282099 23707 ts_tablet_manager.cc:1397] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent bootstrapping tablet: real 0.085s	user 0.065s	sys 0.015s
12:38:56.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.293332 23707 raft_consensus.cc:348] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:56.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.294824 23707 raft_consensus.cc:729] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5818823eb4422d94f6c2c5957eb31d, State: Initialized, Role: FOLLOWER
12:38:56.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.295625 23707 consensus_queue.cc:260] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 10.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bc5edb0f947ffb1ca060f8a31d9ff" member_type: VOTER last_known_addr { host: "127.19.102.195" port: 36907 } } peers { permanent_uuid: "ee5818823eb4422d94f6c2c5957eb31d" member_type: VOTER last_known_addr { host: "127.19.102.194" port: 44869 } } peers { permanent_uuid: "78eb3b7a2de2414faa706d782f134af4" member_type: VOTER last_known_addr { host: "127.19.102.193" port: 34365 } }
12:38:56.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.318862 23707 ts_tablet_manager.cc:1428] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d: Time spent starting tablet: real 0.037s	user 0.023s	sys 0.012s
12:38:56.330 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:56.347 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1374) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786451897283, tablet=77cb47d5bfc441bd91b99ae31dc3eed3, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
12:38:56.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.347543 23882 heartbeater.cc:347] Connected to a master server at 127.19.102.252:36537
12:38:56.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.347921 23882 heartbeater.cc:464] Registering TS with master...
12:38:56.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.348953 23813 raft_consensus.cc:3037] T 77cb47d5bfc441bd91b99ae31dc3eed3 P ee5818823eb4422d94f6c2c5957eb31d [term 10 FOLLOWER]: Advancing to term 11
12:38:56.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.349053 23882 heartbeater.cc:511] Master 127.19.102.252:36537 requested a full tablet report, sending...
12:38:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.350749 23899 heartbeater.cc:347] Connected to a master server at 127.19.102.253:33449
12:38:56.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.350916 23899 heartbeater.cc:464] Registering TS with master...
12:38:56.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.351428 23899 heartbeater.cc:511] Master 127.19.102.253:33449 requested a full tablet report, sending...
12:38:56.354 [DEBUG - main] (Connection.java:575) [peer master-127.19.102.254:39737(127.19.102.254:39737)] disconnecting while in state READY
12:38:56.356 [DEBUG - main] (Connection.java:575) [peer ee5818823eb4422d94f6c2c5957eb31d(127.19.102.194:44869)] disconnecting while in state TERMINATED
12:38:56.356 [DEBUG - main] (Connection.java:575) [peer master-127.19.102.253:33449(127.19.102.253:33449)] disconnecting while in state READY
12:38:56.356 [DEBUG - main] (Connection.java:575) [peer master-127.19.102.252:36537(127.19.102.252:36537)] disconnecting while in state READY
12:38:56.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.361264 23883 heartbeater.cc:347] Connected to a master server at 127.19.102.254:39737
12:38:56.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.361459 23883 heartbeater.cc:464] Registering TS with master...
12:38:56.362 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.19.102.252:36537(127.19.102.252:36537)] handling channelInactive
12:38:56.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.362017 23883 heartbeater.cc:511] Master 127.19.102.254:39737 requested a full tablet report, sending...
12:38:56.362 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.19.102.254:39737(127.19.102.254:39737)] handling channelInactive
12:38:56.362 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.19.102.254:39737(127.19.102.254:39737)] cleaning up while in state READY due to: connection closed
12:38:56.362 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.19.102.253:33449(127.19.102.253:33449)] handling channelInactive
12:38:56.363 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.19.102.253:33449(127.19.102.253:33449)] cleaning up while in state READY due to: connection closed
12:38:56.363 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.19.102.252:36537(127.19.102.252:36537)] cleaning up while in state READY due to: connection closed
12:38:56.366 [DEBUG - main] (Connection.java:575) [peer 78eb3b7a2de2414faa706d782f134af4(127.19.102.193:34365)] disconnecting while in state TERMINATED
12:38:56.367 [DEBUG - main] (Connection.java:575) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] disconnecting while in state READY
12:38:56.368 [DEBUG - kudu-nio-1] (Connection.java:239) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] handling channelInactive
12:38:56.368 [DEBUG - kudu-nio-1] (Connection.java:709) [peer a96bc5edb0f947ffb1ca060f8a31d9ff(127.19.102.195:36907)] cleaning up while in state READY due to: connection closed
12:38:56.369 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
12:38:56.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.375880 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 23373
12:38:56.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.384941 23911 mvcc.cc:205] Tried to move back new op lower bound from 7317307128663941120 to 7317307117020725248. Current Snapshot: MvccSnapshot[applied={T|T < 7317307128663941120}]
12:38:56.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.393298 20056 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:56.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.394593 19927 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:56.396 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.396054 20165 ts_manager.cc:194] Re-registered known tserver with Master: ee5818823eb4422d94f6c2c5957eb31d (127.19.102.194:44869)
12:38:56.396 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.396831 23570 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:56.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.397504 20165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.102.194:44857
12:38:56.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.402066 23882 heartbeater.cc:503] Master 127.19.102.252:36537 was elected leader, sending a full tablet report...
12:38:56.501 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.500826 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 23373
12:38:56.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.527464 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 23671
12:38:56.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.535034 23871 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:56.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.639791 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 23671
12:38:56.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.678920 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 23102
12:38:56.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.689494 23287 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:56.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.826185 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 23102
12:38:56.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.857354 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 19882
12:38:56.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.858875 19988 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:56.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:56.986259 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 19882
12:38:57.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:57.017355 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 19998
12:38:57.018 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:57.018687 20099 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:57.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:57.104970 20146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.102.254:39737: connect: Connection refused (error 111)
12:38:57.107 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 12:38:57.106828 20146 consensus_peers.cc:487] T 00000000000000000000000000000000 P 46a51c8bac89417e84212270593211ff -> Peer 989ce9c8a2d4455db0e751ea56e08c67 (127.19.102.254:39737): Couldn't send request to peer 989ce9c8a2d4455db0e751ea56e08c67. Status: Network error: Client connection negotiation failed: client connection to 127.19.102.254:39737: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
12:38:57.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:57.140425 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 19998
12:38:57.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:57.170130 19867 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task3TN6t_/build/asan/bin/kudu pid 20111
12:38:57.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:57.171283 20209 generic_service.cc:196] Checking for leaks (request via RPC)
12:38:57.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 12:38:57.311223 19867 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3TN6t_/build/asan/bin/kudu with pid 20111
12:38:57.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-11T12:38:57Z chronyd exiting

Time: 45.919

OK (1 test)

