JUnit version 4.13.2
.10:32:25.190 [DEBUG - main] (RandomUtils.java:49) Using random seed: 1786444345183
10:32:25.242 [INFO - main] (RetryRule.java:138) Creating RetryStatement with result reporter and retry count of 0 (no retries)
10:32:25.274 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
10:32:25.285 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskr2Ef5i/test-tmp
10:32:25.308 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/asan/bin
10:32:25.311 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskr2Ef5i/java/build/dist-test/../../../build/asan/bin/kudu, test, mini_cluster, --serialization=pb]
10:32:25.514 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
  num_masters: 3
  num_tservers: 3
  enable_kerberos: false
  cluster_root: "/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703"
  hms_mode: NONE
  mini_kdc_options {
  }
  principal: "kudu"
  mini_oidc_options {
  }
}

10:32:25.726 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
10:32:25.727 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}

10:32:25.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:25.731570 20699 mini_oidc.cc:150] Starting JWKS server
10:32:25.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:25.734761 20699 webserver.cc:533] Webserver started at http://127.0.0.1:36825/ using document root <none> and password file <none>
10:32:25.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:25.735071 20699 mini_oidc.cc:186] Starting OIDC Discovery server
10:32:25.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:25.735939 20699 webserver.cc:533] Webserver started at http://127.0.0.1:35383/ using document root <none> and password file <none>
10:32:25.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-11T10:32:25Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
10:32:25.761 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-11T10:32:25Z Disabled control of system clock
10:32:25.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:25.772567 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:25.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:25.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal
10:32:25.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data
10:32:25.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:25.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:25.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:25.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:25.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:25.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:25.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/logs
10:32:25.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data/info.pb
10:32:25.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:25.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:25.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:25.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:25.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:25.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:25.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:25.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:25.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
10:32:25.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:25.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
10:32:25.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
10:32:25.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.254:39167
10:32:25.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.254
10:32:25.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:25.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:25.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:25.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:25.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:25.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
10:32:25.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:25.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
10:32:25.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36825/jwks.json with env {}
10:32:26.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.107513 20720 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:26.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.107837 20720 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:26.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.107946 20720 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:26.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.118454 20720 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
10:32:26.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.118537 20720 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:26.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.118592 20720 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
10:32:26.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.118657 20720 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
10:32:26.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.118737 20720 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
10:32:26.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.118774 20720 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:36825/jwks.json
10:32:26.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.128726 20720 master_runner.cc:387] Master server non-default flags:
10:32:26.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/logs
10:32:26.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:26.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:26.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:26.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:26.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:26.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.254:39167
10:32:26.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
10:32:26.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:26.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
10:32:26.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36825/jwks.json
10:32:26.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:26.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:26.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data/info.pb
10:32:26.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.254
10:32:26.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:26.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:26.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:26.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:26.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:26.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:26.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:26.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
10:32:26.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:26.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:26.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:26.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:26.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:26.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:26.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.130553 20720 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:26.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.132375 20720 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:26.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.141213 20727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:26.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.141414 20728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:26.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.143396 20732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:26.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.143885 20720 server_base.cc:1061] running on GCE node
10:32:26.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.144624 20720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:26.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.145975 20720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:26.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.147245 20720 hybrid_clock.cc:648] HybridClock initialized: now 1786444346147150 us; error 79 us; skew 500 ppm
10:32:26.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.147765 20720 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:26.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.150262 20720 webserver.cc:533] Webserver started at http://127.20.54.254:46823/ using document root <none> and password file <none>
10:32:26.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.151104 20720 fs_manager.cc:362] Metadata directory not provided
10:32:26.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.151247 20720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:26.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.151713 20720 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
10:32:26.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.154217 20720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data/instance:
10:32:26.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "0548b026ce5043ada90d730da43e3cf3"
10:32:26.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:26 on dist-test-slave-s11t"
10:32:26.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.155169 20720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal/instance:
10:32:26.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "0548b026ce5043ada90d730da43e3cf3"
10:32:26.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:26 on dist-test-slave-s11t"
10:32:26.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.160996 20720 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
10:32:26.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.164623 20744 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:26.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.166114 20720 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
10:32:26.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.166409 20720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal
10:32:26.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "0548b026ce5043ada90d730da43e3cf3"
10:32:26.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:26 on dist-test-slave-s11t"
10:32:26.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.166646 20720 fs_report.cc:389] FS layout report
10:32:26.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:26.168 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal
10:32:26.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal
10:32:26.169 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data/data
10:32:26.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:26.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:26.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:26.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:26.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:26.187 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.187531 20720 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:26.188 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.188344 20720 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:26.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.188575 20720 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:26.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.209868 20720 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.254:39167
10:32:26.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.209901 20825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.254:39167 every 8 connection(s)
10:32:26.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.211280 20720 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/data/info.pb
10:32:26.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.215556 20827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
10:32:26.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.220638 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 20720
10:32:26.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.221092 20699 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-0/wal/instance
10:32:26.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.221612 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } has no permanent_uuid. Determining permanent_uuid...
10:32:26.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.223263 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:26.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:26.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal
10:32:26.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data
10:32:26.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:26.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:26.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:26.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:26.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:26.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/logs
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data/info.pb
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:26.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:26.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:26.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:26.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:26.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
10:32:26.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:26.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
10:32:26.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
10:32:26.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.253:43411
10:32:26.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.253
10:32:26.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:26.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:26.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:26.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:26.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:26.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
10:32:26.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:26.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
10:32:26.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36825/jwks.json with env {}
10:32:26.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.242326 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } has no permanent_uuid. Determining permanent_uuid...
10:32:26.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.243539 20749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.253:43411: connect: Connection refused (error 111)
10:32:26.246 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.245679 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.253:43411: Network error: Client connection negotiation failed: client connection to 127.20.54.253:43411: connect: Connection refused (error 111)
10:32:26.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.295068 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } attempt: 1
10:32:26.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.297495 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.253:43411: Network error: Client connection negotiation failed: client connection to 127.20.54.253:43411: connect: Connection refused (error 111)
10:32:26.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.365808 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } attempt: 2
10:32:26.368 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.368309 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.253:43411: Network error: Client connection negotiation failed: client connection to 127.20.54.253:43411: connect: Connection refused (error 111)
10:32:26.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.459631 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } attempt: 3
10:32:26.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.461994 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.253:43411: Network error: Client connection negotiation failed: client connection to 127.20.54.253:43411: connect: Connection refused (error 111)
10:32:26.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.571544 20833 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:26.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.571872 20833 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:26.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.571954 20833 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:26.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.580273 20833 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
10:32:26.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.580364 20833 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:26.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.580420 20833 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
10:32:26.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.580461 20833 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
10:32:26.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.580514 20833 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
10:32:26.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.580549 20833 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:36825/jwks.json
10:32:26.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.590410 20833 master_runner.cc:387] Master server non-default flags:
10:32:26.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/logs
10:32:26.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:26.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:26.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:26.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:26.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:26.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:26.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data
10:32:26.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal
10:32:26.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
10:32:26.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:26.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:26.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:26.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
10:32:26.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.253:43411
10:32:26.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
10:32:26.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:26.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
10:32:26.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36825/jwks.json
10:32:26.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:26.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:26.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data/info.pb
10:32:26.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.253
10:32:26.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:26.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:26.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:26.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:26.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:26.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:26.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:26.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
10:32:26.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:26.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:26.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:26.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:26.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:26.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:26.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.592412 20833 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:26.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.594472 20833 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:26.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.604463 20843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:26.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.605260 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } attempt: 4
10:32:26.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.605482 20842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:26.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.607247 20845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:26.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.607869 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.253:43411: Network error: Client connection negotiation failed: client connection to 127.20.54.253:43411: connect: Connection refused (error 111)
10:32:26.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.608191 20833 server_base.cc:1061] running on GCE node
10:32:26.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.609328 20833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:26.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.611164 20833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:26.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.612481 20833 hybrid_clock.cc:648] HybridClock initialized: now 1786444346612427 us; error 41 us; skew 500 ppm
10:32:26.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.612946 20833 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:26.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.615381 20833 webserver.cc:533] Webserver started at http://127.20.54.253:38059/ using document root <none> and password file <none>
10:32:26.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.616199 20833 fs_manager.cc:362] Metadata directory not provided
10:32:26.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.616292 20833 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:26.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.616672 20833 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
10:32:26.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.619998 20833 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data/instance:
10:32:26.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3c7bfb06b4484444800dd3a5b385bc05"
10:32:26.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:26 on dist-test-slave-s11t"
10:32:26.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.620944 20833 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal/instance:
10:32:26.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3c7bfb06b4484444800dd3a5b385bc05"
10:32:26.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:26 on dist-test-slave-s11t"
10:32:26.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.626263 20833 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
10:32:26.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.630097 20854 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:26.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.631549 20833 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
10:32:26.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.631747 20833 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal
10:32:26.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3c7bfb06b4484444800dd3a5b385bc05"
10:32:26.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:26 on dist-test-slave-s11t"
10:32:26.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.632021 20833 fs_report.cc:389] FS layout report
10:32:26.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:26.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal
10:32:26.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal
10:32:26.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data/data
10:32:26.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:26.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:26.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:26.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:26.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:26.659 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.658651 20833 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:26.660 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.659741 20833 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:26.660 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.660097 20833 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.682193 20833 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.253:43411
10:32:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.682232 20937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.253:43411 every 8 connection(s)
10:32:26.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.684043 20833 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/data/info.pb
10:32:26.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.687373 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 20833
10:32:26.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.687556 20699 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-1/wal/instance
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.688926 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:26.689 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/logs
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data/info.pb
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:26.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
10:32:26.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.252:34659
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.252
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:26.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36825/jwks.json with env {}
10:32:26.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.689311 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
10:32:26.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.698685 20940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } has no permanent_uuid. Determining permanent_uuid...
10:32:26.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.715842 20940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } has no permanent_uuid. Determining permanent_uuid...
10:32:26.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.725646 20940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
10:32:26.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.726989 20858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:26.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.728652 20940 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:26.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.778031 20940 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 1
10:32:26.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.780170 20940 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:26.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.848417 20940 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 2
10:32:26.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.850762 20940 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:26.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.907169 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } attempt: 5
10:32:26.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.913887 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
10:32:26.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.916127 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:26.942 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.942025 20940 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 3
10:32:26.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.944159 20940 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:26.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:26.967370 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 1
10:32:26.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:26.969472 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:27.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.037730 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 2
10:32:27.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.039877 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:27.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.040830 20942 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:27.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.041121 20942 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:27.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.041249 20942 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:27.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.049538 20942 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
10:32:27.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.049619 20942 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:27.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.049675 20942 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
10:32:27.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.049755 20942 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
10:32:27.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.049808 20942 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
10:32:27.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.049842 20942 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:36825/jwks.json
10:32:27.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.059851 20942 master_runner.cc:387] Master server non-default flags:
10:32:27.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/logs
10:32:27.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:27.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:27.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:27.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:27.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.252:34659
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:36825/jwks.json
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:27.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data/info.pb
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.252
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:27.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.061635 20942 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:27.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.064137 20942 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:27.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.073127 20954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:27.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.073277 20957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:27.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.073158 20953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:27.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.073995 20942 server_base.cc:1061] running on GCE node
10:32:27.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.075191 20942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:27.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.076524 20942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:27.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.077769 20942 hybrid_clock.cc:648] HybridClock initialized: now 1786444347077699 us; error 54 us; skew 500 ppm
10:32:27.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.078203 20942 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:27.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.080541 20942 webserver.cc:533] Webserver started at http://127.20.54.252:41157/ using document root <none> and password file <none>
10:32:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.081369 20942 fs_manager.cc:362] Metadata directory not provided
10:32:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.081504 20942 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:27.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.081990 20942 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
10:32:27.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.084420 20942 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data/instance:
10:32:27.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "2807ab88f71b472fb3e13c8ea6774302"
10:32:27.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:27 on dist-test-slave-s11t"
10:32:27.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.085413 20942 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal/instance:
10:32:27.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "2807ab88f71b472fb3e13c8ea6774302"
10:32:27.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:27 on dist-test-slave-s11t"
10:32:27.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.087415 20940 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 4
10:32:27.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.089422 20940 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:27.092 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.091745 20942 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
10:32:27.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.095388 20965 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:27.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.097011 20942 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
10:32:27.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.097190 20942 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal
10:32:27.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "2807ab88f71b472fb3e13c8ea6774302"
10:32:27.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:27 on dist-test-slave-s11t"
10:32:27.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.097474 20942 fs_report.cc:389] FS layout report
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data/data
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:27.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:27.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:27.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.124890 20942 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:27.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.125963 20942 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:27.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.126300 20942 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:27.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.146107 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 3
10:32:27.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.148639 20827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.54.252:34659: Network error: Client connection negotiation failed: client connection to 127.20.54.252:34659: connect: Connection refused (error 111)
10:32:27.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.148813 20942 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.252:34659
10:32:27.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.148847 21052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.252:34659 every 8 connection(s)
10:32:27.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.150396 20942 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/data/info.pb
10:32:27.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.151188 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 20942
10:32:27.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.151399 20699 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/master-2/wal/instance
10:32:27.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.155306 21053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
10:32:27.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.162727 21053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } has no permanent_uuid. Determining permanent_uuid...
10:32:27.179 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.178930 21053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } has no permanent_uuid. Determining permanent_uuid...
10:32:27.186 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.186090 21053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
10:32:27.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.195815 21053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: Bootstrap starting.
10:32:27.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.199065 21053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: Neither blocks nor log segments found. Creating new log.
10:32:27.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.200083 21053 log.cc:826] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: Log is configured to *not* fsync() on all Append() calls
10:32:27.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.202565 21053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: No bootstrap required, opened a new log
10:32:27.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.207923 21053 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.208181 21053 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
10:32:27.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.208319 21053 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807ab88f71b472fb3e13c8ea6774302, State: Initialized, Role: FOLLOWER
10:32:27.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.209647 21053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.210855 21062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.211138 21062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.211911 21053 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: configured and running, proceeding with master startup.
10:32:27.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.227764 21073 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
10:32:27.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.220609 20942 master_runner.cc:187] Error getting master registration for 127.20.54.254:39167: OK, instance_id { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" instance_seqno: 1786444346204124 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
10:32:27.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.228413 21073 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
10:32:27.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.228965 20942 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
10:32:27.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.325911 20827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 4
10:32:27.337 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.337437 21062 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:32:27.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.337711 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: Bootstrap starting.
10:32:27.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.337725 21062 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.339574 21062 leader_election.cc:290] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0548b026ce5043ada90d730da43e3cf3 (127.20.54.254:39167), 3c7bfb06b4484444800dd3a5b385bc05 (127.20.54.253:43411)
10:32:27.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.342764 20827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: Neither blocks nor log segments found. Creating new log.
10:32:27.346 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.346647 20827 log.cc:826] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: Log is configured to *not* fsync() on all Append() calls
10:32:27.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.351006 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: No bootstrap required, opened a new log
10:32:27.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.358840 20827 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.359198 20827 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
10:32:27.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.359340 20827 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0548b026ce5043ada90d730da43e3cf3, State: Initialized, Role: FOLLOWER
10:32:27.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.360092 20827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.361549 21076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.362224 21076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.362771 20827 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: configured and running, proceeding with master startup.
10:32:27.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.364408 20784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2807ab88f71b472fb3e13c8ea6774302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0548b026ce5043ada90d730da43e3cf3" is_pre_election: true
10:32:27.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.365442 20784 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807ab88f71b472fb3e13c8ea6774302 in term 0.
10:32:27.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.366700 20970 leader_election.cc:304] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0548b026ce5043ada90d730da43e3cf3, 2807ab88f71b472fb3e13c8ea6774302; no voters: 
10:32:27.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.367213 21062 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Leader pre-election won for term 1
10:32:27.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.367384 21062 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
10:32:27.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.367491 21062 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 0 FOLLOWER]: Advancing to term 1
10:32:27.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.370878 21062 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.371694 21062 leader_election.cc:290] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [CANDIDATE]: Term 1 election: Requested vote from peers 0548b026ce5043ada90d730da43e3cf3 (127.20.54.254:39167), 3c7bfb06b4484444800dd3a5b385bc05 (127.20.54.253:43411)
10:32:27.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.372596 20784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2807ab88f71b472fb3e13c8ea6774302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0548b026ce5043ada90d730da43e3cf3"
10:32:27.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.373090 20784 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 0 FOLLOWER]: Advancing to term 1
10:32:27.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.378544 20977 tablet.cc:2406] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: Can't schedule compaction. Clean time has not been advanced past its initial value.
10:32:27.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.381488 20784 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807ab88f71b472fb3e13c8ea6774302 in term 1.
10:32:27.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.382612 20970 leader_election.cc:304] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0548b026ce5043ada90d730da43e3cf3, 2807ab88f71b472fb3e13c8ea6774302; no voters: 
10:32:27.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.382992 21062 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 1 FOLLOWER]: Leader election won for term 1
10:32:27.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.388628 20940 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } attempt: 5
10:32:27.395 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.394963 21062 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [term 1 LEADER]: Becoming Leader. State: Replica: 2807ab88f71b472fb3e13c8ea6774302, State: Running, Role: LEADER
10:32:27.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.396173 21062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.413 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.407510 20940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: Bootstrap starting.
10:32:27.413 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.408245 21079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2807ab88f71b472fb3e13c8ea6774302. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.413 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.408651 21079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: This master's current role is: LEADER
10:32:27.413 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.410050 21083 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
10:32:27.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.413977 20940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: Neither blocks nor log segments found. Creating new log.
10:32:27.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.415999 20940 log.cc:826] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: Log is configured to *not* fsync() on all Append() calls
10:32:27.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.417781 21083 catalog_manager.cc:1520] Initializing Kudu cluster ID...
10:32:27.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.423763 20940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: No bootstrap required, opened a new log
10:32:27.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.439976 20940 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.440732 20784 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [term 1 FOLLOWER]: Refusing update from remote peer 2807ab88f71b472fb3e13c8ea6774302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
10:32:27.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.441686 20751 tablet.cc:2406] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
10:32:27.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.442162 21079 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
10:32:27.442 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.441222 20940 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
10:32:27.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.443174 20940 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7bfb06b4484444800dd3a5b385bc05, State: Initialized, Role: FOLLOWER
10:32:27.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.445225 20940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } }
10:32:27.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.456972 21090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.458837 21090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.459204 20892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2807ab88f71b472fb3e13c8ea6774302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7bfb06b4484444800dd3a5b385bc05"
10:32:27.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.459594 20891 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [term 0 FOLLOWER]: Advancing to term 1
10:32:27.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.463750 20891 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [term 1 FOLLOWER]: Refusing update from remote peer 2807ab88f71b472fb3e13c8ea6774302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.459523 20894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2807ab88f71b472fb3e13c8ea6774302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7bfb06b4484444800dd3a5b385bc05" is_pre_election: true
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.464417 20894 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807ab88f71b472fb3e13c8ea6774302 in term 1.
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.459734 20940 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: configured and running, proceeding with master startup.
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.464720 21079 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.466246 21076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2807ab88f71b472fb3e13c8ea6774302. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.466487 21076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.466495 21088 mvcc.cc:204] Tried to move back new op lower bound from 7317276047102922752 to 7317276046940979200. Current Snapshot: MvccSnapshot[applied={T|T < 7317276047102922752}]
10:32:27.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.489858 21091 mvcc.cc:204] Tried to move back new op lower bound from 7317276047102922752 to 7317276046940979200. Current Snapshot: MvccSnapshot[applied={T|T < 7317276047102922752}]
10:32:27.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.502233 21076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.502535 21076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.508378 21079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.508726 21079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: This master's current role is: LEADER
10:32:27.520 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.515100 21083 catalog_manager.cc:1383] Generated new cluster ID: fe320603035e4155852bf0e56b8a9bd5
10:32:27.521 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.515256 21083 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
10:32:27.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.514544 20720 master_runner.cc:187] Error getting master registration for 127.20.54.253:43411: OK, instance_id { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" instance_seqno: 1786444346676177 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
10:32:27.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.525863 20720 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
10:32:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.527544 21090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2807ab88f71b472fb3e13c8ea6774302. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.527798 21090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.527894 21079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.528172 21079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 [sys.catalog]: This master's current role is: LEADER
10:32:27.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.538630 21090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2807ab88f71b472fb3e13c8ea6774302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0548b026ce5043ada90d730da43e3cf3" member_type: VOTER last_known_addr { host: "127.20.54.254" port: 39167 } } peers { permanent_uuid: "3c7bfb06b4484444800dd3a5b385bc05" member_type: VOTER last_known_addr { host: "127.20.54.253" port: 43411 } } peers { permanent_uuid: "2807ab88f71b472fb3e13c8ea6774302" member_type: VOTER last_known_addr { host: "127.20.54.252" port: 34659 } } }
10:32:27.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.538859 21090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05 [sys.catalog]: This master's current role is: FOLLOWER
10:32:27.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.539805 21103 mvcc.cc:204] Tried to move back new op lower bound from 7317276047102922752 to 7317276046940979200. Current Snapshot: MvccSnapshot[applied={T|T < 7317276047102922752}]
10:32:27.544 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.543963 21112 catalog_manager.cc:1295] Loaded cluster ID: fe320603035e4155852bf0e56b8a9bd5
10:32:27.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.548339 21112 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: loading cluster ID for follower catalog manager: success
10:32:27.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.550141 21112 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: acquiring CA information for follower catalog manager: Not found: root CA entry not found
10:32:27.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:27.551184 21112 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
10:32:27.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.576872 21083 catalog_manager.cc:1406] Generated new certificate authority record
10:32:27.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.579336 21083 catalog_manager.cc:1540] Loading token signing keys...
10:32:27.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.604009 21083 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302: Generated new TSK 0
10:32:27.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.605314 21083 catalog_manager.cc:1550] Initializing in-progress tserver states...
10:32:27.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.620910 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:27.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:27.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:27.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:27.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:27.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:27.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:27.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:27.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:27.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:27.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:27.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:27.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:27.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:27.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:27.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:27.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:27.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:27.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:27.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:27.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:27.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:27.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:0
10:32:27.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:27.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:27.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:27.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:27.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:27.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:27.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:27.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:27.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.645406 21132 catalog_manager.cc:1295] Loaded cluster ID: fe320603035e4155852bf0e56b8a9bd5
10:32:27.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.646014 21132 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: loading cluster ID for follower catalog manager: success
10:32:27.651 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.651360 21132 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: acquiring CA information for follower catalog manager: success
10:32:27.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:27.662281 21132 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3c7bfb06b4484444800dd3a5b385bc05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
10:32:28.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.328629 21139 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:28.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.329296 21139 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:28.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.329553 21139 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:28.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.352051 21139 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:28.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.352391 21139 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.193
10:32:28.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.373481 21139 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:28.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:28.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:28.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:28.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:28.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:28.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:28.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:28.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:28.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:28.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:28.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:28.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:0
10:32:28.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:28.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:28.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:28.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:28.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:28.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:28.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:28.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:28.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:28.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:28.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:28.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:28.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:28.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:28.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:28.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.375918 21139 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:28.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.379206 21139 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:28.395 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.395299 21148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:28.400 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.399948 21149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:28.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.409746 21139 server_base.cc:1061] running on GCE node
10:32:28.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:28.410244 21154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:28.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.411181 21139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:28.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.412534 21139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:28.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.414062 21139 hybrid_clock.cc:648] HybridClock initialized: now 1786444348413946 us; error 99 us; skew 500 ppm
10:32:28.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.414476 21139 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:28.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.417603 21139 webserver.cc:533] Webserver started at http://127.20.54.193:36927/ using document root <none> and password file <none>
10:32:28.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.418630 21139 fs_manager.cc:362] Metadata directory not provided
10:32:28.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.418771 21139 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:28.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.419162 21139 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
10:32:28.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.422933 21139 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/instance:
10:32:28.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:28.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:32:28.424 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.424094 21139 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal/instance:
10:32:28.424 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:28.424 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:32:28.432 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.432065 21139 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.003s
10:32:28.437 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.436970 21161 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.439116 21139 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.439311 21139 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.439574 21139 fs_report.cc:389] FS layout report
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:28.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/data
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:28.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:28.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.472304 21139 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:28.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.474149 21139 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:28.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.474629 21139 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:28.477 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.477046 21139 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:28.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.479992 21139 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
10:32:28.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.480130 21139 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
10:32:28.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.480342 21139 ts_tablet_manager.cc:616] Registered 0 tablets
10:32:28.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.480463 21139 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
10:32:28.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.559021 21112 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: acquiring CA information for follower catalog manager: success
10:32:28.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.559116 21139 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.193:46515
10:32:28.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.559206 21321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.193:46515 every 8 connection(s)
10:32:28.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.562237 21139 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:28.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.563768 21112 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0548b026ce5043ada90d730da43e3cf3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
10:32:28.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.566509 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 21139
10:32:28.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.566859 20699 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal/instance
10:32:28.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.576200 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:28.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:28.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:28.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:28.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:28.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:28.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:28.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:28.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:28.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:28.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:0
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:28.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:28.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:28.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:28.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:28.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:28.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.614199 21323 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:28.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.614686 21323 heartbeater.cc:461] Registering TS with master...
10:32:28.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.616617 21323 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:28.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.617236 21324 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:28.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.617435 21324 heartbeater.cc:461] Registering TS with master...
10:32:28.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.617997 21324 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:28.619 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.619102 21322 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:28.619 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.619318 21322 heartbeater.cc:461] Registering TS with master...
10:32:28.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.619861 20767 ts_manager.cc:194] Registered new tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:28.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.620323 21322 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:28.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.621850 20877 ts_manager.cc:194] Registered new tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:28.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.622056 20998 ts_manager.cc:194] Registered new tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:28.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:28.623754 20998 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.193:47049
10:32:29.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.277796 21339 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:29.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.278146 21339 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:29.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.278268 21339 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:29.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.288398 21339 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:29.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.288570 21339 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.194
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.302013 21339 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:29.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:0
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:29.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:29.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:29.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:29.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.303788 21339 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:29.306 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.305939 21339 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:29.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.318984 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:29.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.321455 21348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:29.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:29.321638 21354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:29.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.326432 21339 server_base.cc:1061] running on GCE node
10:32:29.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.327339 21339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:29.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.328943 21339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:29.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.330510 21339 hybrid_clock.cc:648] HybridClock initialized: now 1786444349330401 us; error 49 us; skew 500 ppm
10:32:29.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.330938 21339 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:29.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.334511 21339 webserver.cc:533] Webserver started at http://127.20.54.194:39945/ using document root <none> and password file <none>
10:32:29.335 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.335551 21339 fs_manager.cc:362] Metadata directory not provided
10:32:29.335 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.335688 21339 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:29.336 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.336174 21339 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
10:32:29.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.339469 21339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/instance:
10:32:29.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:29.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:29 on dist-test-slave-s11t"
10:32:29.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.340516 21339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal/instance:
10:32:29.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:29.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:29 on dist-test-slave-s11t"
10:32:29.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.347580 21339 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.001s	sys 0.005s
10:32:29.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.352609 21362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:29.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.354432 21339 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.354641 21339 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:29 on dist-test-slave-s11t"
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.355044 21339 fs_report.cc:389] FS layout report
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/data
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:29.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:29.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.379076 21339 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:29.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.380301 21339 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:29.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.380820 21339 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:29.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.382366 21339 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:29.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.384936 21339 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
10:32:29.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.385062 21339 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
10:32:29.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.385221 21339 ts_tablet_manager.cc:616] Registered 0 tablets
10:32:29.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.385357 21339 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
10:32:29.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.466125 21339 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.194:46199
10:32:29.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.466288 21549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.194:46199 every 8 connection(s)
10:32:29.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.468075 21339 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:29.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.470343 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 21339
10:32:29.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.470695 20699 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal/instance
10:32:29.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.494290 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:29.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:0
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:29.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:29.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.515158 21553 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:29.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.515177 21550 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:29.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.515647 21553 heartbeater.cc:461] Registering TS with master...
10:32:29.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.515758 21550 heartbeater.cc:461] Registering TS with master...
10:32:29.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.518368 21550 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:29.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.520424 20998 ts_manager.cc:194] Registered new tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:29.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.521826 20998 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.194:53621
10:32:29.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.524833 21553 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:29.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.529251 20767 ts_manager.cc:194] Registered new tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:29.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.529345 21555 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:29.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.529577 21555 heartbeater.cc:461] Registering TS with master...
10:32:29.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.530239 21555 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:29.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.531575 20877 ts_manager.cc:194] Registered new tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:29.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:29.626721 21322 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:30.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.069132 21569 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:30.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.069491 21569 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:30.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.069594 21569 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:30.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.081045 21569 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:30.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.081224 21569 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.195
10:32:30.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.094293 21569 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:30.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:32:30.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:30.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:30.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:30.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:0
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:30.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:30.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:30.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.096220 21569 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:30.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.099175 21569 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:30.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.113736 21579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:30.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.113791 21577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:30.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:30.115598 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:30.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.116454 21569 server_base.cc:1061] running on GCE node
10:32:30.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.117146 21569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:30.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.118129 21569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:30.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.119369 21569 hybrid_clock.cc:648] HybridClock initialized: now 1786444350119296 us; error 44 us; skew 500 ppm
10:32:30.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.119692 21569 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:30.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.122833 21569 webserver.cc:533] Webserver started at http://127.20.54.195:42411/ using document root <none> and password file <none>
10:32:30.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.123739 21569 fs_manager.cc:362] Metadata directory not provided
10:32:30.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.123891 21569 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:30.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.124365 21569 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
10:32:30.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.127272 21569 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/instance:
10:32:30.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:30.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:30 on dist-test-slave-s11t"
10:32:30.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.128342 21569 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal/instance:
10:32:30.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:30.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:30 on dist-test-slave-s11t"
10:32:30.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.135296 21569 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.001s
10:32:30.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.139600 21589 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:30.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.141269 21569 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
10:32:30.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.141426 21569 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:30.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:30.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:30 on dist-test-slave-s11t"
10:32:30.141 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.141606 21569 fs_report.cc:389] FS layout report
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/data
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:30.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:30.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.158056 21569 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:30.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.159121 21569 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:30.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.159596 21569 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:30.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.161120 21569 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:30.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.163563 21569 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
10:32:30.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.163679 21569 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
10:32:30.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.163769 21569 ts_tablet_manager.cc:616] Registered 0 tablets
10:32:30.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.163848 21569 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
10:32:30.239 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.238672 21569 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.195:38899
10:32:30.239 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.238956 21782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.195:38899 every 8 connection(s)
10:32:30.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.240653 21569 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.245414 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 21569
10:32:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.245633 20699 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal/instance
10:32:30.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.294072 21784 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:30.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.294075 21785 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:30.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.294744 21784 heartbeater.cc:461] Registering TS with master...
10:32:30.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.294744 21785 heartbeater.cc:461] Registering TS with master...
10:32:30.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.297030 21785 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:30.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.297066 21784 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:30.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.299279 20998 ts_manager.cc:194] Registered new tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:30.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.299410 20767 ts_manager.cc:194] Registered new tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:30.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.300740 20998 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.195:49805
10:32:30.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.301676 21786 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:30.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.301891 21786 heartbeater.cc:461] Registering TS with master...
10:32:30.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.302493 21786 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:30.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.303712 20877 ts_manager.cc:194] Registered new tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:30.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.310853 20699 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
10:32:30.313 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
10:32:30.314 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}

10:32:30.326 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
  masters {
    id {
      type: MASTER
      index: 0
    }
    bound_rpc_address {
      host: "127.20.54.254"
      port: 39167
    }
    bound_http_address {
      host: "127.20.54.254"
      port: 46823
    }
  }
  masters {
    id {
      type: MASTER
      index: 1
    }
    bound_rpc_address {
      host: "127.20.54.253"
      port: 43411
    }
    bound_http_address {
      host: "127.20.54.253"
      port: 38059
    }
  }
  masters {
    id {
      type: MASTER
      index: 2
    }
    bound_rpc_address {
      host: "127.20.54.252"
      port: 34659
    }
    bound_http_address {
      host: "127.20.54.252"
      port: 41157
    }
  }
}

10:32:30.331 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}

10:32:30.334 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
  tservers {
    id {
      type: TSERVER
      index: 0
    }
    bound_rpc_address {
      host: "127.20.54.193"
      port: 46515
    }
    bound_http_address {
      host: "127.20.54.193"
      port: 36927
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 1
    }
    bound_rpc_address {
      host: "127.20.54.194"
      port: 46199
    }
    bound_http_address {
      host: "127.20.54.194"
      port: 39945
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 2
    }
    bound_rpc_address {
      host: "127.20.54.195"
      port: 38899
    }
    bound_http_address {
      host: "127.20.54.195"
      port: 42411
    }
  }
}

10:32:30.334 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
10:32:30.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:30.528487 21550 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:30.535 [DEBUG - main] (CompositeStrategy.java:59) Unable to load modern Subject methods, relying only on legacy methods
java.lang.NoSuchMethodException: javax.security.auth.Subject.current()
	at java.base/java.lang.Class.getDeclaredMethod(Class.java:2677)
	at org.apache.kudu.client.internals.ModernStrategy.<init>(ModernStrategy.java:51)
	at org.apache.kudu.client.internals.CompositeStrategy.<init>(CompositeStrategy.java:52)
	at org.apache.kudu.client.internals.CompositeStrategy.<clinit>(CompositeStrategy.java:41)
	at org.apache.kudu.client.internals.SecurityManagerCompatibility.get(SecurityManagerCompatibility.java:43)
	at org.apache.kudu.client.SecurityContext.setupSubject(SecurityContext.java:155)
	at org.apache.kudu.client.SecurityContext.<init>(SecurityContext.java:138)
	at org.apache.kudu.client.AsyncKuduClient.<init>(AsyncKuduClient.java:431)
	at org.apache.kudu.client.AsyncKuduClient$AsyncKuduClientBuilder.build(AsyncKuduClient.java:3109)
	at org.apache.kudu.test.KuduTestHarness.before(KuduTestHarness.java:161)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttempt(RetryRule.java:225)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:246)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
10:32:30.599 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication 
10:32:30.746 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.20.54.253:43411(127.20.54.253:43411)] connecting to peer
10:32:30.795 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.20.54.252:34659(127.20.54.252:34659)] connecting to peer
10:32:30.797 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.20.54.254:39167(127.20.54.254:39167)] connecting to peer
10:32:30.818 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.20.54.252:34659(127.20.54.252:34659)] Successfully connected to peer
10:32:30.821 [DEBUG - kudu-nio-2] (Connection.java:757) [peer master-127.20.54.254:39167(127.20.54.254:39167)] Successfully connected to peer
10:32:30.830 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.20.54.253:43411(127.20.54.253:43411)] Successfully connected to peer
10:32:30.973 [DEBUG - kudu-nio-2] (Negotiator.java:528) SASL mechanism PLAIN chosen for peer 127.20.54.254
10:32:30.973 [DEBUG - kudu-nio-0] (Negotiator.java:528) SASL mechanism PLAIN chosen for peer 127.20.54.253
10:32:30.973 [DEBUG - kudu-nio-1] (Negotiator.java:528) SASL mechanism PLAIN chosen for peer 127.20.54.252
10:32:31.193 [DEBUG - kudu-nio-1] (Negotiator.java:918) Authenticated connection [id: 0xb94d6bff, L:/127.0.0.1:33774 - R:/127.20.54.252:34659] using SASL/PLAIN
10:32:31.193 [DEBUG - kudu-nio-0] (Negotiator.java:918) Authenticated connection [id: 0xa01b56ba, L:/127.0.0.1:42990 - R:/127.20.54.253:43411] using SASL/PLAIN
10:32:31.193 [DEBUG - kudu-nio-2] (Negotiator.java:918) Authenticated connection [id: 0x8573d548, L:/127.0.0.1:38706 - R:/127.20.54.254:39167] using SASL/PLAIN
10:32:31.243 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet Kudu Master for table 'Kudu Master' with partition [<start>, <end>)
10:32:31.245 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=CreateTable, tablet=null, attempt=1, TimeoutTracker(timeout=50000, elapsed=533), Traces: [0ms] refreshing cache from master, [22ms] Sub RPC ConnectToMaster: sending RPC to server master-127.20.54.253:43411, [80ms] Sub RPC ConnectToMaster: sending RPC to server master-127.20.54.252:34659, [82ms] Sub RPC ConnectToMaster: sending RPC to server master-127.20.54.254:39167, [493ms] Sub RPC ConnectToMaster: received response from server master-127.20.54.254:39167: OK, [493ms] Sub RPC ConnectToMaster: received response from server master-127.20.54.253:43411: OK, [494ms] Sub RPC ConnectToMaster: received response from server master-127.20.54.252:34659: OK, deferred=null) after lookup
10:32:31.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.304297 21784 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:31.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.320037 20998 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33774:
10:32:31.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) name: "org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698"
10:32:31.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) schema {
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "key"
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: true
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
10:32:31.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column1_i"
10:32:31.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
10:32:31.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
10:32:31.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
10:32:31.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
10:32:31.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
10:32:31.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column2_i"
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
10:32:31.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column3_s"
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: STRING
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: true
10:32:31.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     encoding: DICT_ENCODING
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     compression: LZ4
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 4096
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column4_b"
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: BOOL
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
10:32:31.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) partition_schema {
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   range_schema {
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     columns {
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)       name: "key"
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     }
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
10:32:31.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
10:32:31.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:31.322260 20998 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
10:32:31.376 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:894) Opened table 43d3411832bc41ffadb910f54b6a0424
10:32:31.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.409540 21432 tablet_service.cc:1511] Processing CreateTablet for tablet 43717cb820ff40d99e9add5a0d3a05ce (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698 [id=43d3411832bc41ffadb910f54b6a0424]), partition=RANGE (key) PARTITION UNBOUNDED
10:32:31.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.412012 21432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43717cb820ff40d99e9add5a0d3a05ce. 1 dirs total, 0 dirs full, 0 dirs failed
10:32:31.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.416097 21670 tablet_service.cc:1511] Processing CreateTablet for tablet 43717cb820ff40d99e9add5a0d3a05ce (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698 [id=43d3411832bc41ffadb910f54b6a0424]), partition=RANGE (key) PARTITION UNBOUNDED
10:32:31.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.421972 21670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43717cb820ff40d99e9add5a0d3a05ce. 1 dirs total, 0 dirs full, 0 dirs failed
10:32:31.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.421679 21225 tablet_service.cc:1511] Processing CreateTablet for tablet 43717cb820ff40d99e9add5a0d3a05ce (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698 [id=43d3411832bc41ffadb910f54b6a0424]), partition=RANGE (key) PARTITION UNBOUNDED
10:32:31.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.423563 21225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43717cb820ff40d99e9add5a0d3a05ce. 1 dirs total, 0 dirs full, 0 dirs failed
10:32:31.437 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.437022 21839 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap starting.
10:32:31.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.445779 21839 tablet_bootstrap.cc:654] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Neither blocks nor log segments found. Creating new log.
10:32:31.448 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.448592 21839 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Log is configured to *not* fsync() on all Append() calls
10:32:31.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.450757 21840 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap starting.
10:32:31.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.452211 21841 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap starting.
10:32:31.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.460476 21840 tablet_bootstrap.cc:654] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Neither blocks nor log segments found. Creating new log.
10:32:31.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.462570 21840 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Log is configured to *not* fsync() on all Append() calls
10:32:31.464 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.464572 21839 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: No bootstrap required, opened a new log
10:32:31.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.465299 21839 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.000s
10:32:31.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.472396 21840 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: No bootstrap required, opened a new log
10:32:31.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.473102 21840 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.009s
10:32:31.477 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.476647 21839 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.477 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.477644 21839 raft_consensus.cc:385] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
10:32:31.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.478098 21839 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Initialized, Role: FOLLOWER
10:32:31.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.479202 21839 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.485901 21840 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.486347 21841 tablet_bootstrap.cc:654] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Neither blocks nor log segments found. Creating new log.
10:32:31.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.486838 21840 raft_consensus.cc:385] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
10:32:31.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.487308 21840 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Initialized, Role: FOLLOWER
10:32:31.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.488511 21840 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.489 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.488904 21841 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Log is configured to *not* fsync() on all Append() calls
10:32:31.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.495814 21841 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: No bootstrap required, opened a new log
10:32:31.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.496191 21841 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent bootstrapping tablet: real 0.044s	user 0.000s	sys 0.014s
10:32:31.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.498162 21840 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent starting tablet: real 0.024s	user 0.015s	sys 0.000s
10:32:31.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.498620 21839 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.021s
10:32:31.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:31.503535 21566 tablet.cc:2406] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
10:32:31.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.506296 21841 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.508767 21841 raft_consensus.cc:385] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
10:32:31.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.509135 21841 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Initialized, Role: FOLLOWER
10:32:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.510178 21841 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.514982 21841 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.007s
10:32:31.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:31.527127 21787 tablet.cc:2406] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Can't schedule compaction. Clean time has not been advanced past its initial value.
10:32:31.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:31.577760 21326 tablet.cc:2406] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
10:32:31.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.591661 21851 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:32:31.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.592254 21851 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.595561 21851 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:31.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.610131 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:31.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.610731 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:32:31.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.611106 21705 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 0.
10:32:31.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.611739 21469 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 0.
10:32:31.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.612020 21163 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 
10:32:31.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.612573 21851 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Leader pre-election won for term 1
10:32:31.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.612839 21851 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
10:32:31.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.612995 21851 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 0 FOLLOWER]: Advancing to term 1
10:32:31.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.615731 21851 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.616606 21851 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 1 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:31.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.617094 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:31.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.617332 21469 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 0 FOLLOWER]: Advancing to term 1
10:32:31.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.621405 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:31.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.621693 21705 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 0 FOLLOWER]: Advancing to term 1
10:32:31.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.621668 21469 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 1.
10:32:31.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.622334 21162 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 
10:32:31.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.622864 21851 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Leader election won for term 1
10:32:31.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.623832 21851 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 LEADER]: Becoming Leader. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Running, Role: LEADER
10:32:31.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.624665 21851 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:31.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.625973 21705 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 1.
10:32:31.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.632849 20998 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 099ce31a16a94865909f17171a18a1ac (127.20.54.193). New cstate: current_term: 1 leader_uuid: "099ce31a16a94865909f17171a18a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: HEALTHY } } }
10:32:31.660 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:894) Opened table 43d3411832bc41ffadb910f54b6a0424
10:32:31.711 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:31.718 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299992, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:31.720 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=null, attempt=1, sequence_id=-1, TimeoutTracker(timeout=30000, elapsed=47), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [37ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=null) after lookup
10:32:31.730 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:31.733 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] Successfully connected to peer
10:32:31.828 [DEBUG - kudu-nio-3] (Negotiator.java:918) Authenticated connection [id: 0x846e58ba, L:/127.0.0.1:58670 - R:/127.20.54.193:46515] using TOKEN/null
10:32:31.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.839553 21705 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Refusing update from remote peer 099ce31a16a94865909f17171a18a1ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
10:32:31.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.839751 21469 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Refusing update from remote peer 099ce31a16a94865909f17171a18a1ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
10:32:31.841 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.840987 21851 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
10:32:31.841 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.841295 21854 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
10:32:31.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.861855 21857 mvcc.cc:204] Tried to move back new op lower bound from 7317276065121689600 to 7317276064262533120. Current Snapshot: MvccSnapshot[applied={T|T < 7317276065121689600}]
10:32:31.968 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 3 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:31.969 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.193:46515
10:32:31.970 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:31.972 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.971098 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 21139
10:32:31.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:31.982564 21316 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:32.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:32.087684 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 21139
10:32:32.122 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:32.126 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] handling channelInactive
10:32:32.126 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state READY due to: connection closed
10:32:32.126 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: connection closed
10:32:32.135 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:32.135 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:32.135 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=12), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test)) after lookup
10:32:32.136 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:32.139 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.139 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.139 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.156 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:32.156 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:32.157 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=34), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681))) after lookup
10:32:32.158 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:32.160 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.160 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.160 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.196 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:32.196 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:32.197 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=74), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899))) after lookup
10:32:32.198 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:32.200 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.200 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.200 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.216 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:32.216 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:32.217 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=93), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [74ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [77ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [77ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [90ms] refreshing cache from master, [90ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [93ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538))) after lookup
10:32:32.218 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:32.219 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.220 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.220 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.656 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:32.656 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:32.657 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=534), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [74ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [77ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [77ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [90ms] refreshing cache from master, [90ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [93ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [94ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [97ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [97ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [529ms] refreshing cache from master, [530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [532ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426))) after lookup
10:32:32.658 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:32.659 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.659 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:32.659 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:33.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.418841 21885 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:33.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.419088 21885 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:33.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.419639 21886 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:33.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.419860 21886 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:33.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.427541 21885 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:33.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.427635 21886 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:33.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:33.437495 21592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:33.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:33.443043 21366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:33.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.444420 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:33.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:33.444626 21592 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:33.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.444703 21705 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 1.
10:32:33.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.445309 21365 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 
10:32:33.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.445462 21468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:32:33.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.445695 21468 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 1.
10:32:33.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.445825 21886 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
10:32:33.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.445945 21886 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:33.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.446038 21886 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 1 FOLLOWER]: Advancing to term 2
10:32:33.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.446401 21590 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:32:33.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.446838 21885 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Leader pre-election won for term 2
10:32:33.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.446970 21885 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:33.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.447057 21885 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 1 FOLLOWER]: Advancing to term 2
10:32:33.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:33.447136 21366 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:33.449 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.449096 21886 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:33.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.449785 21886 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 2 election: Requested vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:33.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.450063 21885 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:33.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.450343 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:33.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.450551 21705 raft_consensus.cc:2393] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 51655106b4ae4a3895af2bf5ee4529f9 in current term 2: Already voted for candidate 5bf7fa054f57426e99c2f91a6db32080 in this term.
10:32:33.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.450892 21885 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 2 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:33.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.451304 21468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:33.451 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.451566 21468 raft_consensus.cc:2393] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5bf7fa054f57426e99c2f91a6db32080 in current term 2: Already voted for candidate 51655106b4ae4a3895af2bf5ee4529f9 in this term.
10:32:33.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:33.452404 21366 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:33.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.452558 21366 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9; no voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080
10:32:33.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:33.452613 21592 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:33.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.452761 21592 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9
10:32:33.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.452879 21886 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
10:32:33.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:33.453045 21885 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
10:32:33.456 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:33.457 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:33.458 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=1334), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [74ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [77ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [77ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [90ms] refreshing cache from master, [90ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [93ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [94ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [97ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [97ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [529ms] refreshing cache from master, [530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [532ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [534ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [536ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [536ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1329ms] refreshing cache from master, [1330ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1333ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781))) after lookup
10:32:33.458 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:33.459 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:33.460 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:33.460 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:33.577 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:33.578 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:33.579 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=14, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=1455), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [74ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [77ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [77ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [90ms] refreshing cache from master, [90ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [93ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [94ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [97ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [97ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [529ms] refreshing cache from master, [530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [532ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [534ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [536ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [536ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1329ms] refreshing cache from master, [1330ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1333ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [1335ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [1337ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1337ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1450ms] refreshing cache from master, [1450ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1454ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781) -> (continuation of Deferred@1099454349 after retry RPC@358389178), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781) -> (continuation of Deferred@1099454349 after retry RPC@358389178))) after lookup
10:32:33.580 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:33.581 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:33.581 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:33.581 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:34.656 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:34.657 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:34.658 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=16, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=2534), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [74ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [77ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [77ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [90ms] refreshing cache from master, [90ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [93ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [94ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [97ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [97ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [529ms] refreshing cache from master, [530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [532ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [534ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [536ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [536ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1329ms] refreshing cache from master, [1330ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1333ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [1335ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [1337ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1337ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1450ms] refreshing cache from master, [1450ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1454ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [1456ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [1458ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1458ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2530ms] refreshing cache from master, [2530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [2533ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781) -> (continuation of Deferred@1099454349 after retry RPC@358389178) -> (continuation of Deferred@326309958 after retry RPC@1660141271), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781) -> (continuation of Deferred@1099454349 after retry RPC@358389178) -> (continuation of Deferred@326309958 after retry RPC@1660141271))) after lookup
10:32:34.659 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:34.660 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:34.660 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:34.660 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:35.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.463557 21897 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:32:35.464 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.463820 21897 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:35.464 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.464749 21897 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:35.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.465445 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:35.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.465725 21705 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 2.
10:32:35.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.466291 21365 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 
10:32:35.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.466662 21897 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
10:32:35.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.466804 21897 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
10:32:35.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.466924 21897 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 2 FOLLOWER]: Advancing to term 3
10:32:35.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:35.467705 21366 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:35.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.469456 21897 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:35.470 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.470057 21897 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 3 election: Requested vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:35.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.470757 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:35.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.470973 21705 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 2 FOLLOWER]: Advancing to term 3
10:32:35.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:35.471498 21366 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:35.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.473016 21705 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 3.
10:32:35.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.473547 21365 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:32:35.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.473919 21897 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Leader election won for term 3
10:32:35.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.474335 21897 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 LEADER]: Becoming Leader. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Running, Role: LEADER
10:32:35.475 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.474836 21897 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:35.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.478621 20998 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 reported cstate change: term changed from 1 to 3, leader changed from 099ce31a16a94865909f17171a18a1ac (127.20.54.193) to 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194). New cstate: current_term: 3 leader_uuid: "51655106b4ae4a3895af2bf5ee4529f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:32:35.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.914197 21705 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Refusing update from remote peer 51655106b4ae4a3895af2bf5ee4529f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
10:32:35.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:35.914964 21897 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
10:32:35.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:35.922227 21366 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 -> Peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Couldn't send request to peer 099ce31a16a94865909f17171a18a1ac. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:37.117 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:37.117 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:37.118 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=18, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=4994), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [3ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [9ms] refreshing cache from master, [9ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [12ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [13ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [16ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [16ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [29ms] refreshing cache from master, [30ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [33ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [34ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [37ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [37ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] refreshing cache from master, [69ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [72ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [74ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [77ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [77ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [90ms] refreshing cache from master, [90ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [93ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [94ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [97ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [97ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [529ms] refreshing cache from master, [530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [532ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [534ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [536ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [536ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1329ms] refreshing cache from master, [1330ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1333ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [1335ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [1337ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1337ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1450ms] refreshing cache from master, [1450ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1454ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [1456ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [1458ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [1458ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2530ms] refreshing cache from master, [2530ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [2533ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [2535ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2537ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2537ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [4990ms] refreshing cache from master, [4990ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [4993ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@86913403(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@499ee9d4 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781) -> (continuation of Deferred@1099454349 after retry RPC@358389178) -> (continuation of Deferred@326309958 after retry RPC@1660141271) -> (continuation of Deferred@265947527 after retry RPC@1889541037), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6f918c03 -> wakeup thread Time-limited test -> (continuation of Deferred@222767752 after retry RPC@2107048681) -> (continuation of Deferred@1187845922 after retry RPC@1904211899) -> (continuation of Deferred@1589247311 after retry RPC@197015538) -> (continuation of Deferred@491193466 after retry RPC@1022400426) -> (continuation of Deferred@506741302 after retry RPC@650234781) -> (continuation of Deferred@1099454349 after retry RPC@358389178) -> (continuation of Deferred@326309958 after retry RPC@1660141271) -> (continuation of Deferred@265947527 after retry RPC@1889541037))) after lookup
10:32:37.119 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:37.120 [DEBUG - kudu-nio-4] (Connection.java:757) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] Successfully connected to peer
10:32:37.148 [DEBUG - kudu-nio-4] (Negotiator.java:918) Authenticated connection [id: 0x03aff1af, L:/127.0.0.1:34370 - R:/127.20.54.194:46199] using TOKEN/null
10:32:37.180 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.193:46515
10:32:37.182 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.183113 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:37.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:37.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:37.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.667039 21964 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:37.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.667471 21964 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:37.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.667596 21964 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:37.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.677825 21964 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:37.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.678068 21964 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.193
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.690121 21964 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:37.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:37.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:37.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:37.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.692833 21964 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:37.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.695053 21964 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:37.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.706984 21974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:37.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.707980 21976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:37.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:37.708626 21978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:37.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.709065 21964 server_base.cc:1061] running on GCE node
10:32:37.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.709762 21964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:37.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.710773 21964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:37.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.712001 21964 hybrid_clock.cc:648] HybridClock initialized: now 1786444357711949 us; error 46 us; skew 500 ppm
10:32:37.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.712311 21964 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:37.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.714983 21964 webserver.cc:533] Webserver started at http://127.20.54.193:36927/ using document root <none> and password file <none>
10:32:37.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.715903 21964 fs_manager.cc:362] Metadata directory not provided
10:32:37.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.716076 21964 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:37.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.722538 21964 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.000s	sys 0.004s
10:32:37.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.725628 21989 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:37.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.727382 21964 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.727561 21964 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.728300 21964 fs_report.cc:389] FS layout report
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/data
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:37.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:37.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.754372 21964 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:37.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.755250 21964 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:37.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.755527 21964 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:37.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.756614 21964 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:37.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.758859 22000 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:37.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.765843 21964 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:37.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.765969 21964 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s	user 0.001s	sys 0.000s
10:32:37.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.766115 21964 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:37.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.769425 21964 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:37.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.769563 21964 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s	user 0.003s	sys 0.000s
10:32:37.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.770105 22000 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap starting.
10:32:37.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.793695 22000 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Log is configured to *not* fsync() on all Append() calls
10:32:37.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.807156 22000 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:37.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.807826 22000 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap complete.
10:32:37.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.809157 22000 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.019s
10:32:37.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.817453 22000 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:37.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.818404 22000 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Initialized, Role: FOLLOWER
10:32:37.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.819144 22000 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:37.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.820992 22000 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.001s
10:32:37.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.833279 21964 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.193:46515
10:32:37.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.833298 22159 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.193:46515 every 8 connection(s)
10:32:37.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.834666 21964 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:37.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.838394 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 21964
10:32:37.839 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:37.855 [DEBUG - kudu-nio-4] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 6 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:37.859 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.194:46199
10:32:37.860 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

10:32:37.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.860911 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 21339
10:32:37.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.874953 22160 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:37.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.875377 22160 heartbeater.cc:461] Registering TS with master...
10:32:37.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.876514 22160 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:37.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.876770 22162 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:37.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.876912 22162 heartbeater.cc:461] Registering TS with master...
10:32:37.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.877332 22165 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:37.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.877497 22165 heartbeater.cc:461] Registering TS with master...
10:32:37.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.877506 22162 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:37.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.878010 22165 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:37.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.880281 21540 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:37.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.881122 20767 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:37.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.881361 20877 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:37.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.884478 20998 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:37.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.886013 20998 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.193:49663
10:32:37.993 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:37.992518 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 21339
10:32:38.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:38.020167 20970 connection.cc:573] server connection from 127.20.54.194:53621 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
10:32:38.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:38.020391 20857 connection.cc:573] server connection from 127.20.54.194:57521 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
10:32:38.021 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:38.021 [DEBUG - kudu-nio-4] (Connection.java:239) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] handling channelInactive
10:32:38.021 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state READY due to: connection closed
10:32:38.033 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:38.034 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.035 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.035 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.055 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:38.056 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:38.056 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=3, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=34), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [0ms] delaying RPC due to: Illegal state: connection is terminated, [12ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [14ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [14ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [34ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1117516097(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3216e7cc -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@235f8415 -> wakeup thread Time-limited test)) after lookup
10:32:38.056 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:38.057 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.058 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.058 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.076 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:38.077 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:38.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=5, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=56), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [0ms] delaying RPC due to: Illegal state: connection is terminated, [12ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [14ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [14ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [34ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [35ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [37ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [37ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [55ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1117516097(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3216e7cc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@235f8415 -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316))) after lookup
10:32:38.078 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:38.079 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.079 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.079 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.136 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:38.137 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:38.137 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=7, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=116), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [0ms] delaying RPC due to: Illegal state: connection is terminated, [12ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [14ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [14ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [34ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [35ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [37ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [37ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [55ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [56ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [58ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [58ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [112ms] refreshing cache from master, [112ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [115ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1117516097(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3216e7cc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316) -> (continuation of Deferred@1384028477 after retry RPC@1981859906), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@235f8415 -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316) -> (continuation of Deferred@1384028477 after retry RPC@1981859906))) after lookup
10:32:38.138 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:38.139 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.139 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.139 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.276 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:38.276 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:38.277 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=9, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=255), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [0ms] delaying RPC due to: Illegal state: connection is terminated, [12ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [14ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [14ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [34ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [35ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [37ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [37ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [55ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [56ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [58ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [58ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [112ms] refreshing cache from master, [112ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [115ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [117ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [118ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [118ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [251ms] refreshing cache from master, [251ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [254ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1117516097(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3216e7cc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316) -> (continuation of Deferred@1384028477 after retry RPC@1981859906) -> (continuation of Deferred@779978357 after retry RPC@1584347424), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@235f8415 -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316) -> (continuation of Deferred@1384028477 after retry RPC@1981859906) -> (continuation of Deferred@779978357 after retry RPC@1584347424))) after lookup
10:32:38.278 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:38.279 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.279 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.279 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:38.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:38.888441 22160 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:38.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:38.990103 22181 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:32:38.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:38.990499 22181 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:38.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:38.992338 22181 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:38.994 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:38.994336 21991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:39.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:38.999845 21991 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:39.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.000326 21705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:39.001 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.001122 21993 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac; no voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080
10:32:39.001 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.001487 22181 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
10:32:39.180 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.179773 22188 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:39.180 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.179962 22188 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:39.181 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.180822 22188 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:39.182 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.182344 21590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111) [suppressed 1 similar messages]
10:32:39.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.184067 21590 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:39.190 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.189951 22091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac" is_pre_election: true
10:32:39.190 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.190361 22091 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 1.
10:32:39.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.191284 21592 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 51655106b4ae4a3895af2bf5ee4529f9
10:32:39.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.191668 22188 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Leader pre-election won for term 4
10:32:39.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.191807 22188 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:39.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.191933 22188 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 3 FOLLOWER]: Advancing to term 4
10:32:39.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.194859 22188 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:39.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.195529 22188 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 4 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:39.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.196149 22091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:39.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.196357 22091 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 1 FOLLOWER]: Advancing to term 4
10:32:39.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.197078 21590 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:39.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.201671 22091 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 4.
10:32:39.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.202155 21592 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 51655106b4ae4a3895af2bf5ee4529f9
10:32:39.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.202497 22188 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 FOLLOWER]: Leader election won for term 4
10:32:39.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.202991 22188 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 LEADER]: Becoming Leader. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Running, Role: LEADER
10:32:39.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.203667 22188 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:39.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.207424 20998 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 reported cstate change: term changed from 3 to 4, leader changed from 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194) to 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195). New cstate: current_term: 4 leader_uuid: "5bf7fa054f57426e99c2f91a6db32080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:32:39.296 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:39.296 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:39.297 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=11, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=1275), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [0ms] delaying RPC due to: Illegal state: connection is terminated, [12ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [14ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [14ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [31ms] refreshing cache from master, [31ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [34ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [35ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [37ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [37ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [55ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [56ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [58ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [58ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [112ms] refreshing cache from master, [112ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [115ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [117ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [118ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [118ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [251ms] refreshing cache from master, [251ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [254ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [256ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [258ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [258ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [1271ms] refreshing cache from master, [1271ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1274ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1117516097(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3216e7cc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316) -> (continuation of Deferred@1384028477 after retry RPC@1981859906) -> (continuation of Deferred@779978357 after retry RPC@1584347424) -> (continuation of Deferred@2001914196 after retry RPC@626662639), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@235f8415 -> wakeup thread Time-limited test -> (continuation of Deferred@881394140 after retry RPC@1806259316) -> (continuation of Deferred@1384028477 after retry RPC@1981859906) -> (continuation of Deferred@779978357 after retry RPC@1584347424) -> (continuation of Deferred@2001914196 after retry RPC@626662639))) after lookup
10:32:39.298 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:39.299 [DEBUG - kudu-nio-2] (Connection.java:757) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] Successfully connected to peer
10:32:39.319 [DEBUG - kudu-nio-2] (Negotiator.java:918) Authenticated connection [id: 0x268102db, L:/127.0.0.1:44876 - R:/127.20.54.195:38899] using TOKEN/null
10:32:39.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.325721 22091 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Refusing update from remote peer 5bf7fa054f57426e99c2f91a6db32080: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 10. (index mismatch)
10:32:39.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.326200 21590 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 -> Peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Couldn't send request to peer 51655106b4ae4a3895af2bf5ee4529f9. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:39.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.326720 22196 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.000s
10:32:39.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.338189 22212 mvcc.cc:204] Tried to move back new op lower bound from 7317276095788806144 to 7317276095302066176. Current Snapshot: MvccSnapshot[applied={T|T < 7317276095788806144}]
10:32:39.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.342965 22215 mvcc.cc:204] Tried to move back new op lower bound from 7317276095788806144 to 7317276095302066176. Current Snapshot: MvccSnapshot[applied={T|T < 7317276095788806144}]
10:32:39.369 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.194:46199
10:32:39.370 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.371093 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:39.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:46199
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39945
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:39.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:39.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.715148 22232 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:39.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.715480 22232 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:39.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.715557 22232 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.723932 22232 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:39.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.724085 22232 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.194
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.734719 22232 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:46199
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39945
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:39.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:39.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.736799 22232 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:39.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.738644 22232 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:39.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.748939 22244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:39.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.748894 22241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:39.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:39.749100 22239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:39.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.750555 22232 server_base.cc:1061] running on GCE node
10:32:39.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.751197 22232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:39.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.752192 22232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:39.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.753417 22232 hybrid_clock.cc:648] HybridClock initialized: now 1786444359753344 us; error 53 us; skew 500 ppm
10:32:39.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.753762 22232 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:39.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.755967 22232 webserver.cc:533] Webserver started at http://127.20.54.194:39945/ using document root <none> and password file <none>
10:32:39.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.756820 22232 fs_manager.cc:362] Metadata directory not provided
10:32:39.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.756986 22232 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:39.762 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.762584 22232 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
10:32:39.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.765738 22253 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:39.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.767164 22232 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.767340 22232 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:29 on dist-test-slave-s11t"
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.768169 22232 fs_report.cc:389] FS layout report
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/data
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:39.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:39.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:39.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:39.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:39.783 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.783094 22232 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:39.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.784042 22232 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:39.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.784389 22232 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:39.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.785562 22232 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:39.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.787930 22267 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:39.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.794652 22232 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:39.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.794783 22232 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
10:32:39.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.794926 22232 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:39.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.798026 22232 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:39.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.798157 22232 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s	user 0.002s	sys 0.000s
10:32:39.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.798532 22267 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap starting.
10:32:39.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.822840 22267 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Log is configured to *not* fsync() on all Append() calls
10:32:39.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.842854 22267 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=6 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:39.843 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.843647 22267 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap complete.
10:32:39.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.845525 22267 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent bootstrapping tablet: real 0.047s	user 0.023s	sys 0.023s
10:32:39.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.851222 22267 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:39.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.851979 22267 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Initialized, Role: FOLLOWER
10:32:39.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.853039 22267 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:39.855 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.855700 22267 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
10:32:39.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.863327 22232 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.194:46199
10:32:39.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.863377 22433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.194:46199 every 8 connection(s)
10:32:39.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.865033 22232 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:39.873 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.873353 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 22232
10:32:39.874 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:39.904 [DEBUG - kudu-nio-2] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 9 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:39.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.904537 22439 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:39.905 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.195:38899
10:32:39.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.904989 22439 heartbeater.cc:461] Registering TS with master...
10:32:39.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.905931 22438 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:39.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.906038 22439 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:39.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.906095 22438 heartbeater.cc:461] Registering TS with master...
10:32:39.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.906638 22438 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:39.907 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

10:32:39.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.907701 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 21569
10:32:39.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.909193 20877 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:39.910 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.910389 20767 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:39.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.911945 22434 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:39.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.912093 22434 heartbeater.cc:461] Registering TS with master...
10:32:39.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.912619 22434 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:39.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.914505 20997 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:39.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.916177 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.194:60997
10:32:39.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:39.919615 21773 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:40.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:40.024474 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 21569
10:32:40.047 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:40.048 [DEBUG - kudu-nio-2] (Connection.java:239) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] handling channelInactive
10:32:40.048 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state READY due to: connection closed
10:32:40.053 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:40.054 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.054 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.054 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.076 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:40.077 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:40.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=3, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=29), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [0ms] delaying RPC due to: Illegal state: connection is terminated, [5ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [6ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [7ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1868923616(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1aad5687 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1744e8fd -> wakeup thread Time-limited test)) after lookup
10:32:40.077 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:40.078 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.078 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.078 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.116 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:40.116 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:40.117 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=5, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=69), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [0ms] delaying RPC due to: Illegal state: connection is terminated, [5ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [6ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [7ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [30ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [30ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [68ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1868923616(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1aad5687 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1744e8fd -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630))) after lookup
10:32:40.117 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:40.118 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.118 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.118 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.156 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:40.156 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:40.157 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=7, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=108), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [0ms] delaying RPC due to: Illegal state: connection is terminated, [5ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [6ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [7ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [30ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [30ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [68ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [69ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [70ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [71ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [104ms] refreshing cache from master, [105ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [108ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1868923616(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1aad5687 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1744e8fd -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629))) after lookup
10:32:40.157 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:40.158 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.158 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.159 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.175 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:40.175 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:40.176 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=9, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=127), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [0ms] delaying RPC due to: Illegal state: connection is terminated, [5ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [6ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [7ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [30ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [30ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [68ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [69ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [70ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [71ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [104ms] refreshing cache from master, [105ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [108ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [109ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [111ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [111ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [127ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1868923616(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1aad5687 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629) -> (continuation of Deferred@1174638560 after retry RPC@1203392674), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1744e8fd -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629) -> (continuation of Deferred@1174638560 after retry RPC@1203392674))) after lookup
10:32:40.177 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:40.178 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.178 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.178 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:40.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:40.919057 22434 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:41.176 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:41.177 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:41.177 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=11, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=1129), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [0ms] delaying RPC due to: Illegal state: connection is terminated, [5ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [6ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [7ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [30ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [30ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [68ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [69ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [70ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [71ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [104ms] refreshing cache from master, [105ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [108ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [109ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [111ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [111ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [127ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [128ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [130ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [130ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1125ms] refreshing cache from master, [1125ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1128ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1868923616(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1aad5687 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629) -> (continuation of Deferred@1174638560 after retry RPC@1203392674) -> (continuation of Deferred@90229687 after retry RPC@281720057), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1744e8fd -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629) -> (continuation of Deferred@1174638560 after retry RPC@1203392674) -> (continuation of Deferred@90229687 after retry RPC@281720057))) after lookup
10:32:41.178 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:41.179 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:41.179 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:41.180 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:41.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.197420 22460 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:32:41.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.197700 22460 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:41.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.199381 22460 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:41.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:41.200744 22256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:41.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:41.202610 22256 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:41.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.206571 22091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac" is_pre_election: true
10:32:41.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.207386 22259 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9; no voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080
10:32:41.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.207744 22460 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
10:32:41.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.348347 22476 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5bf7fa054f57426e99c2f91a6db32080)
10:32:41.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.348546 22476 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:41.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.349509 22476 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:41.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:41.353686 21993 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:41.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.359477 22362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:32:41.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.359944 22362 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 3.
10:32:41.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.360908 21991 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 5bf7fa054f57426e99c2f91a6db32080
10:32:41.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.361214 22476 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Leader pre-election won for term 5
10:32:41.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.361379 22476 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5bf7fa054f57426e99c2f91a6db32080)
10:32:41.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.361498 22476 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 4 FOLLOWER]: Advancing to term 5
10:32:41.363 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.363616 22476 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:41.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.364401 22476 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 5 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:41.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.364874 22362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:41.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.365101 22362 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 3 FOLLOWER]: Advancing to term 5
10:32:41.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:41.365967 21993 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:41.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.369102 22362 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 5.
10:32:41.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.369619 21991 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 5bf7fa054f57426e99c2f91a6db32080
10:32:41.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.369984 22476 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 5 FOLLOWER]: Leader election won for term 5
10:32:41.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.370463 22476 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 5 LEADER]: Becoming Leader. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Running, Role: LEADER
10:32:41.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.371017 22476 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:41.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.374866 20997 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac reported cstate change: term changed from 4 to 5, leader changed from 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195) to 099ce31a16a94865909f17171a18a1ac (127.20.54.193). New cstate: current_term: 5 leader_uuid: "099ce31a16a94865909f17171a18a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: HEALTHY } } }
10:32:41.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:41.916791 21993 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac -> Peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Couldn't send request to peer 5bf7fa054f57426e99c2f91a6db32080. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:41.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.991050 22362 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Refusing update from remote peer 099ce31a16a94865909f17171a18a1ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 8. Preceding OpId from leader: term: 5 index: 13. (index mismatch)
10:32:41.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:41.992098 22498 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.000s
10:32:43.076 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:43.077 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:43.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=13, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=3029), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [0ms] delaying RPC due to: Illegal state: connection is terminated, [5ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [6ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [7ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [30ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [30ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [68ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [69ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [70ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [71ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [104ms] refreshing cache from master, [105ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [108ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [109ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [111ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [111ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [124ms] refreshing cache from master, [124ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [127ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [128ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [130ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [130ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1125ms] refreshing cache from master, [1125ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1128ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [1130ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1131ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1132ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [3024ms] refreshing cache from master, [3025ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [3028ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1868923616(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1aad5687 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629) -> (continuation of Deferred@1174638560 after retry RPC@1203392674) -> (continuation of Deferred@90229687 after retry RPC@281720057) -> (continuation of Deferred@751688157 after retry RPC@1768283585), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1744e8fd -> wakeup thread Time-limited test -> (continuation of Deferred@1308191588 after retry RPC@2102030630) -> (continuation of Deferred@929989623 after retry RPC@428021629) -> (continuation of Deferred@1174638560 after retry RPC@1203392674) -> (continuation of Deferred@90229687 after retry RPC@281720057) -> (continuation of Deferred@751688157 after retry RPC@1768283585))) after lookup
10:32:43.078 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:43.079 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] Successfully connected to peer
10:32:43.098 [DEBUG - kudu-nio-1] (Negotiator.java:918) Authenticated connection [id: 0x018b2e96, L:/127.0.0.1:50200 - R:/127.20.54.193:46515] using TOKEN/null
10:32:43.124 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.195:38899
10:32:43.124 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.125064 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:43.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:38899
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=42411
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:43.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:43.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.471865 22527 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:43.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.472170 22527 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:43.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.472246 22527 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:43.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.480654 22527 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:43.481 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.480940 22527 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.195
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.490944 22527 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:38899
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=42411
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:43.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:43.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.492585 22527 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:43.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.494530 22527 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:43.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.505136 22541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:43.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.505115 22538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:43.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:43.505038 22539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:43.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.506891 22527 server_base.cc:1061] running on GCE node
10:32:43.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.507449 22527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:43.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.508409 22527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:43.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.509622 22527 hybrid_clock.cc:648] HybridClock initialized: now 1786444363509575 us; error 36 us; skew 500 ppm
10:32:43.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.509920 22527 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:43.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.512243 22527 webserver.cc:533] Webserver started at http://127.20.54.195:42411/ using document root <none> and password file <none>
10:32:43.513 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.513121 22527 fs_manager.cc:362] Metadata directory not provided
10:32:43.513 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.513309 22527 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:43.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.518970 22527 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.002s
10:32:43.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.522018 22554 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:43.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.523607 22527 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.523739 22527 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:30 on dist-test-slave-s11t"
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.524475 22527 fs_report.cc:389] FS layout report
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:43.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/data
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:43.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:43.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.567116 22527 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:43.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.567963 22527 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:43.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.568214 22527 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:43.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.569265 22527 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:43.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.571419 22567 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:43.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.577665 22527 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:43.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.577754 22527 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
10:32:43.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.577854 22527 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:43.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.580852 22527 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:43.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.581161 22527 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s	user 0.003s	sys 0.000s
10:32:43.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.581590 22567 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap starting.
10:32:43.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.612883 22567 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Log is configured to *not* fsync() on all Append() calls
10:32:43.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.639690 22567 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:43.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.640547 22567 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap complete.
10:32:43.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.642786 22567 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent bootstrapping tablet: real 0.062s	user 0.035s	sys 0.023s
10:32:43.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.645129 22527 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.195:38899
10:32:43.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.645433 22745 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.195:38899 every 8 connection(s)
10:32:43.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.647370 22527 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:43.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.656311 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 22527
10:32:43.657 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:43.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.653753 22567 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:43.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.669589 22567 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Initialized, Role: FOLLOWER
10:32:43.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.670874 22567 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:43.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.674191 22567 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent starting tablet: real 0.031s	user 0.016s	sys 0.000s
10:32:43.682 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 12 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:43.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.695624 22750 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:43.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.696007 22750 heartbeater.cc:461] Registering TS with master...
10:32:43.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.697214 22750 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:43.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.700299 20877 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:43.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.704392 22746 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:43.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.704530 22746 heartbeater.cc:461] Registering TS with master...
10:32:43.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.705080 22746 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:43.705 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.193:46515
10:32:43.706 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:43.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.706418 20997 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:43.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.707046 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 21964
10:32:43.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.707929 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.195:55243
10:32:43.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.708518 22748 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:43.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.708670 22748 heartbeater.cc:461] Registering TS with master...
10:32:43.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.709278 22748 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:43.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.711082 20767 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:43.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.711560 22746 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:43.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.728075 22151 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:43.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:43.837198 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 21964
10:32:43.863 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] handling channelInactive
10:32:43.864 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state READY due to: connection closed
10:32:43.864 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:43.865 [DEBUG - Time-limited test] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:43.866 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.867 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.867 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.875 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:43.876 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:43.876 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=11), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test)) after lookup
10:32:43.877 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:43.878 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.878 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.878 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.896 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:43.896 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:43.896 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=31), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [11ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [13ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [13ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [30ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480))) after lookup
10:32:43.897 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:43.898 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.898 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.898 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.936 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:43.936 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:43.937 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=72), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [11ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [13ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [13ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [30ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [31ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [33ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [33ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [67ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [71ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755))) after lookup
10:32:43.937 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:43.938 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.939 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.939 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.976 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:43.976 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:43.977 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=111), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [11ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [13ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [13ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [30ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [31ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [33ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [33ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [67ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [71ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [72ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [74ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [74ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [107ms] refreshing cache from master, [108ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [111ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970))) after lookup
10:32:43.977 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:43.979 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.979 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:43.979 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:44.156 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:44.156 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:44.156 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=291), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [11ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [13ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [13ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [30ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [31ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [33ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [33ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [67ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [71ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [72ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [74ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [74ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [107ms] refreshing cache from master, [108ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [111ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [112ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [114ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [114ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [287ms] refreshing cache from master, [287ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [290ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970) -> (continuation of Deferred@1538208121 after retry RPC@333732692), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970) -> (continuation of Deferred@1538208121 after retry RPC@333732692))) after lookup
10:32:44.157 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:44.158 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:44.158 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:44.159 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:44.216 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:44.216 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:44.217 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=351), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [11ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [13ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [13ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [30ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [31ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [33ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [33ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [67ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [71ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [72ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [74ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [74ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [107ms] refreshing cache from master, [108ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [111ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [112ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [114ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [114ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [287ms] refreshing cache from master, [287ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [290ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [292ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [293ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [294ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [347ms] refreshing cache from master, [347ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [350ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970) -> (continuation of Deferred@1538208121 after retry RPC@333732692) -> (continuation of Deferred@1556180457 after retry RPC@1842258910), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970) -> (continuation of Deferred@1538208121 after retry RPC@333732692) -> (continuation of Deferred@1556180457 after retry RPC@1842258910))) after lookup
10:32:44.217 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:44.219 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:44.219 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:44.219 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:45.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.083360 22768 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:45.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.083559 22768 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:45.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.084245 22768 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:45.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:45.087787 22259 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:45.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.094091 22678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:45.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.094530 22678 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 4.
10:32:45.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.095471 22256 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:32:45.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.095808 22768 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
10:32:45.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.095983 22768 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:45.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.096120 22768 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 5 FOLLOWER]: Advancing to term 6
10:32:45.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.098913 22768 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:45.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.099643 22768 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 6 election: Requested vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:45.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.100119 22678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:45.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.100356 22678 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 4 FOLLOWER]: Advancing to term 6
10:32:45.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:45.101218 22259 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:45.104 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.104635 22678 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 6.
10:32:45.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.105157 22256 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:32:45.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.105482 22768 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 FOLLOWER]: Leader election won for term 6
10:32:45.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.105918 22768 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 LEADER]: Becoming Leader. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Running, Role: LEADER
10:32:45.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.106436 22768 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:45.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.110370 20997 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 reported cstate change: term changed from 5 to 6, leader changed from 099ce31a16a94865909f17171a18a1ac (127.20.54.193) to 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194). New cstate: current_term: 6 leader_uuid: "51655106b4ae4a3895af2bf5ee4529f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:32:45.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.528306 22678 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Refusing update from remote peer 51655106b4ae4a3895af2bf5ee4529f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 12. Preceding OpId from leader: term: 6 index: 17. (index mismatch)
10:32:45.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:45.529218 22768 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 12, Time since last communication: 0.000s
10:32:45.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:45.545095 22259 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 -> Peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Couldn't send request to peer 099ce31a16a94865909f17171a18a1ac. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:46.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:46.429885 22259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111) [suppressed 4 similar messages]
10:32:47.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:47.732195 22259 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 -> Peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Couldn't send request to peer 099ce31a16a94865909f17171a18a1ac. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
10:32:47.996 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:47.996 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:47.997 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=14, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=4132), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [7ms] refreshing cache from master, [7ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [10ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [11ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [13ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [13ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [30ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [31ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [33ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [33ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [67ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [71ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [72ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [74ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [74ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [107ms] refreshing cache from master, [108ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [111ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [112ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [114ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [114ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [287ms] refreshing cache from master, [287ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [290ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [292ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [293ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [294ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [347ms] refreshing cache from master, [347ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [350ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [352ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [354ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [354ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [4127ms] refreshing cache from master, [4128ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [4131ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1964844781(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6d5576a3 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970) -> (continuation of Deferred@1538208121 after retry RPC@333732692) -> (continuation of Deferred@1556180457 after retry RPC@1842258910) -> (continuation of Deferred@2057721670 after retry RPC@1651432351), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@48435065 -> wakeup thread Time-limited test -> (continuation of Deferred@1553686518 after retry RPC@235535480) -> (continuation of Deferred@659659053 after retry RPC@1719290755) -> (continuation of Deferred@1770052314 after retry RPC@1569894970) -> (continuation of Deferred@1538208121 after retry RPC@333732692) -> (continuation of Deferred@1556180457 after retry RPC@1842258910) -> (continuation of Deferred@2057721670 after retry RPC@1651432351))) after lookup
10:32:47.998 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:47.999 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] Successfully connected to peer
10:32:48.019 [DEBUG - kudu-nio-1] (Negotiator.java:918) Authenticated connection [id: 0x67707427, L:/127.0.0.1:60486 - R:/127.20.54.194:46199] using TOKEN/null
10:32:48.050 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.193:46515
10:32:48.050 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:48.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.051653 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:48.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:48.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:48.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:48.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:48.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.463195 22832 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:48.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.463655 22832 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:48.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.463804 22832 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:48.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.478251 22832 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:48.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.478482 22832 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.193
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.495086 22832 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:48.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:48.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:48.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.497442 22832 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:48.500 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.499895 22832 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:48.520 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.519387 22844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:48.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.519724 22843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:48.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:48.521845 22848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:48.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.523990 22832 server_base.cc:1061] running on GCE node
10:32:48.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.525661 22832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:48.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.526957 22832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:48.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.528262 22832 hybrid_clock.cc:648] HybridClock initialized: now 1786444368528133 us; error 67 us; skew 500 ppm
10:32:48.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.528824 22832 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:48.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.532758 22832 webserver.cc:533] Webserver started at http://127.20.54.193:36927/ using document root <none> and password file <none>
10:32:48.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.534068 22832 fs_manager.cc:362] Metadata directory not provided
10:32:48.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.534341 22832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:48.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.542403 22832 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.002s	sys 0.002s
10:32:48.546 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.546517 22858 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:48.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.548496 22832 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
10:32:48.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.548842 22832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:48.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:48.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:32:48.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.550287 22832 fs_report.cc:389] FS layout report
10:32:48.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:48.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/data
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:48.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:48.586 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.586308 22832 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:48.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.587836 22832 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:48.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.588465 22832 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:48.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.590353 22832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:48.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.594491 22870 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:48.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.601824 22832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:48.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.602139 22832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.001s	sys 0.000s
10:32:48.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.602429 22832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:48.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.606644 22832 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:48.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.606905 22832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.005s	user 0.000s	sys 0.003s
10:32:48.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.607210 22870 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap starting.
10:32:48.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.673681 22870 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Log is configured to *not* fsync() on all Append() calls
10:32:48.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.703775 22870 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:48.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.705435 22870 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap complete.
10:32:48.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.707809 22870 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent bootstrapping tablet: real 0.101s	user 0.040s	sys 0.056s
10:32:48.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.708524 22832 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.193:46515
10:32:48.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.708644 23050 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.193:46515 every 8 connection(s)
10:32:48.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.710649 22832 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:48.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.716755 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 22832
10:32:48.717 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:48.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.716148 22870 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:48.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.717310 22870 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Initialized, Role: FOLLOWER
10:32:48.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.718067 22870 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:48.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.750864 22870 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent starting tablet: real 0.043s	user 0.002s	sys 0.015s
10:32:48.754 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 15 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:48.755 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.194:46199
10:32:48.756 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

10:32:48.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.756654 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 22232
10:32:48.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.786863 22429 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:48.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.791872 22981 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 5 FOLLOWER]: Advancing to term 6
10:32:48.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.793540 23055 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:48.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.793984 23055 heartbeater.cc:461] Registering TS with master...
10:32:48.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.795722 23055 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:48.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.796296 23051 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:48.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.796423 23051 heartbeater.cc:461] Registering TS with master...
10:32:48.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.796965 23051 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:48.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.803416 23057 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:48.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.803604 23057 heartbeater.cc:461] Registering TS with master...
10:32:48.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.804191 23057 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:48.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.816107 20998 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:48.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.817274 20877 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:48.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.817821 20998 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.193:38373
10:32:48.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.818496 20767 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:48.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.820457 23051 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:48.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:48.905331 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 22232
10:32:48.930 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] handling channelInactive
10:32:48.930 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:48.930 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state READY due to: connection closed
10:32:48.931 [DEBUG - Time-limited test] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:48.932 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.932 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.933 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.956 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:48.956 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:48.956 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=25), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test)) after lookup
10:32:48.957 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:48.958 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.959 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.959 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.976 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:48.976 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:48.977 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=46), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [26ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [28ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [28ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [41ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [45ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670))) after lookup
10:32:48.978 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:48.979 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.979 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:48.979 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.016 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:49.016 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:49.016 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=85), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [26ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [28ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [28ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [41ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [45ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [46ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [48ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [48ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110))) after lookup
10:32:49.017 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:49.018 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.018 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.018 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.036 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:49.036 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:49.037 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=106), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [26ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [28ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [28ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [41ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [45ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [46ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [48ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [48ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [88ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [102ms] refreshing cache from master, [102ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [105ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349))) after lookup
10:32:49.037 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:49.038 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.038 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.039 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.156 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:49.156 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:49.157 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=226), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [26ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [28ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [28ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [41ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [45ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [46ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [48ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [48ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [88ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [102ms] refreshing cache from master, [102ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [105ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [106ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [108ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [108ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [221ms] refreshing cache from master, [222ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [224ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349) -> (continuation of Deferred@153241853 after retry RPC@883939453), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349) -> (continuation of Deferred@153241853 after retry RPC@883939453))) after lookup
10:32:49.158 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:49.159 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.159 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.159 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.216 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:49.216 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:49.217 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=286), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [26ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [28ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [28ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [41ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [45ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [46ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [48ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [48ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [88ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [102ms] refreshing cache from master, [102ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [105ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [106ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [108ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [108ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [221ms] refreshing cache from master, [222ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [224ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [226ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [228ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [228ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [281ms] refreshing cache from master, [282ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [284ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349) -> (continuation of Deferred@153241853 after retry RPC@883939453) -> (continuation of Deferred@1348831708 after retry RPC@440651677), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349) -> (continuation of Deferred@153241853 after retry RPC@883939453) -> (continuation of Deferred@1348831708 after retry RPC@440651677))) after lookup
10:32:49.218 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:49.219 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.219 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:49.219 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:50.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.055603 23089 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:50.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.055845 23089 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:50.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.057492 23089 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:50.059 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:50.059000 22555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:50.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:50.061116 22555 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:50.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.065048 22981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac" is_pre_election: true
10:32:50.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.066155 22557 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9
10:32:50.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.066578 23089 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
10:32:50.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.302109 23098 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:50.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.302464 23098 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:50.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.304251 23098 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:50.306 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:50.305938 22860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:50.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:50.307894 22860 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:50.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.312640 22678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:50.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.312973 22678 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 6.
10:32:50.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.313727 22862 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 51655106b4ae4a3895af2bf5ee4529f9
10:32:50.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.314213 23098 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 6 FOLLOWER]: Leader pre-election won for term 7
10:32:50.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.314379 23098 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 6 FOLLOWER]: Starting leader election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:50.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.314497 23098 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 6 FOLLOWER]: Advancing to term 7
10:32:50.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.316915 23098 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:50.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.317597 23098 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 7 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:50.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.318197 22678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:50.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.318418 22678 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 6 FOLLOWER]: Advancing to term 7
10:32:50.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:50.319234 22860 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:50.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.320861 22678 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 7.
10:32:50.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.321328 22862 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 51655106b4ae4a3895af2bf5ee4529f9
10:32:50.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.321604 23098 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Leader election won for term 7
10:32:50.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.322093 23098 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 LEADER]: Becoming Leader. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Running, Role: LEADER
10:32:50.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.322609 23098 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 6.20, Last appended by leader: 20, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:50.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.326821 20998 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac reported cstate change: term changed from 6 to 7, leader changed from 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194) to 099ce31a16a94865909f17171a18a1ac (127.20.54.193). New cstate: current_term: 7 leader_uuid: "099ce31a16a94865909f17171a18a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: HEALTHY } } }
10:32:50.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.712428 22678 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Refusing update from remote peer 099ce31a16a94865909f17171a18a1ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 20. Preceding OpId from leader: term: 7 index: 21. (index mismatch)
10:32:50.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:50.713361 23098 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
10:32:50.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:50.721828 22860 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac -> Peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Couldn't send request to peer 51655106b4ae4a3895af2bf5ee4529f9. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:51.316 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:51.316 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:51.317 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=14, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=2385), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [1ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [25ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [26ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [28ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [28ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [41ms] refreshing cache from master, [42ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [45ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [46ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [48ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [48ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [88ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [102ms] refreshing cache from master, [102ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [105ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [106ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [108ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [108ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [221ms] refreshing cache from master, [222ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [224ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [226ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [228ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [228ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [281ms] refreshing cache from master, [282ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [284ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [286ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [288ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [288ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2381ms] refreshing cache from master, [2381ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [2384ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1289034819(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@3b06a485 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349) -> (continuation of Deferred@153241853 after retry RPC@883939453) -> (continuation of Deferred@1348831708 after retry RPC@440651677) -> (continuation of Deferred@1017960437 after retry RPC@941086938), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@6c7f2591 -> wakeup thread Time-limited test -> (continuation of Deferred@524282437 after retry RPC@184096670) -> (continuation of Deferred@389403017 after retry RPC@1429369110) -> (continuation of Deferred@998536626 after retry RPC@2113411349) -> (continuation of Deferred@153241853 after retry RPC@883939453) -> (continuation of Deferred@1348831708 after retry RPC@440651677) -> (continuation of Deferred@1017960437 after retry RPC@941086938))) after lookup
10:32:51.317 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:51.318 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] Successfully connected to peer
10:32:51.335 [DEBUG - kudu-nio-1] (Negotiator.java:918) Authenticated connection [id: 0x7a4bc004, L:/127.0.0.1:60434 - R:/127.20.54.193:46515] using TOKEN/null
10:32:51.363 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.194:46199
10:32:51.364 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.365370 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:51.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:46199
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39945
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:51.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:51.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.718298 23141 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:51.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.718652 23141 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:51.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.718778 23141 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:51.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.729705 23141 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:51.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.729926 23141 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.194
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.740180 23141 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:51.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:46199
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39945
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:51.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:51.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.742059 23141 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:51.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.743957 23141 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:51.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.753947 23155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:51.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.754027 23153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:51.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:51.754037 23158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:51.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.755568 23141 server_base.cc:1061] running on GCE node
10:32:51.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.756114 23141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:51.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.757043 23141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:51.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.758285 23141 hybrid_clock.cc:648] HybridClock initialized: now 1786444371758227 us; error 42 us; skew 500 ppm
10:32:51.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.758637 23141 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:51.761 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.761029 23141 webserver.cc:533] Webserver started at http://127.20.54.194:39945/ using document root <none> and password file <none>
10:32:51.761 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.761808 23141 fs_manager.cc:362] Metadata directory not provided
10:32:51.762 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.761946 23141 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:51.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.767724 23141 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
10:32:51.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.770849 23165 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:51.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.772264 23141 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.772516 23141 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:29 on dist-test-slave-s11t"
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.773360 23141 fs_report.cc:389] FS layout report
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/data
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:51.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:51.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:51.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.794162 23141 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:51.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.795143 23141 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:51.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.795521 23141 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:51.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.796595 23141 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:51.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.798781 23177 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:51.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.804822 23141 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:51.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.804942 23141 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s	user 0.001s	sys 0.000s
10:32:51.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.805073 23141 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.808059 23141 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.808197 23141 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s	user 0.000s	sys 0.000s
10:32:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.808501 23177 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap starting.
10:32:51.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.834399 23177 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Log is configured to *not* fsync() on all Append() calls
10:32:51.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.858711 23141 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.194:46199
10:32:51.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.858832 23350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.194:46199 every 8 connection(s)
10:32:51.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.860496 23141 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:51.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.862928 23177 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=0 applied=20 ignored=0} inserts{seen=15 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:51.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.863615 23177 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap complete.
10:32:51.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.869410 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 23141
10:32:51.870 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:51.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.872186 23177 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent bootstrapping tablet: real 0.064s	user 0.043s	sys 0.012s
10:32:51.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.882196 23177 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:51.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.883193 23177 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Initialized, Role: FOLLOWER
10:32:51.884 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 18 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:51.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.884042 23177 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 6.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:51.884 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.193:46515
10:32:51.885 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:51.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.885627 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 22832
10:32:51.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.891393 23177 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.006s
10:32:51.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.898065 23351 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:51.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.898607 23351 heartbeater.cc:461] Registering TS with master...
10:32:51.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.899973 23351 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.913686 20997 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.915952 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.194:40163
10:32:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.916127 23045 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:51.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.918735 23352 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:51.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.918905 23352 heartbeater.cc:461] Registering TS with master...
10:32:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.919541 23352 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.920056 23351 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.920068 23353 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.920703 23353 heartbeater.cc:461] Registering TS with master...
10:32:51.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.920998 23353 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:51.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.921738 20767 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:51.923 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:51.922935 20877 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:52.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:52.021896 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 22832
10:32:52.048 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:52.048 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] handling channelInactive
10:32:52.048 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state READY due to: connection closed
10:32:52.049 [DEBUG - Time-limited test] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:52.050 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.050 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.051 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:52.077 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:52.078 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=29), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1135382682(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@46db274b -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@427c4460 -> wakeup thread Time-limited test)) after lookup
10:32:52.079 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:52.080 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.080 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.080 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.096 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:52.096 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:52.097 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=48), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [47ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1135382682(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@46db274b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@427c4460 -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086))) after lookup
10:32:52.097 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:52.099 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.099 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.099 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.115 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:52.116 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:52.116 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=67), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [47ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [48ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [50ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [50ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [63ms] refreshing cache from master, [63ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [66ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1135382682(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@46db274b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@427c4460 -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800))) after lookup
10:32:52.117 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:52.118 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.118 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.118 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.175 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:52.176 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:52.176 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=127), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [47ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [48ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [50ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [50ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [63ms] refreshing cache from master, [63ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [66ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [68ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [69ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [123ms] refreshing cache from master, [123ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [126ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1135382682(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@46db274b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800) -> (continuation of Deferred@1413011534 after retry RPC@1847486559), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@427c4460 -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800) -> (continuation of Deferred@1413011534 after retry RPC@1847486559))) after lookup
10:32:52.176 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:52.177 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.178 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.178 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.336 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:52.336 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:52.336 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=288), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [47ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [48ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [50ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [50ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [63ms] refreshing cache from master, [63ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [66ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [68ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [69ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [123ms] refreshing cache from master, [123ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [126ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [127ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [129ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [129ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [284ms] refreshing cache from master, [284ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [286ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1135382682(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@46db274b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800) -> (continuation of Deferred@1413011534 after retry RPC@1847486559) -> (continuation of Deferred@1529485577 after retry RPC@952078050), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@427c4460 -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800) -> (continuation of Deferred@1413011534 after retry RPC@1847486559) -> (continuation of Deferred@1529485577 after retry RPC@952078050))) after lookup
10:32:52.337 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:52.338 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.338 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:52.338 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:53.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.283438 23373 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:53.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.283706 23373 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:53.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.284482 23373 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:53.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:53.287436 22557 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:53.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.292807 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:32:53.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.293242 23281 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 6.
10:32:53.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.294088 22555 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:32:53.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.294391 23373 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Leader pre-election won for term 8
10:32:53.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.294528 23373 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:32:53.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.294641 23373 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 7 FOLLOWER]: Advancing to term 8
10:32:53.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.296641 23373 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:53.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.297561 23373 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 8 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:53.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.298076 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:53.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.298310 23281 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 6 FOLLOWER]: Advancing to term 8
10:32:53.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:53.299160 22557 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:32:53.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.302146 23281 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 8.
10:32:53.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.302574 22555 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:32:53.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.302973 23373 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 FOLLOWER]: Leader election won for term 8
10:32:53.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.303442 23373 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 LEADER]: Becoming Leader. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Running, Role: LEADER
10:32:53.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.303965 23373 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:53.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.307807 20997 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 reported cstate change: term changed from 7 to 8, leader changed from 099ce31a16a94865909f17171a18a1ac (127.20.54.193) to 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195). New cstate: current_term: 8 leader_uuid: "5bf7fa054f57426e99c2f91a6db32080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:32:53.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.704854 23281 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Refusing update from remote peer 5bf7fa054f57426e99c2f91a6db32080: Log matching property violated. Preceding OpId in replica: term: 6 index: 20. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
10:32:53.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:53.705920 23373 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 20, Time since last communication: 0.000s
10:32:53.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:53.721889 22557 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 -> Peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Couldn't send request to peer 099ce31a16a94865909f17171a18a1ac. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:54.076 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:54.076 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:54.077 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=2028), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [29ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [47ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [48ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [50ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [50ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [63ms] refreshing cache from master, [63ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [66ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [68ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [69ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [69ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [123ms] refreshing cache from master, [123ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [126ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [127ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [129ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [129ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [284ms] refreshing cache from master, [284ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [286ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [288ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [289ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [289ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2023ms] refreshing cache from master, [2023ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [2027ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@1135382682(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@46db274b -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800) -> (continuation of Deferred@1413011534 after retry RPC@1847486559) -> (continuation of Deferred@1529485577 after retry RPC@952078050) -> (continuation of Deferred@538509803 after retry RPC@1679115156), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@427c4460 -> wakeup thread Time-limited test -> (continuation of Deferred@1459946010 after retry RPC@2004485086) -> (continuation of Deferred@1416280832 after retry RPC@1932080800) -> (continuation of Deferred@1413011534 after retry RPC@1847486559) -> (continuation of Deferred@1529485577 after retry RPC@952078050) -> (continuation of Deferred@538509803 after retry RPC@1679115156))) after lookup
10:32:54.077 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:54.078 [DEBUG - kudu-nio-0] (Connection.java:757) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] Successfully connected to peer
10:32:54.095 [DEBUG - kudu-nio-0] (Negotiator.java:918) Authenticated connection [id: 0x4c182d60, L:/127.0.0.1:37178 - R:/127.20.54.195:38899] using TOKEN/null
10:32:54.118 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.193:46515
10:32:54.118 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.119122 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:54.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:54.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:54.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:54.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:54.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.459800 23427 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:54.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.460098 23427 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:54.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.460172 23427 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:54.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.468494 23427 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:54.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.468647 23427 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.193
10:32:54.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.478473 23427 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:54.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:32:54.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:54.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:54.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:54.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:54.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:54.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.480368 23427 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:54.482 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.482261 23427 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:54.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.494626 23444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:54.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.494616 23442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:54.495 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.495648 23448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:54.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.496531 23427 server_base.cc:1061] running on GCE node
10:32:54.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.497349 23427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:54.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.498523 23427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:54.499 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.499795 23427 hybrid_clock.cc:648] HybridClock initialized: now 1786444374499740 us; error 37 us; skew 500 ppm
10:32:54.500 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.500185 23427 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:54.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.502614 23427 webserver.cc:533] Webserver started at http://127.20.54.193:36927/ using document root <none> and password file <none>
10:32:54.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.503504 23427 fs_manager.cc:362] Metadata directory not provided
10:32:54.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.503664 23427 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:54.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.509627 23427 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.002s	sys 0.001s
10:32:54.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.512802 23459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:54.514 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.514140 23427 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.514333 23427 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.515154 23427 fs_report.cc:389] FS layout report
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/data
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:54.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:54.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:54.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:54.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:54.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:54.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.536641 23427 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:54.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.537616 23427 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:54.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.537961 23427 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:54.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.539016 23427 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:54.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.541168 23467 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:54.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.546891 23427 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:54.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.547013 23427 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s	user 0.000s	sys 0.000s
10:32:54.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.547153 23427 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:54.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.550202 23427 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:54.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.550333 23427 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s	user 0.002s	sys 0.000s
10:32:54.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.550678 23467 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap starting.
10:32:54.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.578076 23467 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Log is configured to *not* fsync() on all Append() calls
10:32:54.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.598742 23664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.193:46515 every 8 connection(s)
10:32:54.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.598737 23427 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.193:46515
10:32:54.601 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.600819 23427 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:32:54.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.609443 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 23427
10:32:54.610 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:54.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.614833 23467 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=24 ignored=0} inserts{seen=18 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:54.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.615653 23467 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap complete.
10:32:54.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.617686 23467 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent bootstrapping tablet: real 0.067s	user 0.046s	sys 0.018s
10:32:54.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.626622 23467 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:54.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.627856 23467 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Initialized, Role: FOLLOWER
10:32:54.629 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.628660 23467 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:54.647 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 21 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:54.649 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.195:38899
10:32:54.650 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

10:32:54.650 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.650408 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 22527
10:32:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.665383 23666 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:54.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.665817 23666 heartbeater.cc:461] Registering TS with master...
10:32:54.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.666953 23666 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:54.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.670184 20767 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:54.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.671201 23665 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:54.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.671334 23665 heartbeater.cc:461] Registering TS with master...
10:32:54.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.672405 23665 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:54.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.673637 20997 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:54.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.675290 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.193:34175
10:32:54.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.678669 23467 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent starting tablet: real 0.061s	user 0.015s	sys 0.045s
10:32:54.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.679515 23665 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:54.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.683413 23667 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:54.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.683537 23667 heartbeater.cc:461] Registering TS with master...
10:32:54.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.683760 23667 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:54.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.685228 20877 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:54.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.700351 22736 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:54.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:54.805423 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 22527
10:32:54.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.831811 20856 connection.cc:573] server connection from 127.20.54.195:39461 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
10:32:54.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:54.832084 20748 connection.cc:573] server connection from 127.20.54.195:35129 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
10:32:54.833 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:54.833 [DEBUG - kudu-nio-0] (Connection.java:239) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] handling channelInactive
10:32:54.833 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state READY due to: connection closed
10:32:54.833 [DEBUG - Time-limited test] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:54.835 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.835 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.835 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.856 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:54.856 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:54.857 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=23), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [2ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@889801503(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@2fe183a2 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2fb73749 -> wakeup thread Time-limited test)) after lookup
10:32:54.857 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:54.859 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.859 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.859 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.876 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:54.876 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:54.876 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=43), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [2ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [26ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [26ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [39ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [42ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@889801503(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@2fe183a2 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2fb73749 -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568))) after lookup
10:32:54.877 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:54.877 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.878 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.878 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.896 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:54.896 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:54.896 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=63), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [2ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [26ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [26ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [39ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [42ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [43ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [45ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [45ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [62ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@889801503(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@2fe183a2 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2fb73749 -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673))) after lookup
10:32:54.897 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:54.898 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.898 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.898 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.916 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:54.917 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:54.917 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=84), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [2ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [26ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [26ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [39ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [42ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [43ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [45ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [45ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [62ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [63ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [65ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [80ms] refreshing cache from master, [80ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [83ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@889801503(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@2fe183a2 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673) -> (continuation of Deferred@210912852 after retry RPC@398310584), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2fb73749 -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673) -> (continuation of Deferred@210912852 after retry RPC@398310584))) after lookup
10:32:54.918 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:54.919 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.919 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:54.919 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:55.136 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:55.136 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:32:55.137 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=303), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [2ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [26ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [26ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [39ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [42ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [43ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [45ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [45ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [62ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [63ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [65ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [80ms] refreshing cache from master, [80ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [83ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [84ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [86ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [86ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [299ms] refreshing cache from master, [300ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [302ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@889801503(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@2fe183a2 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673) -> (continuation of Deferred@210912852 after retry RPC@398310584) -> (continuation of Deferred@458860283 after retry RPC@1643872596), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2fb73749 -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673) -> (continuation of Deferred@210912852 after retry RPC@398310584) -> (continuation of Deferred@458860283 after retry RPC@1643872596))) after lookup
10:32:55.137 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:32:55.139 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:55.139 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:55.139 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:32:55.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:55.979732 23683 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:32:55.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:55.980010 23683 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:55.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:55.981891 23683 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:55.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:55.983843 23461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:55.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:55.985994 23461 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:55.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:55.989868 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:32:55.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:55.990819 23460 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac; no voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080
10:32:55.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:55.991258 23683 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
10:32:56.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.122485 23688 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5bf7fa054f57426e99c2f91a6db32080)
10:32:56.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.122730 23688 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:56.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.124397 23688 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:56.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.126856 23167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:56.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.128705 23167 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:56.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.133920 23571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac" is_pre_election: true
10:32:56.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.134284 23571 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 7.
10:32:56.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.135188 23169 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 5bf7fa054f57426e99c2f91a6db32080
10:32:56.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.135578 23688 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Leader pre-election won for term 9
10:32:56.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.135725 23688 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5bf7fa054f57426e99c2f91a6db32080)
10:32:56.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.135844 23688 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 8 FOLLOWER]: Advancing to term 9
10:32:56.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.137979 23688 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:56.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.138624 23688 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 9 election: Requested vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:32:56.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.139211 23571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac"
10:32:56.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.139410 23571 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 7 FOLLOWER]: Advancing to term 9
10:32:56.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.140050 23167 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:32:56.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.143338 23571 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 9.
10:32:56.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.143808 23169 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 5bf7fa054f57426e99c2f91a6db32080
10:32:56.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.144145 23688 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 FOLLOWER]: Leader election won for term 9
10:32:56.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.144603 23688 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 LEADER]: Becoming Leader. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Running, Role: LEADER
10:32:56.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.145186 23688 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 8.28, Last appended by leader: 28, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:56.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.149035 20997 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 reported cstate change: term changed from 8 to 9, leader changed from 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195) to 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194). New cstate: current_term: 9 leader_uuid: "51655106b4ae4a3895af2bf5ee4529f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:32:56.356 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:56.356 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:56.357 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=1523), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [2ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [20ms] refreshing cache from master, [20ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [26ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [26ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [39ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [42ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [43ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [45ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [45ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [59ms] refreshing cache from master, [59ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [62ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [63ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [65ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [65ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [80ms] refreshing cache from master, [80ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [83ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [84ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [86ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [86ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [299ms] refreshing cache from master, [300ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [302ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [304ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [306ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [306ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1520ms] refreshing cache from master, [1520ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1523ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@889801503(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@2fe183a2 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673) -> (continuation of Deferred@210912852 after retry RPC@398310584) -> (continuation of Deferred@458860283 after retry RPC@1643872596) -> (continuation of Deferred@221177686 after retry RPC@820131462), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2fb73749 -> wakeup thread Time-limited test -> (continuation of Deferred@45771241 after retry RPC@346098568) -> (continuation of Deferred@1298720991 after retry RPC@1359562673) -> (continuation of Deferred@210912852 after retry RPC@398310584) -> (continuation of Deferred@458860283 after retry RPC@1643872596) -> (continuation of Deferred@221177686 after retry RPC@820131462))) after lookup
10:32:56.358 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:56.358 [DEBUG - kudu-nio-7] (Connection.java:757) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] Successfully connected to peer
10:32:56.385 [DEBUG - kudu-nio-7] (Negotiator.java:918) Authenticated connection [id: 0x4a5342e7, L:/127.0.0.1:48796 - R:/127.20.54.194:46199] using TOKEN/null
10:32:56.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.390661 23571 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Refusing update from remote peer 51655106b4ae4a3895af2bf5ee4529f9: Log matching property violated. Preceding OpId in replica: term: 7 index: 24. Preceding OpId from leader: term: 9 index: 30. (index mismatch)
10:32:56.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.391083 23167 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 -> Peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Couldn't send request to peer 5bf7fa054f57426e99c2f91a6db32080. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:56.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.391628 23693 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 24, Time since last communication: 0.000s
10:32:56.402 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.401962 23716 mvcc.cc:204] Tried to move back new op lower bound from 7317276165687402496 to 7317276164694441984. Current Snapshot: MvccSnapshot[applied={T|T < 7317276165687402496}]
10:32:56.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.402323 23714 mvcc.cc:204] Tried to move back new op lower bound from 7317276165687402496 to 7317276164694441984. Current Snapshot: MvccSnapshot[applied={T|T < 7317276165687402496 or (T in {7317276165687402496})}]
10:32:56.420 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.195:38899
10:32:56.421 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

10:32:56.421 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.421733 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:56.421 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:38899
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=42411
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:56.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:56.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.769657 23732 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:56.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.770097 23732 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:56.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.770228 23732 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:56.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.779179 23732 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:56.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.779405 23732 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.195
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.789683 23732 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:38899
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=42411
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:56.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:56.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:56.792 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.791852 23732 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:56.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.793843 23732 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:56.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.804340 23738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:56.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.804625 23741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:56.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:56.804368 23739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:56.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.805783 23732 server_base.cc:1061] running on GCE node
10:32:56.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.806385 23732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:56.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.807456 23732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:56.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.808744 23732 hybrid_clock.cc:648] HybridClock initialized: now 1786444376808629 us; error 60 us; skew 500 ppm
10:32:56.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.809124 23732 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:56.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.811394 23732 webserver.cc:533] Webserver started at http://127.20.54.195:42411/ using document root <none> and password file <none>
10:32:56.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.812201 23732 fs_manager.cc:362] Metadata directory not provided
10:32:56.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.812347 23732 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:56.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.817993 23732 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
10:32:56.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.821158 23748 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:56.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.822615 23732 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
10:32:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.822818 23732 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:30 on dist-test-slave-s11t"
10:32:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.823586 23732 fs_report.cc:389] FS layout report
10:32:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:56.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/data
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:56.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:56.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.843843 23732 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:56.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.844830 23732 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:56.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.845180 23732 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:56.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.846254 23732 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:56.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.848498 23756 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:56.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.857121 23732 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:56.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.857275 23732 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.001s	sys 0.000s
10:32:56.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.857440 23732 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:56.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.861140 23732 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:56.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.861280 23732 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s	user 0.004s	sys 0.000s
10:32:56.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.862154 23756 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap starting.
10:32:56.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.900595 23756 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Log is configured to *not* fsync() on all Append() calls
10:32:56.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.956902 23756 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=21 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:56.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.958010 23756 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap complete.
10:32:56.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.959695 23756 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent bootstrapping tablet: real 0.098s	user 0.061s	sys 0.017s
10:32:56.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.968518 23756 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:56.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.970170 23756 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Initialized, Role: FOLLOWER
10:32:56.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.971451 23756 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 8.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:56.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:56.976599 23756 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.004s
10:32:57.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.036041 23732 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.195:38899
10:32:57.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.036625 23938 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.195:38899 every 8 connection(s)
10:32:57.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.038072 23732 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:32:57.043 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:57.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.043336 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 23732
10:32:57.078 [DEBUG - kudu-nio-7] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 24 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:57.080 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.194:46199
10:32:57.081 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

10:32:57.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.081966 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 23141
10:32:57.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.109313 23941 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:57.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.109809 23941 heartbeater.cc:461] Registering TS with master...
10:32:57.111 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.111732 23941 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:57.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.109149 23944 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:57.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.113443 23944 heartbeater.cc:461] Registering TS with master...
10:32:57.114 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.114581 23944 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:57.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.117295 20997 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:57.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.117610 20767 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:57.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.118870 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.195:45697
10:32:57.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.119673 23345 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:57.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.122956 23946 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:57.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.123183 23946 heartbeater.cc:461] Registering TS with master...
10:32:57.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.123783 23946 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:57.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.125964 20876 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:57.261 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:57.261219 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 23141
10:32:57.291 [DEBUG - kudu-nio-7] (Connection.java:239) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] handling channelInactive
10:32:57.292 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:57.292 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state READY due to: connection closed
10:32:57.293 [DEBUG - Time-limited test] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:57.294 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.294 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.294 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.316 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:57.316 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:57.316 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=24), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [2ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [20ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@555585560(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6ac4bb -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@5199d7c8 -> wakeup thread Time-limited test)) after lookup
10:32:57.317 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:57.317 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.318 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.318 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.335 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:57.335 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:57.336 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=43), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [2ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [20ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [26ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [26ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [43ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@555585560(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6ac4bb -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@5199d7c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142))) after lookup
10:32:57.336 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:57.337 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.337 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.338 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.376 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:57.377 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:57.377 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=85), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [2ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [20ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [26ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [26ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [43ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [44ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [45ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [46ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@555585560(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6ac4bb -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@5199d7c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927))) after lookup
10:32:57.378 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:57.379 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.379 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.379 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.396 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:57.396 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:57.397 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=104), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [2ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [20ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [26ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [26ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [43ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [44ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [45ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [46ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [87ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [101ms] refreshing cache from master, [101ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [104ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@555585560(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6ac4bb -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927) -> (continuation of Deferred@531363123 after retry RPC@24570791), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@5199d7c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927) -> (continuation of Deferred@531363123 after retry RPC@24570791))) after lookup
10:32:57.397 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:57.398 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.399 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.399 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.716 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:57.717 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:57.717 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=425), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [2ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [20ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [26ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [26ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [43ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [44ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [45ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [46ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [87ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [101ms] refreshing cache from master, [101ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [104ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [105ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [107ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [107ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [420ms] refreshing cache from master, [420ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [424ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@555585560(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6ac4bb -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927) -> (continuation of Deferred@531363123 after retry RPC@24570791) -> (continuation of Deferred@15722283 after retry RPC@913448770), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@5199d7c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927) -> (continuation of Deferred@531363123 after retry RPC@24570791) -> (continuation of Deferred@15722283 after retry RPC@913448770))) after lookup
10:32:57.718 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:32:57.719 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.719 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state CONNECTING due to: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:57.720 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199
10:32:58.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.123124 23941 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:58.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.328301 23968 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:58.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.328516 23968 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:58.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.329417 23968 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:58.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:58.331928 23460 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:58.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.339237 23862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 32 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080" is_pre_election: true
10:32:58.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.339854 23862 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 8.
10:32:58.341 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.340941 23461 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 51655106b4ae4a3895af2bf5ee4529f9
10:32:58.341 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.341245 23968 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Leader pre-election won for term 10
10:32:58.341 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.341389 23968 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 51655106b4ae4a3895af2bf5ee4529f9)
10:32:58.341 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.341482 23968 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 9 FOLLOWER]: Advancing to term 10
10:32:58.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.343573 23968 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:58.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.344318 23968 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 10 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:32:58.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.344936 23862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 32 } } ignore_live_leader: false dest_uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:32:58.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.345149 23862 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 8 FOLLOWER]: Advancing to term 10
10:32:58.346 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:58.345942 23460 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111)
10:32:58.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.349216 23862 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 099ce31a16a94865909f17171a18a1ac in term 10.
10:32:58.349 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.349677 23461 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 5bf7fa054f57426e99c2f91a6db32080; no voters: 51655106b4ae4a3895af2bf5ee4529f9
10:32:58.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.349993 23968 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Leader election won for term 10
10:32:58.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.350464 23968 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 LEADER]: Becoming Leader. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Running, Role: LEADER
10:32:58.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.351204 23968 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 9.32, Last appended by leader: 32, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:58.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.355475 20997 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac reported cstate change: term changed from 9 to 10, leader changed from 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194) to 099ce31a16a94865909f17171a18a1ac (127.20.54.193). New cstate: current_term: 10 leader_uuid: "099ce31a16a94865909f17171a18a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: HEALTHY } } }
10:32:58.855 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:58.856 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:58.856 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=1564), Traces: [0ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [2ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [2ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [20ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [23ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [24ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [26ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [26ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [40ms] refreshing cache from master, [40ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [43ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [44ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [45ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [46ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [84ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [86ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [87ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [87ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [101ms] refreshing cache from master, [101ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [104ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [105ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [107ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [107ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [420ms] refreshing cache from master, [420ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [424ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [426ms] sending RPC to server 51655106b4ae4a3895af2bf5ee4529f9, [427ms] received response from server 51655106b4ae4a3895af2bf5ee4529f9: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [428ms] delaying RPC due to: Network error: Failed to connect to peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199): Connection refused: /127.20.54.194:46199, [1560ms] refreshing cache from master, [1561ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [1563ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@555585560(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6ac4bb -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927) -> (continuation of Deferred@531363123 after retry RPC@24570791) -> (continuation of Deferred@15722283 after retry RPC@913448770) -> (continuation of Deferred@645161885 after retry RPC@14348318), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@5199d7c8 -> wakeup thread Time-limited test -> (continuation of Deferred@1684719488 after retry RPC@1543044142) -> (continuation of Deferred@871605603 after retry RPC@804660927) -> (continuation of Deferred@531363123 after retry RPC@24570791) -> (continuation of Deferred@15722283 after retry RPC@913448770) -> (continuation of Deferred@645161885 after retry RPC@14348318))) after lookup
10:32:58.857 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:58.858 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] Successfully connected to peer
10:32:58.880 [DEBUG - kudu-nio-6] (Negotiator.java:918) Authenticated connection [id: 0x2449a109, L:/127.0.0.1:57550 - R:/127.20.54.193:46515] using TOKEN/null
10:32:58.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.886250 23862 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Refusing update from remote peer 099ce31a16a94865909f17171a18a1ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 28. Preceding OpId from leader: term: 10 index: 34. (index mismatch)
10:32:58.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:58.887250 23460 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac -> Peer 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199): Couldn't send request to peer 51655106b4ae4a3895af2bf5ee4529f9. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.194:46199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:32:58.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.887271 23996 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 28, Time since last communication: 0.000s
10:32:58.915 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.194:46199
10:32:58.915 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:58.916393 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:32:58.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:46199
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39945
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:32:59.266 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.266505 24021 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:32:59.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.266858 24021 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:32:59.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.266968 24021 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:32:59.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.276530 24021 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:32:59.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.276782 24021 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.194
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.286761 24021 tablet_server_runner.cc:78] Tablet server non-default flags:
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/logs
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.194:46199
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.194
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39945
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:32:59.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.194
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:32:59.288 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:32:59.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.288947 24021 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:32:59.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.290805 24021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:32:59.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.300622 24028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:59.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.300601 24029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:59.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:32:59.301331 24031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:32:59.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.301964 24021 server_base.cc:1061] running on GCE node
10:32:59.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.302601 24021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:32:59.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.303619 24021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:32:59.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.304845 24021 hybrid_clock.cc:648] HybridClock initialized: now 1786444379304769 us; error 51 us; skew 500 ppm
10:32:59.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.305246 24021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:32:59.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.307663 24021 webserver.cc:533] Webserver started at http://127.20.54.194:39945/ using document root <none> and password file <none>
10:32:59.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.308488 24021 fs_manager.cc:362] Metadata directory not provided
10:32:59.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.308643 24021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:32:59.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.314666 24021 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
10:32:59.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.317842 24044 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:32:59.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.319195 24021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.319396 24021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:29 on dist-test-slave-s11t"
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.320187 24021 fs_report.cc:389] FS layout report
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/wal
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/data
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:32:59.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:32:59.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:32:59.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:32:59.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:32:59.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:32:59.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.344076 24021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:32:59.346 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.345860 24021 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:32:59.346 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.346467 24021 kserver.cc:163] Server-wide thread pool size limit: 3276
10:32:59.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.348229 24021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:32:59.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.351364 24057 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:32:59.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.359086 24021 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:32:59.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.359297 24021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.001s	sys 0.000s
10:32:59.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.359579 24021 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:32:59.364 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.364584 24021 ts_tablet_manager.cc:616] Registered 1 tablets
10:32:59.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.364953 24021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.005s	user 0.002s	sys 0.000s
10:32:59.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.365029 24057 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap starting.
10:32:59.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.437939 24057 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Log is configured to *not* fsync() on all Append() calls
10:32:59.501 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.500984 24057 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=24 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:32:59.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.502463 24057 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Bootstrap complete.
10:32:59.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.503518 24021 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.194:46199
10:32:59.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.503896 24223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.194:46199 every 8 connection(s)
10:32:59.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.504437 24057 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent bootstrapping tablet: real 0.140s	user 0.070s	sys 0.063s
10:32:59.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.505381 24021 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-1/data/info.pb
10:32:59.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.510067 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 24021
10:32:59.510 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:59.514 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.513537 24057 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:59.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.514902 24057 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Initialized, Role: FOLLOWER
10:32:59.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.516086 24057 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 9.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:32:59.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.554147 24057 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9: Time spent starting tablet: real 0.050s	user 0.016s	sys 0.031s
10:32:59.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.556932 24225 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:32:59.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.557351 24225 heartbeater.cc:461] Registering TS with master...
10:32:59.558 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 27 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:32:59.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.558568 24225 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:32:59.559 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.193:46515
10:32:59.559 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:32:59.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.560609 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 23427
10:32:59.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.563089 20997 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:59.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.563617 24226 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:32:59.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.563745 24226 heartbeater.cc:461] Registering TS with master...
10:32:59.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.564324 24226 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:32:59.565 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.564993 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.194:33317
10:32:59.566 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.565799 20767 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:59.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.567287 24227 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:32:59.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.567443 24227 heartbeater.cc:461] Registering TS with master...
10:32:59.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.567907 24227 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:32:59.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.569321 20876 ts_manager.cc:194] Re-registered known tserver with Master: 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199)
10:32:59.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.570497 24225 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:32:59.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.582406 23656 generic_service.cc:196] Checking for leaks (request via RPC)
10:32:59.684 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:32:59.684514 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 23427
10:32:59.706 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:32:59.706 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] handling channelInactive
10:32:59.706 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state READY due to: connection closed
10:32:59.707 [DEBUG - Time-limited test] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:59.708 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.708 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.709 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.716 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:59.717 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:59.717 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=10), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test)) after lookup
10:32:59.718 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:59.719 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.719 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.719 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.736 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:59.736 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:59.736 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=29), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [10ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [12ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [12ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852))) after lookup
10:32:59.737 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:59.738 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.738 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.738 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.776 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:59.776 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:59.777 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=70), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [10ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [12ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [12ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [30ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [69ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536))) after lookup
10:32:59.777 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:59.778 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.779 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.779 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.836 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:32:59.836 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:32:59.836 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=129), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [10ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [12ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [12ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [30ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [69ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [70ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [72ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [72ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [125ms] refreshing cache from master, [125ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [128ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625))) after lookup
10:32:59.837 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:32:59.838 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.838 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:32:59.838 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:00.236 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:00.238 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:33:00.238 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=531), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [10ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [12ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [12ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [30ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [69ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [70ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [72ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [72ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [125ms] refreshing cache from master, [125ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [128ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [130ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [131ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [131ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [526ms] refreshing cache from master, [526ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [529ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625) -> (continuation of Deferred@2022927713 after retry RPC@764656904), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625) -> (continuation of Deferred@2022927713 after retry RPC@764656904))) after lookup
10:33:00.239 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:33:00.240 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:00.240 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:00.240 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:00.576 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:00.576 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)[L],51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:33:00.577 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=870), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [10ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [12ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [12ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [30ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [69ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [70ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [72ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [72ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [125ms] refreshing cache from master, [125ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [128ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [130ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [131ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [131ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [526ms] refreshing cache from master, [526ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [529ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [532ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [533ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [534ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [866ms] refreshing cache from master, [866ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [869ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625) -> (continuation of Deferred@2022927713 after retry RPC@764656904) -> (continuation of Deferred@978601592 after retry RPC@169854919), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625) -> (continuation of Deferred@2022927713 after retry RPC@764656904) -> (continuation of Deferred@978601592 after retry RPC@169854919))) after lookup
10:33:00.577 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] connecting to peer
10:33:00.578 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:00.578 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] cleaning up while in state CONNECTING due to: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:00.578 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515
10:33:01.039 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.039368 24250 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:33:01.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.039626 24250 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:01.042 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.041918 24250 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:01.043 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:01.043478 23752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:33:01.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:01.045357 23752 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:33:01.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.051856 24159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 11 candidate_status { last_received { term: 10 index: 36 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:33:01.052 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.052305 24159 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 9.
10:33:01.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.053339 23749 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:33:01.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.053709 24250 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Leader pre-election won for term 11
10:33:01.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.053844 24250 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 099ce31a16a94865909f17171a18a1ac)
10:33:01.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.053957 24250 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 10 FOLLOWER]: Advancing to term 11
10:33:01.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.056012 24250 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:01.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.056703 24250 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 11 election: Requested vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:01.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.057130 24159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "5bf7fa054f57426e99c2f91a6db32080" candidate_term: 11 candidate_status { last_received { term: 10 index: 36 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9"
10:33:01.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.057346 24159 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 9 FOLLOWER]: Advancing to term 11
10:33:01.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:01.058439 23752 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111)
10:33:01.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.061475 24159 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf7fa054f57426e99c2f91a6db32080 in term 11.
10:33:01.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.061947 23749 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080; no voters: 099ce31a16a94865909f17171a18a1ac
10:33:01.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.062278 24250 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 11 FOLLOWER]: Leader election won for term 11
10:33:01.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.062752 24250 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 11 LEADER]: Becoming Leader. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Running, Role: LEADER
10:33:01.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.063324 24250 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 10.36, Last appended by leader: 36, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:01.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.067417 20998 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 reported cstate change: term changed from 10 to 11, leader changed from 099ce31a16a94865909f17171a18a1ac (127.20.54.193) to 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195). New cstate: current_term: 11 leader_uuid: "5bf7fa054f57426e99c2f91a6db32080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:33:01.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:01.460870 23752 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 -> Peer 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515): Couldn't send request to peer 099ce31a16a94865909f17171a18a1ac. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.193:46515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:33:01.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.462708 24159 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Refusing update from remote peer 5bf7fa054f57426e99c2f91a6db32080: Log matching property violated. Preceding OpId in replica: term: 9 index: 32. Preceding OpId from leader: term: 11 index: 37. (index mismatch)
10:33:01.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:01.463599 24250 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 32, Time since last communication: 0.000s
10:33:03.055 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:03.056 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:03.056 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=14, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=3349), Traces: [0ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [2ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [2ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [6ms] refreshing cache from master, [6ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [9ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [10ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [12ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [12ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [25ms] refreshing cache from master, [25ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [28ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [30ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [31ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [31ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [65ms] refreshing cache from master, [65ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [69ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [70ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [72ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [72ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [125ms] refreshing cache from master, [125ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [128ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [130ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [131ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [131ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [526ms] refreshing cache from master, [526ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [529ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [532ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [533ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [534ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [866ms] refreshing cache from master, [866ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [869ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [870ms] sending RPC to server 099ce31a16a94865909f17171a18a1ac, [871ms] received response from server 099ce31a16a94865909f17171a18a1ac: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [871ms] delaying RPC due to: Network error: Failed to connect to peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515): Connection refused: /127.20.54.193:46515, [3345ms] refreshing cache from master, [3345ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [3348ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@948595996(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@6f872013 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625) -> (continuation of Deferred@2022927713 after retry RPC@764656904) -> (continuation of Deferred@978601592 after retry RPC@169854919) -> (continuation of Deferred@33644554 after retry RPC@1993649778), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@7177a683 -> wakeup thread Time-limited test -> (continuation of Deferred@1300555369 after retry RPC@17498852) -> (continuation of Deferred@1535262177 after retry RPC@257909536) -> (continuation of Deferred@896573708 after retry RPC@1103270625) -> (continuation of Deferred@2022927713 after retry RPC@764656904) -> (continuation of Deferred@978601592 after retry RPC@169854919) -> (continuation of Deferred@33644554 after retry RPC@1993649778))) after lookup
10:33:03.057 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:03.058 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] Successfully connected to peer
10:33:03.075 [DEBUG - kudu-nio-6] (Negotiator.java:918) Authenticated connection [id: 0x4e04e987, L:/127.0.0.1:54712 - R:/127.20.54.195:38899] using TOKEN/null
10:33:03.098 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.193:46515
10:33:03.099 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.099901 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:33:03.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:33:03.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:33:03.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.447003 24325 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:33:03.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.447350 24325 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:33:03.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.447464 24325 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:33:03.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.455870 24325 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:33:03.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.456089 24325 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.193
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.467044 24325 tablet_server_runner.cc:78] Tablet server non-default flags:
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/logs
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.193:46515
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:33:03.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.193
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36927
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.193
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:33:03.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:33:03.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.469197 24325 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:33:03.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.471061 24325 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:33:03.482 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.481305 24332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:33:03.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.485116 24336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:33:03.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:03.485364 24333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:33:03.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.485956 24325 server_base.cc:1061] running on GCE node
10:33:03.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.486636 24325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:33:03.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.487672 24325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:33:03.489 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.488909 24325 hybrid_clock.cc:648] HybridClock initialized: now 1786444383488829 us; error 52 us; skew 500 ppm
10:33:03.489 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.489277 24325 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:33:03.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.491499 24325 webserver.cc:533] Webserver started at http://127.20.54.193:36927/ using document root <none> and password file <none>
10:33:03.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.492282 24325 fs_manager.cc:362] Metadata directory not provided
10:33:03.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.492424 24325 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:33:03.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.498111 24325 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.000s
10:33:03.501 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.501291 24344 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:33:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.502943 24325 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.503108 24325 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "099ce31a16a94865909f17171a18a1ac"
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:28 on dist-test-slave-s11t"
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.504073 24325 fs_report.cc:389] FS layout report
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/wal
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/data
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:33:03.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:33:03.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.554826 24325 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:33:03.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.556236 24325 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:33:03.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.556774 24325 kserver.cc:163] Server-wide thread pool size limit: 3276
10:33:03.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.558130 24325 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:33:03.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.561383 24356 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:33:03.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.568823 24325 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:33:03.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.569010 24325 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s	user 0.000s	sys 0.001s
10:33:03.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.569286 24325 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:33:03.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.573871 24325 ts_tablet_manager.cc:616] Registered 1 tablets
10:33:03.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.574131 24325 ts_tablet_manager.cc:595] Time spent register tablets: real 0.005s	user 0.002s	sys 0.002s
10:33:03.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.574487 24356 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap starting.
10:33:03.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.639137 24356 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Log is configured to *not* fsync() on all Append() calls
10:33:03.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.655260 24325 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.193:46515
10:33:03.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.655301 24529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.193:46515 every 8 connection(s)
10:33:03.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.657102 24325 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-0/data/info.pb
10:33:03.662 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.662145 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 24325
10:33:03.662 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:33:03.688 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 30 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:33:03.689 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.20.54.195:38899
10:33:03.689 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

10:33:03.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.690486 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 23732
10:33:03.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.704859 24530 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:33:03.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.705322 24530 heartbeater.cc:461] Registering TS with master...
10:33:03.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.706449 24530 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:33:03.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.709824 20997 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:03.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.711426 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.193:48391
10:33:03.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.730635 24531 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:33:03.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.730849 24531 heartbeater.cc:461] Registering TS with master...
10:33:03.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.731298 24531 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:33:03.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.732930 20767 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:03.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.735267 24533 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:33:03.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.735428 24533 heartbeater.cc:461] Registering TS with master...
10:33:03.735 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.735767 24533 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:33:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.737353 20877 ts_manager.cc:194] Re-registered known tserver with Master: 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:03.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.739406 23932 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:03.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.740926 24356 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=27 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:33:03.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.741689 24356 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Bootstrap complete.
10:33:03.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.743438 24356 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent bootstrapping tablet: real 0.169s	user 0.083s	sys 0.032s
10:33:03.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.748608 24356 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:03.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.749372 24356 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099ce31a16a94865909f17171a18a1ac, State: Initialized, Role: FOLLOWER
10:33:03.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.750175 24356 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 10.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:03.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.751828 24530 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:33:03.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.752156 24356 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
10:33:03.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:03.847371 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 23732
10:33:03.875 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] handling channelInactive
10:33:03.875 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state READY due to: connection closed
10:33:03.875 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:33:03.876 [DEBUG - Time-limited test] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:03.877 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.877 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.877 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.896 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:03.897 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:03.897 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=2, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=21), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test)) after lookup
10:33:03.898 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:03.899 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.899 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.899 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.915 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:03.915 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:03.916 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=4, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=40), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [21ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [23ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [23ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [36ms] refreshing cache from master, [36ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [39ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815))) after lookup
10:33:03.916 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:03.917 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.917 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.918 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.955 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:03.955 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:03.956 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=6, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=80), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [21ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [23ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [23ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [36ms] refreshing cache from master, [36ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [39ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [40ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [42ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [42ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [79ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120))) after lookup
10:33:03.956 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:03.957 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.957 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:03.958 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.035 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:04.036 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:04.036 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=8, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=160), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [21ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [23ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [23ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [36ms] refreshing cache from master, [36ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [39ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [40ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [42ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [42ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [79ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [80ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [81ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [82ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [156ms] refreshing cache from master, [156ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [159ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842))) after lookup
10:33:04.036 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:04.037 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.038 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.038 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.056 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:04.056 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:04.057 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=10, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=181), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [21ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [23ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [23ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [36ms] refreshing cache from master, [36ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [39ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [40ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [42ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [42ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [79ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [80ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [81ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [82ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [156ms] refreshing cache from master, [156ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [159ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [160ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [162ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [162ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [179ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842) -> (continuation of Deferred@596611196 after retry RPC@985784364), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842) -> (continuation of Deferred@596611196 after retry RPC@985784364))) after lookup
10:33:04.057 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:04.058 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.058 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.058 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.516 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:04.516 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199),5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)[L]]}]
10:33:04.516 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=12, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=640), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [21ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [23ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [23ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [36ms] refreshing cache from master, [36ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [39ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [40ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [42ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [42ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [79ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [80ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [81ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [82ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [156ms] refreshing cache from master, [156ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [159ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [160ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [162ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [162ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [179ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [181ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [182ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [182ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [636ms] refreshing cache from master, [636ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [639ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842) -> (continuation of Deferred@596611196 after retry RPC@985784364) -> (continuation of Deferred@1698774984 after retry RPC@1345614796), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842) -> (continuation of Deferred@596611196 after retry RPC@985784364) -> (continuation of Deferred@1698774984 after retry RPC@1345614796))) after lookup
10:33:04.517 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] connecting to peer
10:33:04.518 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.518 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] cleaning up while in state CONNECTING due to: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:04.518 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899) for tablet 43717cb820ff40d99e9add5a0d3a05ce: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899
10:33:05.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.069000 24554 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
10:33:05.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.069283 24554 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:05.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.071157 24554 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194:46199), 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:33:05.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:05.073318 24348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:33:05.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:05.075253 24348 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:33:05.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.076668 24563 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5bf7fa054f57426e99c2f91a6db32080)
10:33:05.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.076902 24563 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:05.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.078358 24563 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:05.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.080581 24159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "099ce31a16a94865909f17171a18a1ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 36 } } ignore_live_leader: false dest_uuid: "51655106b4ae4a3895af2bf5ee4529f9" is_pre_election: true
10:33:05.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:05.080916 24048 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:33:05.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.081035 24159 raft_consensus.cc:2393] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 099ce31a16a94865909f17171a18a1ac in current term 11: Already voted for candidate 5bf7fa054f57426e99c2f91a6db32080 in this term.
10:33:05.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.082309 24345 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac; no voters: 51655106b4ae4a3895af2bf5ee4529f9, 5bf7fa054f57426e99c2f91a6db32080
10:33:05.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:05.082690 24048 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:33:05.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.083081 24554 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 10 FOLLOWER]: Advancing to term 11
10:33:05.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.087884 24554 raft_consensus.cc:2749] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
10:33:05.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.088905 24459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 12 candidate_status { last_received { term: 11 index: 40 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac" is_pre_election: true
10:33:05.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.089263 24459 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 11.
10:33:05.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.090201 24050 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 5bf7fa054f57426e99c2f91a6db32080
10:33:05.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.090617 24563 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Leader pre-election won for term 12
10:33:05.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.090770 24563 raft_consensus.cc:493] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5bf7fa054f57426e99c2f91a6db32080)
10:33:05.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.090899 24563 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 11 FOLLOWER]: Advancing to term 12
10:33:05.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.092883 24563 raft_consensus.cc:515] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:05.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.093519 24563 leader_election.cc:290] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 12 election: Requested vote from peers 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899), 099ce31a16a94865909f17171a18a1ac (127.20.54.193:46515)
10:33:05.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.094164 24459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43717cb820ff40d99e9add5a0d3a05ce" candidate_uuid: "51655106b4ae4a3895af2bf5ee4529f9" candidate_term: 12 candidate_status { last_received { term: 11 index: 40 } } ignore_live_leader: false dest_uuid: "099ce31a16a94865909f17171a18a1ac"
10:33:05.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.094350 24459 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 11 FOLLOWER]: Advancing to term 12
10:33:05.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:05.095062 24048 leader_election.cc:336] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111)
10:33:05.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.096562 24459 raft_consensus.cc:2468] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51655106b4ae4a3895af2bf5ee4529f9 in term 12.
10:33:05.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.097070 24050 leader_election.cc:304] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 099ce31a16a94865909f17171a18a1ac, 51655106b4ae4a3895af2bf5ee4529f9; no voters: 5bf7fa054f57426e99c2f91a6db32080
10:33:05.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.097463 24563 raft_consensus.cc:2804] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 12 FOLLOWER]: Leader election won for term 12
10:33:05.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.098060 24563 raft_consensus.cc:697] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [term 12 LEADER]: Becoming Leader. State: Replica: 51655106b4ae4a3895af2bf5ee4529f9, State: Running, Role: LEADER
10:33:05.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.098594 24563 consensus_queue.cc:237] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 11.40, Last appended by leader: 40, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:05.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.102495 20997 catalog_manager.cc:5719] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 reported cstate change: term changed from 11 to 12, leader changed from 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195) to 51655106b4ae4a3895af2bf5ee4529f9 (127.20.54.194). New cstate: current_term: 12 leader_uuid: "51655106b4ae4a3895af2bf5ee4529f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } health_report { overall_health: UNKNOWN } } }
10:33:05.652 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.652163 24459 raft_consensus.cc:1275] T 43717cb820ff40d99e9add5a0d3a05ce P 099ce31a16a94865909f17171a18a1ac [term 12 FOLLOWER]: Refusing update from remote peer 51655106b4ae4a3895af2bf5ee4529f9: Log matching property violated. Preceding OpId in replica: term: 10 index: 36. Preceding OpId from leader: term: 12 index: 41. (index mismatch)
10:33:05.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:05.653331 24590 consensus_queue.cc:1048] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 36, Time since last communication: 0.000s
10:33:05.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:05.671983 24048 consensus_peers.cc:597] T 43717cb820ff40d99e9add5a0d3a05ce P 51655106b4ae4a3895af2bf5ee4529f9 -> Peer 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899): Couldn't send request to peer 5bf7fa054f57426e99c2f91a6db32080. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.195:38899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:33:06.696 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2538) Learned about tablet 43717cb820ff40d99e9add5a0d3a05ce for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698' with partition [<start>, <end>)
10:33:06.697 [DEBUG - kudu-nio-1] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=43717cb820ff40d99e9add5a0d3a05ce@[099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515),51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)[L],5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)]}]
10:33:06.697 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=43717cb820ff40d99e9add5a0d3a05ce, attempt=14, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=2821), Traces: [0ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [1ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [1ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [17ms] refreshing cache from master, [17ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [20ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [21ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [23ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [23ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [36ms] refreshing cache from master, [36ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [39ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [40ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [42ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [42ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [76ms] refreshing cache from master, [76ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [79ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [80ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [81ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [82ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [156ms] refreshing cache from master, [156ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [159ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [160ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [162ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [162ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [177ms] refreshing cache from master, [177ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [179ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [181ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [182ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [182ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [636ms] refreshing cache from master, [636ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [639ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, [641ms] sending RPC to server 5bf7fa054f57426e99c2f91a6db32080, [642ms] received response from server 5bf7fa054f57426e99c2f91a6db32080: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [643ms] delaying RPC due to: Network error: Failed to connect to peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899): Connection refused: /127.20.54.195:38899, [2817ms] refreshing cache from master, [2817ms] Sub RPC GetTableLocations: sending RPC to server master-127.20.54.252:34659, [2820ms] Sub RPC GetTableLocations: received response from server master-127.20.54.252:34659: OK, deferred=Deferred@2044080104(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$236/0x00007fdd9036bed0@1903a849 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842) -> (continuation of Deferred@596611196 after retry RPC@985784364) -> (continuation of Deferred@1698774984 after retry RPC@1345614796) -> (continuation of Deferred@1325091898 after retry RPC@1025698874), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@311b8ffa -> wakeup thread Time-limited test -> (continuation of Deferred@585357478 after retry RPC@1340798815) -> (continuation of Deferred@1629420105 after retry RPC@1469121120) -> (continuation of Deferred@462823570 after retry RPC@1145811842) -> (continuation of Deferred@596611196 after retry RPC@985784364) -> (continuation of Deferred@1698774984 after retry RPC@1345614796) -> (continuation of Deferred@1325091898 after retry RPC@1025698874))) after lookup
10:33:06.698 [DEBUG - kudu-nio-1] (Connection.java:748) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] connecting to peer
10:33:06.699 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] Successfully connected to peer
10:33:06.717 [DEBUG - kudu-nio-6] (Negotiator.java:918) Authenticated connection [id: 0x3a3186d8, L:/127.0.0.1:42282 - R:/127.20.54.194:46199] using TOKEN/null
10:33:06.739 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.20.54.195:38899
10:33:06.739 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:06.740245 20699 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:33:06.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:38899
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=42411
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:33:06.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
10:33:07.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.081714 24623 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
10:33:07.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.082080 24623 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
10:33:07.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.082192 24623 flags.cc:432] Enabled unsafe flag: --never_fsync=true
10:33:07.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.090803 24623 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
10:33:07.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.091030 24623 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.54.195
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.101016 24623 tablet_server_runner.cc:78] Tablet server non-default flags:
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/logs
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.20.54.212:42285
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.20.54.195:38899
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:33:07.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.20.54.195
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=42411
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.20.54.254:39167,127.20.54.253:43411,127.20.54.252:34659
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.20.54.195
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 14880490066cc6e355a232e5a6411307cd134162
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 11 Aug 2026 10:24:06 UTC on bdcb31816ec0
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 17617
10:33:07.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
10:33:07.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.103132 24623 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
10:33:07.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.105110 24623 file_cache.cc:504] Constructed file cache file cache with capacity 419430
10:33:07.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.117241 24640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:33:07.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.117229 24639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:33:07.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.117676 24644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
10:33:07.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.118906 24623 server_base.cc:1061] running on GCE node
10:33:07.119 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.119529 24623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
10:33:07.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.120517 24623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
10:33:07.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.121766 24623 hybrid_clock.cc:648] HybridClock initialized: now 1786444387121697 us; error 49 us; skew 500 ppm
10:33:07.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.122128 24623 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
10:33:07.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.124305 24623 webserver.cc:533] Webserver started at http://127.20.54.195:42411/ using document root <none> and password file <none>
10:33:07.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.125123 24623 fs_manager.cc:362] Metadata directory not provided
10:33:07.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.125268 24623 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
10:33:07.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.130571 24623 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.002s
10:33:07.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.133941 24652 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
10:33:07.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.135229 24623 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.135424 24623 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data,/tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "5bf7fa054f57426e99c2f91a6db32080"
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-11 10:32:30 on dist-test-slave-s11t"
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.136178 24623 fs_report.cc:389] FS layout report
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/wal
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/data
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
10:33:07.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
10:33:07.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
10:33:07.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
10:33:07.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
10:33:07.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
10:33:07.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.150015 24623 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
10:33:07.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.150951 24623 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
10:33:07.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.151302 24623 kserver.cc:163] Server-wide thread pool size limit: 3276
10:33:07.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.152299 24623 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
10:33:07.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.154672 24664 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
10:33:07.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.160383 24623 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
10:33:07.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.160487 24623 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s	user 0.001s	sys 0.000s
10:33:07.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.160619 24623 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
10:33:07.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.163609 24623 ts_tablet_manager.cc:616] Registered 1 tablets
10:33:07.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.163733 24623 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s	user 0.000s	sys 0.000s
10:33:07.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.164289 24664 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap starting.
10:33:07.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.202780 24664 log.cc:826] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Log is configured to *not* fsync() on all Append() calls
10:33:07.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.208850 24623 rpc_server.cc:307] RPC server started. Bound to: 127.20.54.195:38899
10:33:07.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.208956 24859 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.54.195:38899 every 8 connection(s)
10:33:07.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.210608 24623 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskr2Ef5i/test-tmp/mini-kudu-cluster844239517724560703/ts-2/data/info.pb
10:33:07.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.211922 20699 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu as pid 24623
10:33:07.212 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
10:33:07.256 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:33:07.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.259799 24860 heartbeater.cc:344] Connected to a master server at 127.20.54.252:34659
10:33:07.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.260206 24860 heartbeater.cc:461] Registering TS with master...
10:33:07.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.261461 24860 heartbeater.cc:507] Master 127.20.54.252:34659 requested a full tablet report, sending...
10:33:07.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.265231 20997 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:33:07.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.267192 20997 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.54.195:48723
10:33:07.269 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1786444350698, tablet=43717cb820ff40d99e9add5a0d3a05ce, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
10:33:07.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.271327 24863 heartbeater.cc:344] Connected to a master server at 127.20.54.254:39167
10:33:07.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.271545 24863 heartbeater.cc:461] Registering TS with master...
10:33:07.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.272033 24863 heartbeater.cc:507] Master 127.20.54.254:39167 requested a full tablet report, sending...
10:33:07.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.274340 20767 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:33:07.276 [DEBUG - main] (Connection.java:575) [peer 099ce31a16a94865909f17171a18a1ac(127.20.54.193:46515)] disconnecting while in state TERMINATED
10:33:07.277 [DEBUG - main] (Connection.java:575) [peer 5bf7fa054f57426e99c2f91a6db32080(127.20.54.195:38899)] disconnecting while in state TERMINATED
10:33:07.278 [DEBUG - main] (Connection.java:575) [peer master-127.20.54.253:43411(127.20.54.253:43411)] disconnecting while in state READY
10:33:07.281 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.20.54.253:43411(127.20.54.253:43411)] handling channelInactive
10:33:07.282 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.20.54.253:43411(127.20.54.253:43411)] cleaning up while in state READY due to: connection closed
10:33:07.282 [DEBUG - main] (Connection.java:575) [peer master-127.20.54.254:39167(127.20.54.254:39167)] disconnecting while in state READY
10:33:07.283 [DEBUG - kudu-nio-2] (Connection.java:239) [peer master-127.20.54.254:39167(127.20.54.254:39167)] handling channelInactive
10:33:07.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.282315 24864 heartbeater.cc:344] Connected to a master server at 127.20.54.253:43411
10:33:07.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.282527 24864 heartbeater.cc:461] Registering TS with master...
10:33:07.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.282963 24864 heartbeater.cc:507] Master 127.20.54.253:43411 requested a full tablet report, sending...
10:33:07.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.285548 20877 ts_manager.cc:194] Re-registered known tserver with Master: 5bf7fa054f57426e99c2f91a6db32080 (127.20.54.195:38899)
10:33:07.286 [DEBUG - main] (Connection.java:575) [peer master-127.20.54.252:34659(127.20.54.252:34659)] disconnecting while in state READY
10:33:07.286 [DEBUG - kudu-nio-2] (Connection.java:709) [peer master-127.20.54.254:39167(127.20.54.254:39167)] cleaning up while in state READY due to: connection closed
10:33:07.286 [DEBUG - main] (Connection.java:575) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] disconnecting while in state READY
10:33:07.287 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.20.54.252:34659(127.20.54.252:34659)] handling channelInactive
10:33:07.287 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.20.54.252:34659(127.20.54.252:34659)] cleaning up while in state READY due to: connection closed
10:33:07.287 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] handling channelInactive
10:33:07.288 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 51655106b4ae4a3895af2bf5ee4529f9(127.20.54.194:46199)] cleaning up while in state READY due to: connection closed
10:33:07.289 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
10:33:07.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.304733 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 24325
10:33:07.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.318506 24523 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:07.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.323954 24664 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=30 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
10:33:07.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.325297 24664 tablet_bootstrap.cc:492] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Bootstrap complete.
10:33:07.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.327523 24664 ts_tablet_manager.cc:1403] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent bootstrapping tablet: real 0.164s	user 0.072s	sys 0.024s
10:33:07.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.336664 24664 raft_consensus.cc:359] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 11 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:07.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.338369 24664 raft_consensus.cc:740] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf7fa054f57426e99c2f91a6db32080, State: Initialized, Role: FOLLOWER
10:33:07.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.339499 24664 consensus_queue.cc:260] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 11.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51655106b4ae4a3895af2bf5ee4529f9" member_type: VOTER last_known_addr { host: "127.20.54.194" port: 46199 } } peers { permanent_uuid: "5bf7fa054f57426e99c2f91a6db32080" member_type: VOTER last_known_addr { host: "127.20.54.195" port: 38899 } } peers { permanent_uuid: "099ce31a16a94865909f17171a18a1ac" member_type: VOTER last_known_addr { host: "127.20.54.193" port: 46515 } }
10:33:07.341 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.340736 24860 heartbeater.cc:499] Master 127.20.54.252:34659 was elected leader, sending a full tablet report...
10:33:07.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.345037 24664 ts_tablet_manager.cc:1434] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.004s
10:33:07.422 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.422245 24768 raft_consensus.cc:3060] T 43717cb820ff40d99e9add5a0d3a05ce P 5bf7fa054f57426e99c2f91a6db32080 [term 11 FOLLOWER]: Advancing to term 12
10:33:07.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.493111 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 24325
10:33:07.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.529354 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 24021
10:33:07.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.539250 24214 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:07.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.658416 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 24021
10:33:07.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.685153 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 24623
10:33:07.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.692422 24848 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:07.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.790745 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 24623
10:33:07.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.818456 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 20720
10:33:07.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.819772 20821 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:07.942 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.942592 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 20720
10:33:07.975 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.975461 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 20833
10:33:07.977 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:07.976738 20925 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:07.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.979779 20970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.54.254:39167: connect: Connection refused (error 111)
10:33:07.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260811 10:33:07.981477 20970 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2807ab88f71b472fb3e13c8ea6774302 -> Peer 0548b026ce5043ada90d730da43e3cf3 (127.20.54.254:39167): Couldn't send request to peer 0548b026ce5043ada90d730da43e3cf3. Status: Network error: Client connection negotiation failed: client connection to 127.20.54.254:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
10:33:08.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:08.098449 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 20833
10:33:08.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:08.131441 20699 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu pid 20942
10:33:08.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:08.132823 21047 generic_service.cc:196] Checking for leaks (request via RPC)
10:33:08.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260811 10:33:08.259424 20699 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskr2Ef5i/build/asan/bin/kudu with pid 20942
10:33:08.321 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-11T10:33:08Z chronyd exiting

Time: 43.728

OK (1 test)

