JUnit version 4.13.2
.02:10:51.287 [DEBUG - main] (RandomUtils.java:49) Using random seed: 1783476651281
02:10:51.338 [INFO - main] (RetryRule.java:138) Creating RetryStatement with result reporter and retry count of 0 (no retries)
02:10:51.368 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
02:10:51.378 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-tasknKVXmV/test-tmp
02:10:51.402 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/debug/bin
02:10:51.405 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-tasknKVXmV/java/build/dist-test/../../../build/debug/bin/kudu, test, mini_cluster, --serialization=pb]
02:10:51.603 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
  num_masters: 3
  num_tservers: 3
  enable_kerberos: false
  cluster_root: "/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347"
  hms_mode: NONE
  mini_kdc_options {
  }
  principal: "kudu"
  mini_oidc_options {
  }
}

02:10:51.616 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
02:10:51.617 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}

02:10:51.619 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.618711 20299 mini_oidc.cc:150] Starting JWKS server
02:10:51.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.620515 20299 webserver.cc:534] Webserver started at http://127.0.0.1:34549/ using document root <none> and password file <none>
02:10:51.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.620719 20299 mini_oidc.cc:186] Starting OIDC Discovery server
02:10:51.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.621003 20299 webserver.cc:534] Webserver started at http://127.0.0.1:45019/ using document root <none> and password file <none>
02:10:51.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-07-08T02:10:51Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
02:10:51.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-07-08T02:10:51Z Disabled control of system clock
02:10:51.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.634617 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:51.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:51.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal
02:10:51.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data
02:10:51.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:51.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:51.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:51.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:51.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:51.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:51.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/logs
02:10:51.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data/info.pb
02:10:51.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:51.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:51.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:51.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:51.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:51.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:51.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:51.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:51.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:10:51.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:51.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:10:51.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:10:51.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.254:37903
02:10:51.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.254
02:10:51.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:51.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:51.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:51.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:51.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:51.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:10:51.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:51.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:10:51.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34549/jwks.json with env {}
02:10:51.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.733825 20310 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:51.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.734021 20310 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:51.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.734061 20310 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:51.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.738209 20310 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:10:51.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.738270 20310 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:51.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.738304 20310 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:10:51.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.738322 20310 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:10:51.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.738338 20310 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:10:51.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.738351 20310 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:34549/jwks.json
02:10:51.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.743046 20310 master_runner.cc:387] Master server non-default flags:
02:10:51.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:51.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:51.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:51.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:51.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data
02:10:51.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal
02:10:51.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:10:51.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.254:37903
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34549/jwks.json
02:10:51.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:51.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:51.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data/info.pb
02:10:51.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.254
02:10:51.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:51.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:51.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.20310
02:10:51.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:51.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:51.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:51.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:51.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/logs
02:10:51.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:51.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:51.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:51.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:10:51.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:51.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:51.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:51.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:51.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:51.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.744114 20310 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:51.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.745204 20310 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:51.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.750980 20318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:51.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.751008 20316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:51.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.751201 20310 server_base.cc:1061] running on GCE node
02:10:51.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.751263 20315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:51.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.751847 20310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:51.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.753005 20310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:51.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.754154 20310 hybrid_clock.cc:648] HybridClock initialized: now 1783476651754102 us; error 71 us; skew 500 ppm
02:10:51.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.755802 20310 webserver.cc:534] Webserver started at http://127.19.210.254:39379/ using document root <none> and password file <none>
02:10:51.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.756393 20310 fs_manager.cc:362] Metadata directory not provided
02:10:51.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.756454 20310 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:51.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.756692 20310 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:10:51.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.758313 20310 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data/instance:
02:10:51.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bceaf730e391404a9c2b78d1602ce2d9"
02:10:51.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:51 on dist-test-slave-7tzg"
02:10:51.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.758821 20310 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal/instance:
02:10:51.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bceaf730e391404a9c2b78d1602ce2d9"
02:10:51.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:51 on dist-test-slave-7tzg"
02:10:51.762 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.762434 20310 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
02:10:51.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.764674 20327 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:51.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.765687 20310 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
02:10:51.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.765790 20310 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal
02:10:51.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bceaf730e391404a9c2b78d1602ce2d9"
02:10:51.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:51 on dist-test-slave-7tzg"
02:10:51.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.765862 20310 fs_report.cc:389] FS layout report
02:10:51.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:51.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal
02:10:51.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal
02:10:51.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data/data
02:10:51.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:51.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:51.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:51.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:51.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:51.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:51.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:51.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:51.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:51.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:51.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:51.783 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.783375 20310 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:51.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.783983 20310 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:51.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.784143 20310 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:51.790 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.790685 20310 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.254:37903
02:10:51.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.790705 20380 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.254:37903 every 8 connection(s)
02:10:51.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.791597 20310 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/data/info.pb
02:10:51.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.794378 20381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:10:51.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.797242 20381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } has no permanent_uuid. Determining permanent_uuid...
02:10:51.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.800432 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 20310
02:10:51.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.800606 20299 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-0/wal/instance
02:10:51.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.802170 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:51.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:51.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal
02:10:51.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data
02:10:51.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:51.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:51.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:51.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:51.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:51.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/logs
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data/info.pb
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:51.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:51.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:51.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:51.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:51.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:10:51.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:51.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:10:51.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:10:51.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.253:42855
02:10:51.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.253
02:10:51.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:51.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:51.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:51.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:51.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:51.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:10:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:10:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34549/jwks.json with env {}
02:10:51.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.808148 20381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } has no permanent_uuid. Determining permanent_uuid...
02:10:51.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.809000 20329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.253:42855: connect: Connection refused (error 111)
02:10:51.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.810191 20381 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.253:42855: Network error: Client connection negotiation failed: client connection to 127.19.210.253:42855: connect: Connection refused (error 111)
02:10:51.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.859434 20381 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } attempt: 1
02:10:51.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.861093 20381 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.253:42855: Network error: Client connection negotiation failed: client connection to 127.19.210.253:42855: connect: Connection refused (error 111)
02:10:51.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.902980 20385 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:51.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.903172 20385 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:51.904 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.903231 20385 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:51.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.906729 20385 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:10:51.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.906790 20385 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:51.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.906817 20385 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:10:51.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.906839 20385 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:10:51.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.906862 20385 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:10:51.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.906879 20385 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:34549/jwks.json
02:10:51.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.910995 20385 master_runner.cc:387] Master server non-default flags:
02:10:51.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:51.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:51.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:51.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:51.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data
02:10:51.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal
02:10:51.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:10:51.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:51.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:51.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.253:42855
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34549/jwks.json
02:10:51.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data/info.pb
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.253
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.20385
02:10:51.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/logs
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:51.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:10:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:51.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.912321 20385 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:51.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.913772 20385 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.919881 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.919862 20391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:51.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.920670 20385 server_base.cc:1061] running on GCE node
02:10:51.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.919875 20392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:51.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.921489 20385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:51.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.922536 20385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:51.924 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.924026 20385 hybrid_clock.cc:648] HybridClock initialized: now 1783476651924006 us; error 38 us; skew 500 ppm
02:10:51.925 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.925774 20385 webserver.cc:534] Webserver started at http://127.19.210.253:34521/ using document root <none> and password file <none>
02:10:51.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.926358 20385 fs_manager.cc:362] Metadata directory not provided
02:10:51.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.926417 20385 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:51.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.926640 20385 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:10:51.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.928157 20385 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data/instance:
02:10:51.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "2830e0d7326749baadb6b39e54c05270"
02:10:51.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:51 on dist-test-slave-7tzg"
02:10:51.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.928681 20385 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal/instance:
02:10:51.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "2830e0d7326749baadb6b39e54c05270"
02:10:51.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:51 on dist-test-slave-7tzg"
02:10:51.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.929255 20381 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } attempt: 2
02:10:51.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.930716 20381 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.253:42855: Network error: Client connection negotiation failed: client connection to 127.19.210.253:42855: connect: Connection refused (error 111)
02:10:51.932 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.932284 20385 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
02:10:51.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.934383 20400 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:51.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.935374 20385 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:10:51.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.935487 20385 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal
02:10:51.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "2830e0d7326749baadb6b39e54c05270"
02:10:51.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:51 on dist-test-slave-7tzg"
02:10:51.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.935577 20385 fs_report.cc:389] FS layout report
02:10:51.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:51.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal
02:10:51.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data/data
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:51.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:51.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:51.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:51.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:51.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:51.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.947403 20385 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:51.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.948061 20385 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:51.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.948226 20385 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:51.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.954988 20385 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.253:42855
02:10:51.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.954998 20453 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.253:42855 every 8 connection(s)
02:10:51.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.956071 20385 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/data/info.pb
02:10:51.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.958806 20454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:10:51.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.959206 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 20385
02:10:51.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.959291 20299 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-1/wal/instance
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.959955 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:51.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/logs
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data/info.pb
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:51.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.252:44531
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.252
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:10:51.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:51.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:10:51.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34549/jwks.json with env {}
02:10:51.964 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.963356 20454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } has no permanent_uuid. Determining permanent_uuid...
02:10:51.974 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.973781 20454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } has no permanent_uuid. Determining permanent_uuid...
02:10:51.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:51.980037 20454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
02:10:51.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.980986 20402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.252:44531: connect: Connection refused (error 111)
02:10:51.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:51.982079 20454 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.252:44531: Network error: Client connection negotiation failed: client connection to 127.19.210.252:44531: connect: Connection refused (error 111)
02:10:52.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.021873 20381 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } attempt: 3
02:10:52.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.026144 20381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
02:10:52.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.027436 20381 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.252:44531: Network error: Client connection negotiation failed: client connection to 127.19.210.252:44531: connect: Connection refused (error 111)
02:10:52.031 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.031275 20454 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } attempt: 1
02:10:52.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.032342 20454 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.252:44531: Network error: Client connection negotiation failed: client connection to 127.19.210.252:44531: connect: Connection refused (error 111)
02:10:52.058 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.058585 20381 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } attempt: 1
02:10:52.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.060000 20381 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.252:44531: Network error: Client connection negotiation failed: client connection to 127.19.210.252:44531: connect: Connection refused (error 111)
02:10:52.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.067408 20456 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:52.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.067617 20456 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:52.068 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.067663 20456 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:52.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.071063 20456 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:10:52.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.071117 20456 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:52.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.071135 20456 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:10:52.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.071151 20456 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:10:52.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.071167 20456 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:10:52.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.071180 20456 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:34549/jwks.json
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.074990 20456 master_runner.cc:387] Master server non-default flags:
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:10:52.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.252:44531
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34549/jwks.json
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data/info.pb
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.252
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:52.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.20456
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/logs
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:52.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:52.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.075935 20456 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:52.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.077029 20456 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:52.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.082947 20467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.082995 20465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.082957 20464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.083779 20456 server_base.cc:1061] running on GCE node
02:10:52.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.084371 20456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:52.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.085317 20456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:52.086 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.086457 20456 hybrid_clock.cc:648] HybridClock initialized: now 1783476652086404 us; error 74 us; skew 500 ppm
02:10:52.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.088131 20456 webserver.cc:534] Webserver started at http://127.19.210.252:44985/ using document root <none> and password file <none>
02:10:52.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.088762 20456 fs_manager.cc:362] Metadata directory not provided
02:10:52.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.088821 20456 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:52.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.089062 20456 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:10:52.090 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.090638 20456 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data/instance:
02:10:52.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "db986057f6a34d4981d2c9332db05737"
02:10:52.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.091152 20456 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal/instance:
02:10:52.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "db986057f6a34d4981d2c9332db05737"
02:10:52.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.094369 20456 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
02:10:52.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.096458 20473 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:52.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.097388 20456 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:10:52.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.097501 20456 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal
02:10:52.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "db986057f6a34d4981d2c9332db05737"
02:10:52.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.097596 20456 fs_report.cc:389] FS layout report
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data/data
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:52.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:52.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:52.100 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.100515 20454 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } attempt: 2
02:10:52.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.102002 20454 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.210.252:44531: Network error: Client connection negotiation failed: client connection to 127.19.210.252:44531: connect: Connection refused (error 111)
02:10:52.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.116556 20456 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:52.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.117295 20456 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:52.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.117475 20456 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:52.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.124539 20456 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.252:44531
02:10:52.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.124534 20525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.252:44531 every 8 connection(s)
02:10:52.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.125593 20456 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/data/info.pb
02:10:52.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.128263 20526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:10:52.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.130091 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 20456
02:10:52.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.130198 20299 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/master-2/wal/instance
02:10:52.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.133491 20526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } has no permanent_uuid. Determining permanent_uuid...
02:10:52.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.135176 20381 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } attempt: 2
02:10:52.142 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.142124 20526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } has no permanent_uuid. Determining permanent_uuid...
02:10:52.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.145246 20381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: Bootstrap starting.
02:10:52.147 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.146989 20526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
02:10:52.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.147868 20381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: Neither blocks nor log segments found. Creating new log.
02:10:52.149 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.149067 20381 log.cc:826] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: Log is configured to *not* fsync() on all Append() calls
02:10:52.151 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.151264 20381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: No bootstrap required, opened a new log
02:10:52.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.153117 20526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: Bootstrap starting.
02:10:52.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.154120 20381 raft_consensus.cc:359] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.154264 20381 raft_consensus.cc:385] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:10:52.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.154315 20381 raft_consensus.cc:740] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceaf730e391404a9c2b78d1602ce2d9, State: Initialized, Role: FOLLOWER
02:10:52.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.155230 20526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: Neither blocks nor log segments found. Creating new log.
02:10:52.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.155259 20381 consensus_queue.cc:260] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.155948 20526 log.cc:826] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: Log is configured to *not* fsync() on all Append() calls
02:10:52.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.156399 20381 sys_catalog.cc:565] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: configured and running, proceeding with master startup.
02:10:52.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.157800 20526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: No bootstrap required, opened a new log
02:10:52.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.161101 20548 catalog_manager.cc:1593] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
02:10:52.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.161458 20548 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
02:10:52.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.161887 20536 sys_catalog.cc:455] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.162271 20536 sys_catalog.cc:458] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.160818 20310 master_runner.cc:187] Error getting master registration for 127.19.210.253:42855: OK, instance_id { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" instance_seqno: 1783476651953256 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
02:10:52.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.162940 20310 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
02:10:52.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.164178 20526 raft_consensus.cc:359] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.164366 20526 raft_consensus.cc:385] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:10:52.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.164418 20526 raft_consensus.cc:740] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db986057f6a34d4981d2c9332db05737, State: Initialized, Role: FOLLOWER
02:10:52.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.164877 20526 consensus_queue.cc:260] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.165316 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.165416 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.166015 20526 sys_catalog.cc:565] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: configured and running, proceeding with master startup.
02:10:52.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.170827 20561 catalog_manager.cc:1593] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
02:10:52.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.170902 20561 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
02:10:52.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.171514 20456 master_runner.cc:187] Error getting master registration for 127.19.210.253:42855: OK, instance_id { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" instance_seqno: 1783476651953256 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
02:10:52.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.173727 20456 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
02:10:52.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.193172 20454 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } attempt: 3
02:10:52.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.200409 20454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270: Bootstrap starting.
02:10:52.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.203724 20454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270: Neither blocks nor log segments found. Creating new log.
02:10:52.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.204578 20454 log.cc:826] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270: Log is configured to *not* fsync() on all Append() calls
02:10:52.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.206441 20454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270: No bootstrap required, opened a new log
02:10:52.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.209931 20454 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.210107 20454 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:10:52.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.210157 20454 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2830e0d7326749baadb6b39e54c05270, State: Initialized, Role: FOLLOWER
02:10:52.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.210644 20454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.211158 20565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.211529 20565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.211861 20454 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: configured and running, proceeding with master startup.
02:10:52.217 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.217396 20577 catalog_manager.cc:1593] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
02:10:52.217 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.217777 20577 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
02:10:52.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.220175 20385 master_runner.cc:264] No leader master found from master 2830e0d7326749baadb6b39e54c05270
02:10:52.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.220237 20385 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
02:10:52.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.285497 20333 tablet.cc:2406] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:10:52.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.350942 20565 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:10:52.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.351117 20565 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.352190 20565 leader_election.cc:290] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bceaf730e391404a9c2b78d1602ce2d9 (127.19.210.254:37903), db986057f6a34d4981d2c9332db05737 (127.19.210.252:44531)
02:10:52.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.352849 20355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2830e0d7326749baadb6b39e54c05270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceaf730e391404a9c2b78d1602ce2d9" is_pre_election: true
02:10:52.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.353121 20355 raft_consensus.cc:2468] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2830e0d7326749baadb6b39e54c05270 in term 0.
02:10:52.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.353582 20401 leader_election.cc:304] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2830e0d7326749baadb6b39e54c05270, bceaf730e391404a9c2b78d1602ce2d9; no voters: 
02:10:52.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.353837 20565 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Leader pre-election won for term 1
02:10:52.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.353900 20565 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
02:10:52.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.353941 20565 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 0 FOLLOWER]: Advancing to term 1
02:10:52.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.354329 20500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2830e0d7326749baadb6b39e54c05270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db986057f6a34d4981d2c9332db05737" is_pre_election: true
02:10:52.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.354564 20500 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2830e0d7326749baadb6b39e54c05270 in term 0.
02:10:52.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.355077 20565 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.355444 20565 leader_election.cc:290] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [CANDIDATE]: Term 1 election: Requested vote from peers bceaf730e391404a9c2b78d1602ce2d9 (127.19.210.254:37903), db986057f6a34d4981d2c9332db05737 (127.19.210.252:44531)
02:10:52.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.355860 20355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2830e0d7326749baadb6b39e54c05270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceaf730e391404a9c2b78d1602ce2d9"
02:10:52.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.355967 20355 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 0 FOLLOWER]: Advancing to term 1
02:10:52.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.356925 20500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2830e0d7326749baadb6b39e54c05270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db986057f6a34d4981d2c9332db05737"
02:10:52.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.357033 20500 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 0 FOLLOWER]: Advancing to term 1
02:10:52.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.357187 20355 raft_consensus.cc:2468] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2830e0d7326749baadb6b39e54c05270 in term 1.
02:10:52.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.357474 20401 leader_election.cc:304] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2830e0d7326749baadb6b39e54c05270, bceaf730e391404a9c2b78d1602ce2d9; no voters: 
02:10:52.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.357623 20565 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 1 FOLLOWER]: Leader election won for term 1
02:10:52.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.357806 20565 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [term 1 LEADER]: Becoming Leader. State: Replica: 2830e0d7326749baadb6b39e54c05270, State: Running, Role: LEADER
02:10:52.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.358064 20500 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2830e0d7326749baadb6b39e54c05270 in term 1.
02:10:52.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.358212 20565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } }
02:10:52.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.360064 20579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2830e0d7326749baadb6b39e54c05270. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.360159 20579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: This master's current role is: LEADER
02:10:52.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.361057 20581 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
02:10:52.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.362171 20581 catalog_manager.cc:1519] Initializing Kudu cluster ID...
02:10:52.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.367206 20500 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [term 1 FOLLOWER]: Refusing update from remote peer 2830e0d7326749baadb6b39e54c05270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:10:52.368 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.367785 20565 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:10:52.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.368623 20479 tablet.cc:2406] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.371721 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2830e0d7326749baadb6b39e54c05270. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.372727 20355 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [term 1 FOLLOWER]: Refusing update from remote peer 2830e0d7326749baadb6b39e54c05270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.371796 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.373762 20565 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.376958 20536 sys_catalog.cc:455] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2830e0d7326749baadb6b39e54c05270. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.377055 20536 sys_catalog.cc:458] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.377285 20583 mvcc.cc:204] Tried to move back new op lower bound from 7305120368089083904 to 7305120368062783488. Current Snapshot: MvccSnapshot[applied={T|T < 7305120368089083904}]
02:10:52.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.377523 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.377617 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.376479 20565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.377991 20565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: This master's current role is: LEADER
02:10:52.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.379494 20582 mvcc.cc:204] Tried to move back new op lower bound from 7305120368089083904 to 7305120368062783488. Current Snapshot: MvccSnapshot[applied={T|T < 7305120368089083904}]
02:10:52.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.382961 20565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.383074 20565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270 [sys.catalog]: This master's current role is: LEADER
02:10:52.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.383635 20581 catalog_manager.cc:1382] Generated new cluster ID: 5818950e12d549cc8b8fe0e5ec85fc5d
02:10:52.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.383687 20581 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
02:10:52.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.384135 20536 sys_catalog.cc:455] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2830e0d7326749baadb6b39e54c05270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceaf730e391404a9c2b78d1602ce2d9" member_type: VOTER last_known_addr { host: "127.19.210.254" port: 37903 } } peers { permanent_uuid: "2830e0d7326749baadb6b39e54c05270" member_type: VOTER last_known_addr { host: "127.19.210.253" port: 42855 } } peers { permanent_uuid: "db986057f6a34d4981d2c9332db05737" member_type: VOTER last_known_addr { host: "127.19.210.252" port: 44531 } } }
02:10:52.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.384227 20536 sys_catalog.cc:458] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9 [sys.catalog]: This master's current role is: FOLLOWER
02:10:52.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.385561 20585 mvcc.cc:204] Tried to move back new op lower bound from 7305120368089083904 to 7305120368062783488. Current Snapshot: MvccSnapshot[applied={T|T < 7305120368089083904}]
02:10:52.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.408767 20581 catalog_manager.cc:1405] Generated new certificate authority record
02:10:52.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.409816 20581 catalog_manager.cc:1539] Loading token signing keys...
02:10:52.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.427830 20581 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 2830e0d7326749baadb6b39e54c05270: Generated new TSK 0
02:10:52.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.429255 20581 catalog_manager.cc:1549] Initializing in-progress tserver states...
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.442719 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.443 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:0
02:10:52.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:10:52.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.547514 20591 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:52.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.547717 20591 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:52.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.547757 20591 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:52.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.550971 20591 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:52.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.551069 20591 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.193
02:10:52.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.554731 20591 tablet_server_runner.cc:78] Tablet server non-default flags:
02:10:52.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:0
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:52.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.20591
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:10:52.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:52.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:52.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:52.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:52.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:52.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.555797 20591 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:52.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.557245 20591 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:52.563 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.563479 20596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.563 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.563479 20597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.563879 20591 server_base.cc:1061] running on GCE node
02:10:52.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.563521 20599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.564234 20591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:52.564 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.564805 20591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:52.566 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.565963 20591 hybrid_clock.cc:648] HybridClock initialized: now 1783476652565943 us; error 34 us; skew 500 ppm
02:10:52.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.567611 20591 webserver.cc:534] Webserver started at http://127.19.210.193:36261/ using document root <none> and password file <none>
02:10:52.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.568164 20591 fs_manager.cc:362] Metadata directory not provided
02:10:52.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.568222 20591 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:52.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.568434 20591 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:10:52.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.569936 20591 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/instance:
02:10:52.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:10:52.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.570436 20591 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal/instance:
02:10:52.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:10:52.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.573 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.573719 20591 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
02:10:52.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.575722 20605 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:52.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.576714 20591 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.576813 20591 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.576910 20591 fs_report.cc:389] FS layout report
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/data
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:52.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:52.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:52.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.602555 20591 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:52.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.603314 20591 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:52.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.603492 20591 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:52.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.604115 20591 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:10:52.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.605149 20591 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
02:10:52.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.605196 20591 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
02:10:52.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.605247 20591 ts_tablet_manager.cc:616] Registered 0 tablets
02:10:52.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.605268 20591 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
02:10:52.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.613772 20591 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.193:41573
02:10:52.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.613835 20719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.193:41573 every 8 connection(s)
02:10:52.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.614683 20591 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:10:52.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.619728 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 20591
02:10:52.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.619835 20299 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal/instance
02:10:52.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.623643 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:52.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:52.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:10:52.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:0
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.623976 20722 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.624195 20722 heartbeater.cc:461] Registering TS with master...
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.624336 20720 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.624397 20720 heartbeater.cc:461] Registering TS with master...
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.624855 20720 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:10:52.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.624855 20722 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:10:52.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.626384 20490 ts_manager.cc:194] Registered new tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:10:52.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.626471 20417 ts_manager.cc:194] Registered new tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:10:52.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.626590 20721 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:10:52.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.626658 20721 heartbeater.cc:461] Registering TS with master...
02:10:52.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.626842 20721 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:10:52.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.627727 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.193:59351
02:10:52.629 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.628981 20337 ts_manager.cc:194] Registered new tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:10:52.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.724325 20727 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:52.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.724506 20727 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:52.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.724581 20727 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:52.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.727926 20727 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:52.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.728039 20727 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.194
02:10:52.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.731755 20727 tablet_server_runner.cc:78] Tablet server non-default flags:
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:0
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.20727
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:52.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.733057 20727 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:52.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.734328 20727 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:52.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.740221 20735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.740291 20732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.740249 20733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.740427 20727 server_base.cc:1061] running on GCE node
02:10:52.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.741212 20727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:52.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.741751 20727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:52.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.742913 20727 hybrid_clock.cc:648] HybridClock initialized: now 1783476652742903 us; error 23 us; skew 500 ppm
02:10:52.744 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.744592 20727 webserver.cc:534] Webserver started at http://127.19.210.194:39205/ using document root <none> and password file <none>
02:10:52.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.745136 20727 fs_manager.cc:362] Metadata directory not provided
02:10:52.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.745195 20727 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:52.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.745429 20727 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:10:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.746929 20727 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/instance:
02:10:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:10:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.747424 20727 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal/instance:
02:10:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:10:52.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.750 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.750458 20727 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
02:10:52.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.752390 20741 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:52.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.753265 20727 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:10:52.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.753368 20727 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:52.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:10:52.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.753456 20727 fs_report.cc:389] FS layout report
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/data
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:52.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:52.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:52.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.764035 20727 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:52.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.764696 20727 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:52.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.764861 20727 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:52.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.765389 20727 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:10:52.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.766350 20727 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
02:10:52.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.766397 20727 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
02:10:52.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.766443 20727 ts_tablet_manager.cc:616] Registered 0 tablets
02:10:52.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.766462 20727 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
02:10:52.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.775002 20727 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.194:43205
02:10:52.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.775043 20854 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.194:43205 every 8 connection(s)
02:10:52.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.775955 20727 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:10:52.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.780087 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 20727
02:10:52.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.780189 20299 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal/instance
02:10:52.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.784157 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:52.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:52.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:10:52.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:10:52.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:0
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.785539 20856 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.785616 20857 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.785735 20856 heartbeater.cc:461] Registering TS with master...
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.785738 20857 heartbeater.cc:461] Registering TS with master...
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.786301 20856 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:10:52.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.786301 20857 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:10:52.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.787360 20337 ts_manager.cc:194] Registered new tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:52.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.787401 20417 ts_manager.cc:194] Registered new tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:52.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.788053 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.194:35791
02:10:52.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.788374 20855 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:10:52.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.788460 20855 heartbeater.cc:461] Registering TS with master...
02:10:52.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.788669 20855 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:10:52.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.789319 20490 ts_manager.cc:194] Registered new tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:52.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.885773 20862 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:52.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.885946 20862 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:52.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.885984 20862 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:52.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.889147 20862 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:52.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.889237 20862 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.195
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.892861 20862 tablet_server_runner.cc:78] Tablet server non-default flags:
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:0
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.20862
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:52.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:52.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:52.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.893757 20862 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:52.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.894747 20862 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:52.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.900424 20867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.900468 20868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.900578 20862 server_base.cc:1061] running on GCE node
02:10:52.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:52.900461 20870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:52.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.901237 20862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:52.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.901793 20862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:52.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.902935 20862 hybrid_clock.cc:648] HybridClock initialized: now 1783476652902915 us; error 31 us; skew 500 ppm
02:10:52.904 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.904628 20862 webserver.cc:534] Webserver started at http://127.19.210.195:44723/ using document root <none> and password file <none>
02:10:52.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.905154 20862 fs_manager.cc:362] Metadata directory not provided
02:10:52.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.905212 20862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:52.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.905429 20862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:10:52.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.906903 20862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/instance:
02:10:52.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3aea1f3e173e4803a7ae66840502f546"
02:10:52.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.907385 20862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal/instance:
02:10:52.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3aea1f3e173e4803a7ae66840502f546"
02:10:52.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.910 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.910578 20862 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
02:10:52.912 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.912672 20876 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:52.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.913689 20862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
02:10:52.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.913785 20862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3aea1f3e173e4803a7ae66840502f546"
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.913872 20862 fs_report.cc:389] FS layout report
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/data
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:52.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:52.923 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.923437 20862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:52.924 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.923995 20862 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:52.924 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.924156 20862 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:52.924 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.924712 20862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:10:52.925 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.925686 20862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
02:10:52.925 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.925734 20862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
02:10:52.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.925779 20862 ts_tablet_manager.cc:616] Registered 0 tablets
02:10:52.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.925800 20862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
02:10:52.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.934127 20862 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.195:42923
02:10:52.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.934159 20989 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.195:42923 every 8 connection(s)
02:10:52.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.935027 20862 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:10:52.942 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.942189 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 20862
02:10:52.942 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.942286 20299 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal/instance
02:10:52.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.946664 20992 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:10:52.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.946715 20990 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:10:52.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.946916 20992 heartbeater.cc:461] Registering TS with master...
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.947446 20991 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.947495 20992 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.947526 20991 heartbeater.cc:461] Registering TS with master...
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.947681 20991 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.946916 20990 heartbeater.cc:461] Registering TS with master...
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.948138 20990 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:10:52.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.948462 20417 ts_manager.cc:194] Registered new tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:52.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.948930 20337 ts_manager.cc:194] Registered new tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:52.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.949047 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.195:41913
02:10:52.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.949352 20490 ts_manager.cc:194] Registered new tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:52.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:52.963708 20299 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
02:10:52.964 [DEBUG - main] (MiniKuduCluster.java:195) Response: 
02:10:52.965 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}

02:10:52.973 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
  masters {
    id {
      type: MASTER
      index: 0
    }
    bound_rpc_address {
      host: "127.19.210.254"
      port: 37903
    }
    bound_http_address {
      host: "127.19.210.254"
      port: 39379
    }
  }
  masters {
    id {
      type: MASTER
      index: 1
    }
    bound_rpc_address {
      host: "127.19.210.253"
      port: 42855
    }
    bound_http_address {
      host: "127.19.210.253"
      port: 34521
    }
  }
  masters {
    id {
      type: MASTER
      index: 2
    }
    bound_rpc_address {
      host: "127.19.210.252"
      port: 44531
    }
    bound_http_address {
      host: "127.19.210.252"
      port: 44985
    }
  }
}

02:10:52.976 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}

02:10:52.978 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
  tservers {
    id {
      type: TSERVER
      index: 0
    }
    bound_rpc_address {
      host: "127.19.210.193"
      port: 41573
    }
    bound_http_address {
      host: "127.19.210.193"
      port: 36261
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 1
    }
    bound_rpc_address {
      host: "127.19.210.194"
      port: 43205
    }
    bound_http_address {
      host: "127.19.210.194"
      port: 39205
    }
  }
  tservers {
    id {
      type: TSERVER
      index: 2
    }
    bound_rpc_address {
      host: "127.19.210.195"
      port: 42923
    }
    bound_http_address {
      host: "127.19.210.195"
      port: 44723
    }
  }
}

02:10:52.978 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
02:10:53.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.163154 20548 catalog_manager.cc:1294] Loaded cluster ID: 5818950e12d549cc8b8fe0e5ec85fc5d
02:10:53.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.163231 20548 catalog_manager.cc:1587] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: loading cluster ID for follower catalog manager: success
02:10:53.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.165599 20548 catalog_manager.cc:1609] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: acquiring CA information for follower catalog manager: success
02:10:53.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.166792 20548 catalog_manager.cc:1637] T 00000000000000000000000000000000 P bceaf730e391404a9c2b78d1602ce2d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
02:10:53.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.171891 20561 catalog_manager.cc:1294] Loaded cluster ID: 5818950e12d549cc8b8fe0e5ec85fc5d
02:10:53.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.171995 20561 catalog_manager.cc:1587] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: loading cluster ID for follower catalog manager: success
02:10:53.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.173753 20561 catalog_manager.cc:1609] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: acquiring CA information for follower catalog manager: success
02:10:53.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.174739 20561 catalog_manager.cc:1637] T 00000000000000000000000000000000 P db986057f6a34d4981d2c9332db05737: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
02:10:53.185 [DEBUG - main] (CompositeStrategy.java:59) Unable to load modern Subject methods, relying only on legacy methods
java.lang.NoSuchMethodException: javax.security.auth.Subject.current()
	at java.base/java.lang.Class.getDeclaredMethod(Class.java:2677)
	at org.apache.kudu.client.internals.ModernStrategy.<init>(ModernStrategy.java:51)
	at org.apache.kudu.client.internals.CompositeStrategy.<init>(CompositeStrategy.java:52)
	at org.apache.kudu.client.internals.CompositeStrategy.<clinit>(CompositeStrategy.java:41)
	at org.apache.kudu.client.internals.SecurityManagerCompatibility.get(SecurityManagerCompatibility.java:43)
	at org.apache.kudu.client.SecurityContext.setupSubject(SecurityContext.java:155)
	at org.apache.kudu.client.SecurityContext.<init>(SecurityContext.java:138)
	at org.apache.kudu.client.AsyncKuduClient.<init>(AsyncKuduClient.java:431)
	at org.apache.kudu.client.AsyncKuduClient$AsyncKuduClientBuilder.build(AsyncKuduClient.java:3109)
	at org.apache.kudu.test.KuduTestHarness.before(KuduTestHarness.java:161)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttempt(RetryRule.java:225)
	at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:246)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
02:10:53.249 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication 
02:10:53.409 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.19.210.253:42855(127.19.210.253:42855)] connecting to peer
02:10:53.455 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.19.210.252:44531(127.19.210.252:44531)] connecting to peer
02:10:53.456 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.19.210.254:37903(127.19.210.254:37903)] connecting to peer
02:10:53.471 [DEBUG - kudu-nio-2] (Connection.java:757) [peer master-127.19.210.254:37903(127.19.210.254:37903)] Successfully connected to peer
02:10:53.471 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.19.210.253:42855(127.19.210.253:42855)] Successfully connected to peer
02:10:53.471 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.19.210.252:44531(127.19.210.252:44531)] Successfully connected to peer
02:10:53.554 [DEBUG - kudu-nio-2] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.19.210.254
02:10:53.554 [DEBUG - kudu-nio-1] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.19.210.252
02:10:53.558 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism PLAIN chosen for peer 127.19.210.253
02:10:53.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.629590 20722 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:10:53.762 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x928af58f, L:/127.0.0.1:59536 - R:/127.19.210.253:42855] using SASL/PLAIN
02:10:53.762 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0xa87bf3cf, L:/127.0.0.1:50200 - R:/127.19.210.252:44531] using SASL/PLAIN
02:10:53.764 [DEBUG - kudu-nio-2] (Negotiator.java:919) Authenticated connection [id: 0x8cbbce28, L:/127.0.0.1:53506 - R:/127.19.210.254:37903] using SASL/PLAIN
02:10:53.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.789697 20857 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:10:53.801 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet Kudu Master for table 'Kudu Master' with partition [<start>, <end>)
02:10:53.802 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=CreateTable, tablet=null, attempt=1, TimeoutTracker(timeout=50000, elapsed=426), Traces: [0ms] refreshing cache from master, [21ms] Sub RPC ConnectToMaster: sending RPC to server master-127.19.210.253:42855, [76ms] Sub RPC ConnectToMaster: sending RPC to server master-127.19.210.252:44531, [77ms] Sub RPC ConnectToMaster: sending RPC to server master-127.19.210.254:37903, [398ms] Sub RPC ConnectToMaster: received response from server master-127.19.210.252:44531: OK, [398ms] Sub RPC ConnectToMaster: received response from server master-127.19.210.254:37903: OK, [400ms] Sub RPC ConnectToMaster: received response from server master-127.19.210.253:42855: OK, deferred=null) after lookup
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.843264 20417 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59536:
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) name: "org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364"
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) schema {
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "key"
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: true
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:10:53.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column1_i"
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column2_i"
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: INT32
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column3_s"
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: STRING
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: true
02:10:53.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     encoding: DICT_ENCODING
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     compression: LZ4
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 4096
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   columns {
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     name: "column4_b"
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     type: BOOL
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_key: false
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_nullable: false
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     cfile_block_size: 0
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     immutable: false
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     is_auto_incrementing: false
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) partition_schema {
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   range_schema {
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     columns {
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)       name: "key"
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)     }
02:10:53.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)   }
02:10:53.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) }
02:10:53.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:53.846694 20417 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
02:10:53.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.869469 20653 tablet_service.cc:1511] Processing CreateTablet for tablet 49ba254cf896489cb98aaa7fafc55cc1 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364 [id=57349073663d41c8b3550c16008f5ea3]), partition=RANGE (key) PARTITION UNBOUNDED
02:10:53.871 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.871115 20653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49ba254cf896489cb98aaa7fafc55cc1. 1 dirs total, 0 dirs full, 0 dirs failed
02:10:53.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.873540 20789 tablet_service.cc:1511] Processing CreateTablet for tablet 49ba254cf896489cb98aaa7fafc55cc1 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364 [id=57349073663d41c8b3550c16008f5ea3]), partition=RANGE (key) PARTITION UNBOUNDED
02:10:53.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.874475 20789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49ba254cf896489cb98aaa7fafc55cc1. 1 dirs total, 0 dirs full, 0 dirs failed
02:10:53.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.874696 20924 tablet_service.cc:1511] Processing CreateTablet for tablet 49ba254cf896489cb98aaa7fafc55cc1 (DEFAULT_TABLE table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364 [id=57349073663d41c8b3550c16008f5ea3]), partition=RANGE (key) PARTITION UNBOUNDED
02:10:53.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.875509 20924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49ba254cf896489cb98aaa7fafc55cc1. 1 dirs total, 0 dirs full, 0 dirs failed
02:10:53.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.887295 21024 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap starting.
02:10:53.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.888840 21023 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap starting.
02:10:53.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.890180 21024 tablet_bootstrap.cc:654] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Neither blocks nor log segments found. Creating new log.
02:10:53.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.890532 21022 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap starting.
02:10:53.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.891264 21024 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Log is configured to *not* fsync() on all Append() calls
02:10:53.891 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:894) Opened table 57349073663d41c8b3550c16008f5ea3
02:10:53.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.892467 21022 tablet_bootstrap.cc:654] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Neither blocks nor log segments found. Creating new log.
02:10:53.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.893199 21024 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: No bootstrap required, opened a new log
02:10:53.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.893265 21022 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Log is configured to *not* fsync() on all Append() calls
02:10:53.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.893368 21024 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
02:10:53.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.894308 21023 tablet_bootstrap.cc:654] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Neither blocks nor log segments found. Creating new log.
02:10:53.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.894858 21023 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Log is configured to *not* fsync() on all Append() calls
02:10:53.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.897270 21024 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.897665 21024 raft_consensus.cc:385] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:10:53.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.897846 21024 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Initialized, Role: FOLLOWER
02:10:53.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.897897 21022 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: No bootstrap required, opened a new log
02:10:53.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.897897 21023 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: No bootstrap required, opened a new log
02:10:53.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.898061 21023 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent bootstrapping tablet: real 0.009s	user 0.003s	sys 0.000s
02:10:53.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.898061 21022 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent bootstrapping tablet: real 0.008s	user 0.003s	sys 0.000s
02:10:53.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.898729 21024 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.899719 21024 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
02:10:53.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.901471 21023 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.902078 21022 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.902422 21022 raft_consensus.cc:385] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:10:53.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.902595 21022 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Initialized, Role: FOLLOWER
02:10:53.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.903190 21022 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.904 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.904131 21022 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
02:10:53.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.906469 21023 raft_consensus.cc:385] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:10:53.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.906545 21023 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Initialized, Role: FOLLOWER
02:10:53.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.906978 21023 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.907789 21023 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
02:10:53.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.908969 20992 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:10:53.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.922752 21028 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:10:53.923 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.922906 21028 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.924 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.923890 21028 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:53.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.927443 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba" is_pre_election: true
02:10:53.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.927672 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546" is_pre_election: true
02:10:53.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.927723 20809 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 0.
02:10:53.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.927870 20944 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 0.
02:10:53.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.928146 20609 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 
02:10:53.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.928349 21028 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Leader pre-election won for term 1
02:10:53.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.928417 21028 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
02:10:53.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.928457 21028 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 0 FOLLOWER]: Advancing to term 1
02:10:53.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.929277 21028 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.929628 21028 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 1 election: Requested vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:53.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.929894 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:10:53.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.929955 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546"
02:10:53.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.929998 20809 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 0 FOLLOWER]: Advancing to term 1
02:10:53.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.930073 20944 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 0 FOLLOWER]: Advancing to term 1
02:10:53.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.930723 20809 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 1.
02:10:53.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.930970 20609 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 
02:10:53.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.931113 21028 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 FOLLOWER]: Leader election won for term 1
02:10:53.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.931119 20944 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 1.
02:10:53.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.931283 21028 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 LEADER]: Becoming Leader. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Running, Role: LEADER
02:10:53.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.931586 21028 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:53.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:53.934741 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193). New cstate: current_term: 1 leader_uuid: "8fa0772e177e429798c32dd085f59ae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:10:53.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:53.937148 20993 tablet.cc:2406] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:10:53.945 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:894) Opened table 57349073663d41c8b3550c16008f5ea3
02:10:53.961 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:53.968 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299993, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:53.969 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=null, attempt=1, sequence_id=-1, TimeoutTracker(timeout=30000, elapsed=16), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [8ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=null) after lookup
02:10:53.978 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:53.980 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] Successfully connected to peer
02:10:54.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:54.027837 20858 tablet.cc:2406] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
02:10:54.042 [DEBUG - kudu-nio-3] (Negotiator.java:919) Authenticated connection [id: 0x5351d13b, L:/127.0.0.1:48506 - R:/127.19.210.193:41573] using TOKEN/null
02:10:54.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.049530 20809 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Refusing update from remote peer 8fa0772e177e429798c32dd085f59ae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:10:54.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.049530 20944 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Refusing update from remote peer 8fa0772e177e429798c32dd085f59ae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
02:10:54.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.050213 21028 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:10:54.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.050400 21033 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
02:10:54.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.069396 21036 mvcc.cc:204] Tried to move back new op lower bound from 7305120374978891776 to 7305120374506459136. Current Snapshot: MvccSnapshot[applied={T|T < 7305120374978891776 or (T in {7305120374978891776})}]
02:10:54.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.069892 21038 mvcc.cc:204] Tried to move back new op lower bound from 7305120374978891776 to 7305120374506459136. Current Snapshot: MvccSnapshot[applied={T|T < 7305120374978891776}]
02:10:54.143 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 3 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:10:54.145 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.193:41573
02:10:54.146 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:10:54.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:54.146432 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 20591
02:10:54.153 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:10:54.155 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] handling channelInactive
02:10:54.156 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state READY due to: connection closed
02:10:54.156 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: connection closed
02:10:54.180 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:54.180 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:54.181 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=27), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test)) after lookup
02:10:54.181 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:54.183 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.184 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.184 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.200 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:54.200 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:54.201 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=47), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [28ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [31ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [31ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [47ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230))) after lookup
02:10:54.202 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:54.203 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.203 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.204 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.220 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:54.220 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:54.221 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=67), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [28ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [31ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [31ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [47ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [48ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [51ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [51ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236))) after lookup
02:10:54.221 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:54.223 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.223 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.223 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.320 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:54.320 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:54.321 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=167), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [28ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [31ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [31ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [47ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [48ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [51ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [51ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [68ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [70ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [71ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [164ms] refreshing cache from master, [164ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [166ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146))) after lookup
02:10:54.322 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:54.324 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.324 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.324 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.820 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:54.820 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:54.821 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=667), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [28ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [31ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [31ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [47ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [48ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [51ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [51ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [68ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [70ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [71ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [164ms] refreshing cache from master, [164ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [166ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [168ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [171ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [171ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [664ms] refreshing cache from master, [664ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [667ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146) -> (continuation of Deferred@1169732026 after retry RPC@298098252), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146) -> (continuation of Deferred@1169732026 after retry RPC@298098252))) after lookup
02:10:54.822 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:54.823 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.823 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:54.823 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:55.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.596141 21051 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:10:55.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.596288 21052 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:10:55.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.596287 21051 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:55.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.596388 21052 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:55.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.597491 21051 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:55.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.597497 21052 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:55.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:55.598177 20743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:55.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:55.599162 20743 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:55.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:55.599629 20878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:55.600 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:55.600468 20878 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:55.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602092 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546" is_pre_election: true
02:10:55.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602229 20944 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 1.
02:10:55.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602268 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba" is_pre_election: true
02:10:55.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602388 20809 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 1.
02:10:55.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602562 20745 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:10:55.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602807 21051 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Leader pre-election won for term 2
02:10:55.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602870 21051 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:10:55.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.602911 21051 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 1 FOLLOWER]: Advancing to term 2
02:10:55.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.603134 20880 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:10:55.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.603471 21052 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Leader pre-election won for term 2
02:10:55.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.603533 21052 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:10:55.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.603569 21052 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 1 FOLLOWER]: Advancing to term 2
02:10:55.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.603933 21051 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:55.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.604274 21051 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 2 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:55.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.604557 21052 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:55.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.604676 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546"
02:10:55.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.604822 20944 raft_consensus.cc:2393] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b4491bc31634e84a3162ba1d70813ba in current term 2: Already voted for candidate 3aea1f3e173e4803a7ae66840502f546 in this term.
02:10:55.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.604918 21052 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 2 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:55.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:55.605247 20743 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:55.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.605350 20745 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba; no voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6
02:10:55.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.605489 21051 raft_consensus.cc:2749] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
02:10:55.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.605592 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:10:55.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.605746 20809 raft_consensus.cc:2393] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3aea1f3e173e4803a7ae66840502f546 in current term 2: Already voted for candidate 3b4491bc31634e84a3162ba1d70813ba in this term.
02:10:55.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:55.606118 20878 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:55.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.606175 20878 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546; no voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6
02:10:55.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:55.606292 21052 raft_consensus.cc:2749] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
02:10:56.240 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:56.240 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:10:56.241 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=2087), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [28ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [31ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [31ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [47ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [48ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [51ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [51ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [68ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [70ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [71ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [164ms] refreshing cache from master, [164ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [166ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [168ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [171ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [171ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [664ms] refreshing cache from master, [664ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [667ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [668ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [670ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [670ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2084ms] refreshing cache from master, [2084ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [2087ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146) -> (continuation of Deferred@1169732026 after retry RPC@298098252) -> (continuation of Deferred@771985642 after retry RPC@1187879078), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146) -> (continuation of Deferred@1169732026 after retry RPC@298098252) -> (continuation of Deferred@771985642 after retry RPC@1187879078))) after lookup
02:10:56.242 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:10:56.243 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:56.243 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:56.243 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:10:57.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.350652 21061 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:10:57.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.350790 21061 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:57.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.351361 21061 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.351904 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546" is_pre_election: true
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.352051 20944 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 2.
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.352344 20745 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.352502 21061 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Leader pre-election won for term 3
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.352583 21061 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.352619 21061 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 2 FOLLOWER]: Advancing to term 3
02:10:57.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.352772 20743 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:57.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.353637 21061 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:57.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.353936 21061 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 3 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:10:57.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.354346 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546"
02:10:57.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.354477 20944 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 2 FOLLOWER]: Advancing to term 3
02:10:57.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.354839 20743 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:10:57.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.355360 20944 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 3.
02:10:57.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.355638 20745 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:10:57.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.355816 21061 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Leader election won for term 3
02:10:57.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.355955 21061 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 LEADER]: Becoming Leader. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Running, Role: LEADER
02:10:57.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.356299 21061 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:57.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.358805 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba reported cstate change: term changed from 1 to 3, leader changed from 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193) to 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194). New cstate: current_term: 3 leader_uuid: "3b4491bc31634e84a3162ba1d70813ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:10:57.720 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:57.721 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:57.722 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=14, sequence_id=4, TimeoutTracker(timeout=30000, elapsed=3567), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [3ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: connection closed, [3ms] delaying RPC due to: Network error: connection closed, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [27ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [28ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [31ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [31ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [47ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [48ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [51ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [51ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [64ms] refreshing cache from master, [64ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [66ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [68ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [70ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [71ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [164ms] refreshing cache from master, [164ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [166ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [168ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [171ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [171ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [664ms] refreshing cache from master, [664ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [667ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [668ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [670ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [670ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2084ms] refreshing cache from master, [2084ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [2087ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [2089ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2090ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2090ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [3564ms] refreshing cache from master, [3564ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [3567ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@369635754(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2e705769 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146) -> (continuation of Deferred@1169732026 after retry RPC@298098252) -> (continuation of Deferred@771985642 after retry RPC@1187879078) -> (continuation of Deferred@1593989511 after retry RPC@1965093481), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@37c42c52 -> wakeup thread Time-limited test -> (continuation of Deferred@595302218 after retry RPC@414451230) -> (continuation of Deferred@1299837030 after retry RPC@632287236) -> (continuation of Deferred@1923987506 after retry RPC@1848848146) -> (continuation of Deferred@1169732026 after retry RPC@298098252) -> (continuation of Deferred@771985642 after retry RPC@1187879078) -> (continuation of Deferred@1593989511 after retry RPC@1965093481))) after lookup
02:10:57.723 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:57.723 [DEBUG - kudu-nio-2] (Connection.java:757) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] Successfully connected to peer
02:10:57.742 [DEBUG - kudu-nio-2] (Negotiator.java:919) Authenticated connection [id: 0x3799556b, L:/127.0.0.1:35594 - R:/127.19.210.194:43205] using TOKEN/null
02:10:57.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.746730 20944 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Refusing update from remote peer 3b4491bc31634e84a3162ba1d70813ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
02:10:57.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.747331 21061 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
02:10:57.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.747488 20743 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba -> Peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Couldn't send request to peer 8fa0772e177e429798c32dd085f59ae6. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:10:57.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.754408 21077 mvcc.cc:204] Tried to move back new op lower bound from 7305120390125191168 to 7305120388534026240. Current Snapshot: MvccSnapshot[applied={T|T < 7305120390125191168}]
02:10:57.770 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.193:41573
02:10:57.772 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:10:57.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.772487 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:57.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:57.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:57.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:10:57.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:57.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:57.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:10:57.871 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.870810 21084 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:57.871 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.871032 21084 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:57.871 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.871094 21084 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:57.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.874402 21084 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:57.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.874509 21084 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.193
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.878367 21084 tablet_server_runner.cc:78] Tablet server non-default flags:
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:57.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.21084
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:57.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.879443 21084 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:57.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.880589 21084 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:57.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.886819 21089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:57.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.886821 21090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:57.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.886895 21084 server_base.cc:1061] running on GCE node
02:10:57.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:57.886914 21092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:57.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.887332 21084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:57.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.887826 21084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:57.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.888967 21084 hybrid_clock.cc:648] HybridClock initialized: now 1783476657888953 us; error 24 us; skew 500 ppm
02:10:57.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.890600 21084 webserver.cc:534] Webserver started at http://127.19.210.193:36261/ using document root <none> and password file <none>
02:10:57.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.891134 21084 fs_manager.cc:362] Metadata directory not provided
02:10:57.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.891208 21084 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:57.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.894656 21084 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.000s
02:10:57.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.896481 21098 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:57.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.897399 21084 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:10:57.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.897527 21084 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:57.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.897886 21084 fs_report.cc:389] FS layout report
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/data
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:57.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:57.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.929096 21084 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:57.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.929790 21084 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:57.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.929931 21084 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:57.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.930540 21084 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:10:57.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.931756 21105 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:10:57.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.934973 21084 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:10:57.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.935036 21084 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
02:10:57.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.935089 21084 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:10:57.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.936830 21084 ts_tablet_manager.cc:616] Registered 1 tablets
02:10:57.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.937036 21084 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
02:10:57.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.937891 21105 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap starting.
02:10:57.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.949087 21084 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.193:41573
02:10:57.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.951251 21212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.193:41573 every 8 connection(s)
02:10:57.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.952436 21084 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:10:57.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.957680 21105 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Log is configured to *not* fsync() on all Append() calls
02:10:57.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.958343 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 21084
02:10:57.958 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:10:57.968 [DEBUG - kudu-nio-2] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 6 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:10:57.970 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.194:43205
02:10:57.970 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

02:10:57.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.969117 21214 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:10:57.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.969374 21214 heartbeater.cc:461] Registering TS with master...
02:10:57.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.969944 21214 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:10:57.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.970809 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 20727
02:10:57.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.973599 21213 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:10:57.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.973657 20345 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:10:57.974 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.974468 21213 heartbeater.cc:461] Registering TS with master...
02:10:57.974 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.974815 21213 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:10:57.979 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:10:57.979 [DEBUG - kudu-nio-2] (Connection.java:239) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] handling channelInactive
02:10:57.979 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state READY due to: connection closed
02:10:57.979 [DEBUG - Time-limited test] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:57.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.976156 20490 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:10:57.981 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:57.981 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:57.981 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:57.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.982189 21215 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:10:57.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.982291 21215 heartbeater.cc:461] Registering TS with master...
02:10:57.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.982523 21215 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:10:57.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.982549 21105 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:10:57.983 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.982930 21105 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap complete.
02:10:57.983 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.983450 20417 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:10:57.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.984306 21105 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.008s
02:10:57.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.984375 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.193:34857
02:10:57.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.987051 21105 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:57.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.987651 21105 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Initialized, Role: FOLLOWER
02:10:57.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.988279 21105 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:57.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.989399 21105 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
02:10:57.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:57.989981 21215 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:10:58.000 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:58.000 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:58.000 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=21), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [2ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [18ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [20ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@73731258(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@76a415e5 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@874e3c7 -> wakeup thread Time-limited test)) after lookup
02:10:58.001 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:58.002 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.002 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.002 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.020 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:58.020 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:58.021 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=41), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [2ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [18ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [20ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [21ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [23ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [23ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [40ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@73731258(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@76a415e5 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@874e3c7 -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005))) after lookup
02:10:58.021 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:58.022 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.023 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.023 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.060 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:58.061 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:58.061 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=82), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [2ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [18ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [20ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [21ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [23ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [23ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [40ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [42ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [44ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [78ms] refreshing cache from master, [78ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [81ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@73731258(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@76a415e5 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@874e3c7 -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796))) after lookup
02:10:58.062 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:58.063 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.063 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.063 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.100 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:58.101 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:58.101 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=122), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [2ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [18ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [20ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [21ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [23ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [23ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [40ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [42ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [44ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [78ms] refreshing cache from master, [78ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [81ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [83ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [84ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [85ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [118ms] refreshing cache from master, [118ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [121ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@73731258(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@76a415e5 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796) -> (continuation of Deferred@488126442 after retry RPC@942739035), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@874e3c7 -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796) -> (continuation of Deferred@488126442 after retry RPC@942739035))) after lookup
02:10:58.102 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:58.104 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.104 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.104 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.540 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:58.541 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:58.541 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=562), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [2ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [18ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [20ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [21ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [23ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [23ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [40ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [42ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [44ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [78ms] refreshing cache from master, [78ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [81ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [83ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [84ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [85ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [118ms] refreshing cache from master, [118ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [121ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [123ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [125ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [125ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [558ms] refreshing cache from master, [559ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [561ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@73731258(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@76a415e5 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796) -> (continuation of Deferred@488126442 after retry RPC@942739035) -> (continuation of Deferred@1811137173 after retry RPC@353316989), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@874e3c7 -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796) -> (continuation of Deferred@488126442 after retry RPC@942739035) -> (continuation of Deferred@1811137173 after retry RPC@353316989))) after lookup
02:10:58.542 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:10:58.543 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.543 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:58.543 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:10:59.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.270601 21226 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b4491bc31634e84a3162ba1d70813ba)
02:10:59.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.270752 21226 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:59.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.271267 21226 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:59.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.272511 20880 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:10:59.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.276018 21164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6" is_pre_election: true
02:10:59.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.276322 21164 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 1.
02:10:59.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.277062 20878 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6; no voters: 3b4491bc31634e84a3162ba1d70813ba
02:10:59.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.277244 21226 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Leader pre-election won for term 4
02:10:59.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.277310 21226 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b4491bc31634e84a3162ba1d70813ba)
02:10:59.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.277341 21226 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 3 FOLLOWER]: Advancing to term 4
02:10:59.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.278115 21226 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:59.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.278478 21226 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 4 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:59.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.278829 21164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6"
02:10:59.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.278970 21164 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 1 FOLLOWER]: Advancing to term 4
02:10:59.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.279783 20880 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:10:59.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.280884 21164 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 4.
02:10:59.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.281193 20878 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6; no voters: 3b4491bc31634e84a3162ba1d70813ba
02:10:59.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.281363 21226 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 FOLLOWER]: Leader election won for term 4
02:10:59.281 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.281595 21226 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 LEADER]: Becoming Leader. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Running, Role: LEADER
02:10:59.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.281908 21226 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:59.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.284152 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 reported cstate change: term changed from 3 to 4, leader changed from 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194) to 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195). New cstate: current_term: 4 leader_uuid: "3aea1f3e173e4803a7ae66840502f546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: HEALTHY } } }
02:10:59.640 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:59.641 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:59.641 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=7, TimeoutTracker(timeout=30000, elapsed=1662), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [2ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [18ms] refreshing cache from master, [18ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [20ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [21ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [23ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [23ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] refreshing cache from master, [38ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [40ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [42ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [44ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [44ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [78ms] refreshing cache from master, [78ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [81ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [83ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [84ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [85ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [118ms] refreshing cache from master, [118ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [121ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [123ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [125ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [125ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [558ms] refreshing cache from master, [559ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [561ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [562ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [564ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [564ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [1658ms] refreshing cache from master, [1658ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [1661ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@73731258(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@76a415e5 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796) -> (continuation of Deferred@488126442 after retry RPC@942739035) -> (continuation of Deferred@1811137173 after retry RPC@353316989) -> (continuation of Deferred@71392203 after retry RPC@84834832), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@874e3c7 -> wakeup thread Time-limited test -> (continuation of Deferred@248287800 after retry RPC@554251005) -> (continuation of Deferred@30397114 after retry RPC@1765381796) -> (continuation of Deferred@488126442 after retry RPC@942739035) -> (continuation of Deferred@1811137173 after retry RPC@353316989) -> (continuation of Deferred@71392203 after retry RPC@84834832))) after lookup
02:10:59.642 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:10:59.643 [DEBUG - kudu-nio-1] (Connection.java:757) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] Successfully connected to peer
02:10:59.659 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x894d2bcd, L:/127.0.0.1:49282 - R:/127.19.210.195:42923] using TOKEN/null
02:10:59.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.663420 21164 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Refusing update from remote peer 3aea1f3e173e4803a7ae66840502f546: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 10. (index mismatch)
02:10:59.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.664002 20880 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 -> Peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Couldn't send request to peer 3b4491bc31634e84a3162ba1d70813ba. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:10:59.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.664091 21226 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.000s
02:10:59.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.668592 21246 mvcc.cc:204] Tried to move back new op lower bound from 7305120390188072960 to 7305120388534026240. Current Snapshot: MvccSnapshot[applied={T|T < 7305120390125191168}]
02:10:59.691 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.194:43205
02:10:59.692 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.693218 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:43205
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39205
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:10:59.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.794149 21253 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:10:59.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.794366 21253 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:10:59.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.794426 21253 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:10:59.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.797878 21253 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:10:59.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.797999 21253 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.194
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.801887 21253 tablet_server_runner.cc:78] Tablet server non-default flags:
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:43205
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39205
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.21253
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:10:59.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:10:59.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.803352 21253 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:10:59.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.804451 21253 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:10:59.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.810755 21261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:59.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.810756 21258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:59.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:10:59.810756 21259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:10:59.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.810801 21253 server_base.cc:1061] running on GCE node
02:10:59.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.811803 21253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:10:59.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.812368 21253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:10:59.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.813532 21253 hybrid_clock.cc:648] HybridClock initialized: now 1783476659813512 us; error 32 us; skew 500 ppm
02:10:59.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.815359 21253 webserver.cc:534] Webserver started at http://127.19.210.194:39205/ using document root <none> and password file <none>
02:10:59.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.815956 21253 fs_manager.cc:362] Metadata directory not provided
02:10:59.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.816027 21253 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:10:59.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.819407 21253 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
02:10:59.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.821175 21268 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.822090 21253 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.822181 21253 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.822552 21253 fs_report.cc:389] FS layout report
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/data
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:10:59.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:10:59.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:10:59.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.840044 21253 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:10:59.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.840734 21253 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:10:59.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.840883 21253 kserver.cc:163] Server-wide thread pool size limit: 3276
02:10:59.841 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.841375 21253 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:10:59.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.842562 21275 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:10:59.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.845806 21253 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:10:59.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.845861 21253 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
02:10:59.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.845908 21253 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:10:59.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.847301 21253 ts_tablet_manager.cc:616] Registered 1 tablets
02:10:59.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.847345 21253 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
02:10:59.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.847715 21275 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap starting.
02:10:59.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.857440 21253 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.194:43205
02:10:59.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.857501 21382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.194:43205 every 8 connection(s)
02:10:59.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.858819 21253 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:10:59.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.862650 21275 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Log is configured to *not* fsync() on all Append() calls
02:10:59.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.868724 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 21253
02:10:59.869 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.869400 21384 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.869412 21385 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.869694 21384 heartbeater.cc:461] Registering TS with master...
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.869692 21385 heartbeater.cc:461] Registering TS with master...
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.870301 21385 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.870362 21383 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.870441 21383 heartbeater.cc:461] Registering TS with master...
02:10:59.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.870620 21383 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:10:59.871 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.870302 21384 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:10:59.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.872304 20490 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:59.873 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.873045 20345 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:59.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.874398 20417 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:10:59.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.875146 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.194:40475
02:10:59.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.875727 21275 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=6 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:10:59.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.876108 21275 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap complete.
02:10:59.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.877399 21275 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.013s
02:10:59.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.880153 21275 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:59.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.880736 21275 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Initialized, Role: FOLLOWER
02:10:59.880 [DEBUG - kudu-nio-1] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 9 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:10:59.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.881347 21275 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:10:59.881 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.195:42923
02:10:59.882 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

02:10:59.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.882236 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 20862
02:10:59.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.882869 21275 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
02:10:59.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:10:59.883275 21385 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:10:59.888 [DEBUG - kudu-nio-1] (Connection.java:239) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] handling channelInactive
02:10:59.888 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:10:59.889 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state READY due to: connection closed
02:10:59.889 [DEBUG - Time-limited test] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:10:59.890 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.890 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.890 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.900 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:59.900 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:59.900 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=11), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [1ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1702374220(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@13500309 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@541c1f1f -> wakeup thread Time-limited test)) after lookup
02:10:59.901 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:10:59.902 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.902 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.902 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.920 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:59.920 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:59.921 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=31), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [1ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [13ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1702374220(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@13500309 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@541c1f1f -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377))) after lookup
02:10:59.921 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:10:59.922 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.923 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.923 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.940 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:10:59.941 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:10:59.941 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=52), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [1ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [13ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [34ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1702374220(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@13500309 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@541c1f1f -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103))) after lookup
02:10:59.942 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:10:59.943 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.943 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:10:59.943 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:00.021 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:00.022 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:00.022 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=133), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [1ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [13ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [34ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [54ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [54ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [131ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1702374220(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@13500309 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103) -> (continuation of Deferred@220353431 after retry RPC@971293969), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@541c1f1f -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103) -> (continuation of Deferred@220353431 after retry RPC@971293969))) after lookup
02:11:00.023 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:00.024 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:00.024 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:00.024 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:00.340 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:00.340 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:00.341 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=451), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [1ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [13ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [34ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [54ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [54ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [131ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [133ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [135ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [135ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [448ms] refreshing cache from master, [448ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [451ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1702374220(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@13500309 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103) -> (continuation of Deferred@220353431 after retry RPC@971293969) -> (continuation of Deferred@572062679 after retry RPC@1028357832), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@541c1f1f -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103) -> (continuation of Deferred@220353431 after retry RPC@971293969) -> (continuation of Deferred@572062679 after retry RPC@1028357832))) after lookup
02:11:00.341 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:00.342 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:00.342 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:00.342 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:01.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.044256 21396 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:11:01.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.044425 21396 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:01.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.045543 21396 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:01.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.046219 21272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:01.047 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.047155 21272 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:01.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.049379 21164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6" is_pre_election: true
02:11:01.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.049894 21270 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba; no voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6
02:11:01.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.050158 21396 raft_consensus.cc:2749] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
02:11:01.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.191416 21401 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3aea1f3e173e4803a7ae66840502f546)
02:11:01.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.191591 21401 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:01.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.192730 21401 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:01.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.193763 21102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:01.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.194643 21102 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:01.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.197119 21337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba" is_pre_election: true
02:11:01.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.197314 21337 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 3.
02:11:01.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.197924 21102 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 3aea1f3e173e4803a7ae66840502f546
02:11:01.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.198146 21401 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Leader pre-election won for term 5
02:11:01.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.198210 21401 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3aea1f3e173e4803a7ae66840502f546)
02:11:01.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.198249 21401 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 4 FOLLOWER]: Advancing to term 5
02:11:01.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.199076 21401 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:01.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.199424 21401 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 5 election: Requested vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:01.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.199774 21337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 5 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:11:01.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.199900 21337 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 3 FOLLOWER]: Advancing to term 5
02:11:01.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.200512 21102 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:01.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.201571 21337 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 5.
02:11:01.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.201862 21102 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 3aea1f3e173e4803a7ae66840502f546
02:11:01.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.202014 21401 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 FOLLOWER]: Leader election won for term 5
02:11:01.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.202181 21401 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 LEADER]: Becoming Leader. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Running, Role: LEADER
02:11:01.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.202438 21401 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:01.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.204882 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 reported cstate change: term changed from 4 to 5, leader changed from 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195) to 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193). New cstate: current_term: 5 leader_uuid: "8fa0772e177e429798c32dd085f59ae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:11:01.380 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:01.380 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:01.380 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=10, TimeoutTracker(timeout=30000, elapsed=1491), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [1ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [11ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [13ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [13ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [34ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [51ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [54ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [54ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [128ms] refreshing cache from master, [128ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [131ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [133ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [135ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [135ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [448ms] refreshing cache from master, [448ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [451ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [452ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [453ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [454ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [1488ms] refreshing cache from master, [1488ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [1490ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1702374220(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@13500309 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103) -> (continuation of Deferred@220353431 after retry RPC@971293969) -> (continuation of Deferred@572062679 after retry RPC@1028357832) -> (continuation of Deferred@1095372811 after retry RPC@655465428), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@541c1f1f -> wakeup thread Time-limited test -> (continuation of Deferred@1163507522 after retry RPC@1896463377) -> (continuation of Deferred@1858673979 after retry RPC@1386012103) -> (continuation of Deferred@220353431 after retry RPC@971293969) -> (continuation of Deferred@572062679 after retry RPC@1028357832) -> (continuation of Deferred@1095372811 after retry RPC@655465428))) after lookup
02:11:01.381 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.382 [DEBUG - kudu-nio-0] (Connection.java:757) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] Successfully connected to peer
02:11:01.399 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0xe731fe5c, L:/127.0.0.1:57398 - R:/127.19.210.193:41573] using TOKEN/null
02:11:01.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.403515 21337 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Refusing update from remote peer 8fa0772e177e429798c32dd085f59ae6: Log matching property violated. Preceding OpId in replica: term: 3 index: 8. Preceding OpId from leader: term: 5 index: 14. (index mismatch)
02:11:01.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.404294 21102 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 -> Peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Couldn't send request to peer 3aea1f3e173e4803a7ae66840502f546. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:01.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.404333 21401 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.000s
02:11:01.428 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.195:42923
02:11:01.428 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.429298 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:01.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:42923
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44723
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:01.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:01.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.528741 21426 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:01.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.528926 21426 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:01.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.528980 21426 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:01.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.532336 21426 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:01.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.532446 21426 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.195
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.536151 21426 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:42923
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:11:01.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44723
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.21426
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:01.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.537532 21426 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:01.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.538596 21426 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:01.544 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.544709 21431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:01.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.544703 21432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:01.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:01.544703 21434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:01.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.545051 21426 server_base.cc:1061] running on GCE node
02:11:01.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.545347 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:01.546 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.545862 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:01.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.547014 21426 hybrid_clock.cc:648] HybridClock initialized: now 1783476661546995 us; error 26 us; skew 500 ppm
02:11:01.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.548748 21426 webserver.cc:534] Webserver started at http://127.19.210.195:44723/ using document root <none> and password file <none>
02:11:01.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.549301 21426 fs_manager.cc:362] Metadata directory not provided
02:11:01.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.549374 21426 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:01.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.552564 21426 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.003s
02:11:01.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.554342 21440 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:01.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.555219 21426 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:11:01.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.555318 21426 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:01.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3aea1f3e173e4803a7ae66840502f546"
02:11:01.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.555743 21426 fs_report.cc:389] FS layout report
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/data
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:01.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:01.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.574980 21426 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:01.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.575696 21426 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:01.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.575871 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:01.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.576463 21426 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:01.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.577786 21447 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:01.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.581025 21426 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:01.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.581079 21426 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.001s	sys 0.000s
02:11:01.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.581125 21426 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:01.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.582506 21426 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:01.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.582557 21426 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
02:11:01.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.582891 21447 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap starting.
02:11:01.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.593663 21426 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.195:42923
02:11:01.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.593676 21554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.195:42923 every 8 connection(s)
02:11:01.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.594988 21426 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:01.601 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.601550 21447 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Log is configured to *not* fsync() on all Append() calls
02:11:01.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.604458 21556 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:01.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.604704 21556 heartbeater.cc:461] Registering TS with master...
02:11:01.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.605006 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 21426
02:11:01.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.605252 21556 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:01.605 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:01.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.607003 20345 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:01.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.607249 21557 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:01.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.607326 21557 heartbeater.cc:461] Registering TS with master...
02:11:01.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.607517 21557 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:01.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.608268 20417 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:01.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.608937 21555 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:01.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.609009 21555 heartbeater.cc:461] Registering TS with master...
02:11:01.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.609166 21555 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:01.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.609201 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.195:40307
02:11:01.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.612355 20489 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:01.614 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 12 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:01.620 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.193:41573
02:11:01.620 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:11:01.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.621116 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 21084
02:11:01.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.624982 21447 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:01.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.625413 21447 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap complete.
02:11:01.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.626714 21447 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent bootstrapping tablet: real 0.044s	user 0.030s	sys 0.000s
02:11:01.628 [DEBUG - kudu-nio-0] (Connection.java:239) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] handling channelInactive
02:11:01.628 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state READY due to: connection closed
02:11:01.629 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:01.629 [DEBUG - Time-limited test] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.629585 21447 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:01.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.630038 21447 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Initialized, Role: FOLLOWER
02:11:01.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.630539 21447 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 4.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:01.631 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.631 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.631 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.631472 21447 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
02:11:01.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:01.631788 21557 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:01.639 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:01.640 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:01.640 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=11), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test)) after lookup
02:11:01.641 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.642 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.642 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.642 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.660 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:01.660 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:01.661 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=31), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [13ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [13ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279))) after lookup
02:11:01.661 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.662 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.663 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.663 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.680 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:01.680 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:01.680 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=51), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [13ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [13ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [34ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [34ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [50ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773))) after lookup
02:11:01.681 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.682 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.682 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.682 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.700 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:01.700 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:01.700 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=71), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [13ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [13ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [34ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [34ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [50ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [53ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [53ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [68ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [70ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234))) after lookup
02:11:01.701 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.702 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.702 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.702 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.980 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:01.981 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:01.981 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=351), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [13ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [13ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [34ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [34ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [50ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [53ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [53ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [68ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [70ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [71ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [73ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [73ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [348ms] refreshing cache from master, [348ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [351ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234) -> (continuation of Deferred@698642909 after retry RPC@1872646450), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234) -> (continuation of Deferred@698642909 after retry RPC@1872646450))) after lookup
02:11:01.981 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:01.983 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.983 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:01.983 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:02.260 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:02.260 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:02.261 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=631), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [13ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [13ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [34ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [34ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [50ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [53ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [53ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [68ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [70ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [71ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [73ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [73ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [348ms] refreshing cache from master, [348ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [351ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [352ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [354ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [354ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [628ms] refreshing cache from master, [628ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [631ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234) -> (continuation of Deferred@698642909 after retry RPC@1872646450) -> (continuation of Deferred@843580773 after retry RPC@801293791), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234) -> (continuation of Deferred@698642909 after retry RPC@1872646450) -> (continuation of Deferred@843580773 after retry RPC@801293791))) after lookup
02:11:02.261 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:02.262 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:02.262 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:02.263 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:02.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.927691 21569 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:11:02.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.927834 21569 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:02.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.928232 21569 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:02.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:02.929576 21270 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:02.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.933079 21509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546" is_pre_election: true
02:11:02.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.933377 21509 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 4.
02:11:02.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.934068 21272 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:11:02.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.934247 21569 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Leader pre-election won for term 6
02:11:02.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.934305 21569 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:11:02.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.934333 21569 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 5 FOLLOWER]: Advancing to term 6
02:11:02.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.935063 21569 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:02.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.935391 21569 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 6 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:02.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.935774 21509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546"
02:11:02.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.935899 21509 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 4 FOLLOWER]: Advancing to term 6
02:11:02.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:02.936384 21270 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:02.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.937597 21509 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 6.
02:11:02.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.937875 21272 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:11:02.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.938045 21569 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 FOLLOWER]: Leader election won for term 6
02:11:02.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.938228 21569 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 LEADER]: Becoming Leader. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Running, Role: LEADER
02:11:02.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.938544 21569 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:02.941 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:02.940779 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba reported cstate change: term changed from 5 to 6, leader changed from 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193) to 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194). New cstate: current_term: 6 leader_uuid: "3b4491bc31634e84a3162ba1d70813ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:11:03.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:03.374565 21270 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba -> Peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Couldn't send request to peer 8fa0772e177e429798c32dd085f59ae6. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:03.513 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:03.513566 21509 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Refusing update from remote peer 3b4491bc31634e84a3162ba1d70813ba: Log matching property violated. Preceding OpId in replica: term: 4 index: 12. Preceding OpId from leader: term: 6 index: 17. (index mismatch)
02:11:03.514 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:03.514262 21569 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 12, Time since last communication: 0.000s
02:11:05.660 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:05.661 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:05.661 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=14, sequence_id=13, TimeoutTracker(timeout=30000, elapsed=4032), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [8ms] refreshing cache from master, [8ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [10ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [12ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [13ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [13ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] refreshing cache from master, [28ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [31ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [32ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [34ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [34ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [48ms] refreshing cache from master, [48ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [50ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [52ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [53ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [53ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [68ms] refreshing cache from master, [68ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [70ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [71ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [73ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [73ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [348ms] refreshing cache from master, [348ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [351ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [352ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [354ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [354ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [628ms] refreshing cache from master, [628ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [631ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [632ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [634ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [634ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [4028ms] refreshing cache from master, [4028ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [4031ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@574187510(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@15217ded -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234) -> (continuation of Deferred@698642909 after retry RPC@1872646450) -> (continuation of Deferred@843580773 after retry RPC@801293791) -> (continuation of Deferred@1405117011 after retry RPC@1779808713), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@1f4eb53d -> wakeup thread Time-limited test -> (continuation of Deferred@609834364 after retry RPC@1314192279) -> (continuation of Deferred@929705404 after retry RPC@1577840773) -> (continuation of Deferred@1170081384 after retry RPC@1144015234) -> (continuation of Deferred@698642909 after retry RPC@1872646450) -> (continuation of Deferred@843580773 after retry RPC@801293791) -> (continuation of Deferred@1405117011 after retry RPC@1779808713))) after lookup
02:11:05.662 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:05.663 [DEBUG - kudu-nio-0] (Connection.java:757) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] Successfully connected to peer
02:11:05.680 [DEBUG - kudu-nio-0] (Negotiator.java:919) Authenticated connection [id: 0x42b7e3bc, L:/127.0.0.1:48260 - R:/127.19.210.194:43205] using TOKEN/null
02:11:05.698 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.193:41573
02:11:05.698 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.699214 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:11:05.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:11:05.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:05.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:05.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:05.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:05.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.799180 21609 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:05.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.799378 21609 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:05.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.799420 21609 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:05.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.802686 21609 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:05.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.802793 21609 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.193
02:11:05.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.806679 21609 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:05.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:05.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.21609
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:05.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:05.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.807940 21609 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:05.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.809415 21609 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:05.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.816113 21615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:05.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.816113 21617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:05.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.816159 21614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:05.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.816326 21609 server_base.cc:1061] running on GCE node
02:11:05.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.816839 21609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:05.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.817297 21609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:05.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.818439 21609 hybrid_clock.cc:648] HybridClock initialized: now 1783476665818421 us; error 35 us; skew 500 ppm
02:11:05.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.820026 21609 webserver.cc:534] Webserver started at http://127.19.210.193:36261/ using document root <none> and password file <none>
02:11:05.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.820503 21609 fs_manager.cc:362] Metadata directory not provided
02:11:05.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.820586 21609 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:05.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.823705 21609 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.000s	sys 0.002s
02:11:05.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.825405 21623 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:05.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.826366 21609 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
02:11:05.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.826453 21609 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:05.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:11:05.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.826843 21609 fs_report.cc:389] FS layout report
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/data
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:05.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:05.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.846698 21609 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:05.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.847330 21609 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:05.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.847484 21609 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:05.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.847990 21609 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:05.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.849201 21630 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:05.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.852438 21609 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:05.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.852494 21609 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
02:11:05.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.852571 21609 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:05.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.853928 21609 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:05.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.853981 21609 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
02:11:05.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.854311 21630 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap starting.
02:11:05.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.863828 21609 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.193:41573
02:11:05.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.863873 21738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.193:41573 every 8 connection(s)
02:11:05.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.864926 21609 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:05.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.872014 21630 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Log is configured to *not* fsync() on all Append() calls
02:11:05.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.874433 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 21609
02:11:05.874 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:05.884 [DEBUG - kudu-nio-0] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 15 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:05.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.883494 21741 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:05.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.883715 21741 heartbeater.cc:461] Registering TS with master...
02:11:05.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.884282 21741 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:05.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.885409 21739 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:05.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.885623 21739 heartbeater.cc:461] Registering TS with master...
02:11:05.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:05.886015 21270 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba -> Peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Couldn't send request to peer 8fa0772e177e429798c32dd085f59ae6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
02:11:05.886 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.194:43205
02:11:05.886 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

02:11:05.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.886704 20417 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:05.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.886865 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 21253
02:11:05.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.887593 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.193:51241
02:11:05.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.888016 21739 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:05.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.888337 21740 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:05.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.888408 21740 heartbeater.cc:461] Registering TS with master...
02:11:05.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.889078 20489 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:05.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.889690 21740 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:05.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.890709 20337 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:05.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.892966 21630 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:05.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.893263 21630 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap complete.
02:11:05.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.894302 21630 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent bootstrapping tablet: real 0.040s	user 0.030s	sys 0.000s
02:11:05.894 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:05.894 [DEBUG - kudu-nio-0] (Connection.java:239) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] handling channelInactive
02:11:05.894 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state READY due to: connection closed
02:11:05.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.896391 21630 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:05.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.896727 21630 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Initialized, Role: FOLLOWER
02:11:05.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.897084 21630 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:05.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.897710 21741 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:05.898 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:05.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:05.897812 21630 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent starting tablet: real 0.003s	user 0.003s	sys 0.000s
02:11:05.899 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.899 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.899 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.919 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:05.919 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:05.919 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=3, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=24), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1ms] delaying RPC due to: Illegal state: connection is terminated, [3ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [5ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [5ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@207345576(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6cd9316c -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2a12126f -> wakeup thread Time-limited test)) after lookup
02:11:05.920 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:05.921 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.921 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.921 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.940 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:05.940 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:05.941 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=5, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=46), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1ms] delaying RPC due to: Illegal state: connection is terminated, [3ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [5ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [5ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [27ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [43ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@207345576(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6cd9316c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2a12126f -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783))) after lookup
02:11:05.941 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:05.942 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.943 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.943 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.980 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:05.980 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:05.984 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=7, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=85), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1ms] delaying RPC due to: Illegal state: connection is terminated, [3ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [5ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [5ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [27ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [43ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [49ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [49ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [83ms] refreshing cache from master, [83ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@207345576(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6cd9316c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2a12126f -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668))) after lookup
02:11:05.985 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:05.986 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.986 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:05.986 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:06.059 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:06.060 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:06.060 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=9, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=165), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1ms] delaying RPC due to: Illegal state: connection is terminated, [3ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [5ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [5ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [27ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [43ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [49ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [49ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [83ms] refreshing cache from master, [83ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [90ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [92ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [92ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [162ms] refreshing cache from master, [163ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [165ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@207345576(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6cd9316c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668) -> (continuation of Deferred@238437165 after retry RPC@1218359411), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2a12126f -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668) -> (continuation of Deferred@238437165 after retry RPC@1218359411))) after lookup
02:11:06.061 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:06.062 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:06.062 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:06.062 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:06.979 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:06.980 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:06.980 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=11, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=1085), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1ms] delaying RPC due to: Illegal state: connection is terminated, [3ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [5ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [5ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [27ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [43ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [49ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [49ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [83ms] refreshing cache from master, [83ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [90ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [92ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [92ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [162ms] refreshing cache from master, [163ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [165ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [167ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [168ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [168ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [1083ms] refreshing cache from master, [1083ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [1085ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@207345576(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6cd9316c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668) -> (continuation of Deferred@238437165 after retry RPC@1218359411) -> (continuation of Deferred@513082018 after retry RPC@590689323), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2a12126f -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668) -> (continuation of Deferred@238437165 after retry RPC@1218359411) -> (continuation of Deferred@513082018 after retry RPC@590689323))) after lookup
02:11:06.981 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:06.982 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:06.982 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:06.983 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:07.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.198520 21755 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b4491bc31634e84a3162ba1d70813ba)
02:11:07.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.198659 21755 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:07.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.199671 21755 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:07.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:07.200380 21444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:11:07.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:07.201242 21444 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:11:07.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.203384 21692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6" is_pre_election: true
02:11:07.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.203639 21692 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 5.
02:11:07.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.204008 21442 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6; no voters: 3b4491bc31634e84a3162ba1d70813ba
02:11:07.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.204218 21755 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Leader pre-election won for term 7
02:11:07.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.204286 21755 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3b4491bc31634e84a3162ba1d70813ba)
02:11:07.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.204329 21755 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 6 FOLLOWER]: Advancing to term 7
02:11:07.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.205209 21755 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:07.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.205601 21755 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 7 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:07.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.205978 21692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 7 candidate_status { last_received { term: 6 index: 20 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6"
02:11:07.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.206135 21692 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 5 FOLLOWER]: Advancing to term 7
02:11:07.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:07.206589 21444 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:11:07.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.207858 21692 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 7.
02:11:07.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.208182 21442 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6; no voters: 3b4491bc31634e84a3162ba1d70813ba
02:11:07.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.208364 21755 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 FOLLOWER]: Leader election won for term 7
02:11:07.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.208649 21755 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 LEADER]: Becoming Leader. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Running, Role: LEADER
02:11:07.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.208976 21755 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 6.20, Last appended by leader: 20, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:07.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.211659 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 reported cstate change: term changed from 6 to 7, leader changed from 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194) to 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195). New cstate: current_term: 7 leader_uuid: "3aea1f3e173e4803a7ae66840502f546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: HEALTHY } } }
02:11:07.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:07.614122 21444 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 -> Peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Couldn't send request to peer 3b4491bc31634e84a3162ba1d70813ba. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:07.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.739729 21692 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Refusing update from remote peer 3aea1f3e173e4803a7ae66840502f546: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 7 index: 21. (index mismatch)
02:11:07.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:07.740573 21755 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 16, Time since last communication: 0.000s
02:11:09.599 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:09.600 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:09.600 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=13, sequence_id=16, TimeoutTracker(timeout=30000, elapsed=3705), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1ms] delaying RPC due to: Illegal state: connection is terminated, [3ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [5ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [5ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [22ms] refreshing cache from master, [22ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [25ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [27ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [27ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [43ms] refreshing cache from master, [43ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [49ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [49ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [83ms] refreshing cache from master, [83ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [85ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [90ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [92ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [92ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [162ms] refreshing cache from master, [163ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [165ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [167ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [168ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [168ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [1083ms] refreshing cache from master, [1083ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [1085ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [1087ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [1088ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [1089ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [3702ms] refreshing cache from master, [3703ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [3705ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@207345576(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6cd9316c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668) -> (continuation of Deferred@238437165 after retry RPC@1218359411) -> (continuation of Deferred@513082018 after retry RPC@590689323) -> (continuation of Deferred@1153855684 after retry RPC@1314422811), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@2a12126f -> wakeup thread Time-limited test -> (continuation of Deferred@428401915 after retry RPC@598936783) -> (continuation of Deferred@216046643 after retry RPC@923675668) -> (continuation of Deferred@238437165 after retry RPC@1218359411) -> (continuation of Deferred@513082018 after retry RPC@590689323) -> (continuation of Deferred@1153855684 after retry RPC@1314422811))) after lookup
02:11:09.600 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:09.601 [DEBUG - kudu-nio-7] (Connection.java:757) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] Successfully connected to peer
02:11:09.626 [DEBUG - kudu-nio-7] (Negotiator.java:919) Authenticated connection [id: 0xe7a07247, L:/127.0.0.1:46054 - R:/127.19.210.195:42923] using TOKEN/null
02:11:09.654 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.194:43205
02:11:09.655 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.656225 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:09.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:43205
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39205
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:09.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:09.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.800396 21793 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:09.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.800643 21793 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:09.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.800707 21793 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:09.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.804246 21793 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:09.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.804379 21793 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.194
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.808327 21793 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:43205
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:09.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39205
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.21793
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:09.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:09.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.809886 21793 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:09.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.811105 21793 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:09.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.817903 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:09.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.817914 21802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:09.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:09.818205 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:09.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.818636 21793 server_base.cc:1061] running on GCE node
02:11:09.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.819010 21793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:09.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.819515 21793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:09.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.820659 21793 hybrid_clock.cc:648] HybridClock initialized: now 1783476669820650 us; error 25 us; skew 500 ppm
02:11:09.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.822409 21793 webserver.cc:534] Webserver started at http://127.19.210.194:39205/ using document root <none> and password file <none>
02:11:09.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.822958 21793 fs_manager.cc:362] Metadata directory not provided
02:11:09.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.823017 21793 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:09.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.826172 21793 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
02:11:09.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.828080 21808 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.829037 21793 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.829141 21793 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.829556 21793 fs_report.cc:389] FS layout report
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/data
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:09.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:09.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:09.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.846006 21793 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:09.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.846678 21793 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:09.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.846843 21793 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:09.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.847429 21793 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:09.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.848766 21815 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:09.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.852159 21793 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:09.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.852227 21793 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
02:11:09.852 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.852293 21793 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:09.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.853863 21793 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:09.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.853909 21793 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
02:11:09.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.854218 21815 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap starting.
02:11:09.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.865154 21793 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.194:43205
02:11:09.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.865203 21922 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.194:43205 every 8 connection(s)
02:11:09.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.866294 21793 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:11:09.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.870002 21815 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Log is configured to *not* fsync() on all Append() calls
02:11:09.872 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:09.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.872000 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 21793
02:11:09.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.879398 21925 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:09.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.879654 21925 heartbeater.cc:461] Registering TS with master...
02:11:09.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.880208 21925 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:09.881 [DEBUG - kudu-nio-7] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 18 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:09.882 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.195:42923
02:11:09.882 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

02:11:09.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.882007 20417 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:09.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.882313 21923 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:09.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.882383 21923 heartbeater.cc:461] Registering TS with master...
02:11:09.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.882927 20417 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.194:56879
02:11:09.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.884231 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 21426
02:11:09.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.888625 21923 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:09.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.888746 21815 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=0 applied=20 ignored=0} inserts{seen=15 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:09.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.889137 21815 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap complete.
02:11:09.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.889634 20489 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:09.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.890225 21815 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent bootstrapping tablet: real 0.036s	user 0.022s	sys 0.010s
02:11:09.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.891671 21924 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:09.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.891753 21924 heartbeater.cc:461] Registering TS with master...
02:11:09.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.891950 21924 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:09.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.892815 20337 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:09.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.892840 21815 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:09.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.893229 21815 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Initialized, Role: FOLLOWER
02:11:09.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.893693 21815 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 6.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:09.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.894588 21815 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
02:11:09.894 [DEBUG - kudu-nio-7] (Connection.java:239) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] handling channelInactive
02:11:09.895 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state READY due to: connection closed
02:11:09.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:09.895313 21925 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:09.895 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:09.896 [DEBUG - Time-limited test] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:09.897 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.898 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.898 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.920 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:09.920 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:09.920 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=24), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@819385650(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6307b5fc -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@21977c19 -> wakeup thread Time-limited test)) after lookup
02:11:09.921 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:09.922 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.922 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.922 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.939 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:09.939 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:09.939 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=43), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [26ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [26ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [43ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@819385650(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6307b5fc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@21977c19 -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633))) after lookup
02:11:09.940 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:09.941 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.941 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.941 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.980 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:09.980 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:09.981 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=84), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [26ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [26ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [43ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [44ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [45ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [45ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [83ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@819385650(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6307b5fc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@21977c19 -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434))) after lookup
02:11:09.981 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:09.982 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.982 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:09.983 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:10.080 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:10.080 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:10.081 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=185), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [26ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [26ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [43ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [44ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [45ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [45ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [83ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [85ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [87ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [87ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [181ms] refreshing cache from master, [181ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [184ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@819385650(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6307b5fc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434) -> (continuation of Deferred@1846959752 after retry RPC@223969917), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@21977c19 -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434) -> (continuation of Deferred@1846959752 after retry RPC@223969917))) after lookup
02:11:10.082 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:10.083 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:10.083 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:10.083 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:10.580 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:10.581 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:10.581 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=685), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [26ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [26ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [43ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [44ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [45ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [45ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [83ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [85ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [87ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [87ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [181ms] refreshing cache from master, [181ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [184ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [185ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [187ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [187ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [681ms] refreshing cache from master, [681ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [684ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@819385650(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6307b5fc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434) -> (continuation of Deferred@1846959752 after retry RPC@223969917) -> (continuation of Deferred@1127818851 after retry RPC@36838931), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@21977c19 -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434) -> (continuation of Deferred@1846959752 after retry RPC@223969917) -> (continuation of Deferred@1127818851 after retry RPC@36838931))) after lookup
02:11:10.581 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:10.582 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:10.582 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:10.582 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:11.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.153865 21936 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3aea1f3e173e4803a7ae66840502f546)
02:11:11.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.154012 21936 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:11.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.155143 21936 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:11.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:11.156100 21627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:11.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:11.157109 21627 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:11.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.160001 21877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba" is_pre_election: true
02:11:11.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.160290 21877 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 6.
02:11:11.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.161041 21627 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 3aea1f3e173e4803a7ae66840502f546
02:11:11.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.161271 21936 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Leader pre-election won for term 8
02:11:11.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.161330 21936 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3aea1f3e173e4803a7ae66840502f546)
02:11:11.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.161358 21936 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 7 FOLLOWER]: Advancing to term 8
02:11:11.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.162137 21936 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:11.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.162426 21936 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 8 election: Requested vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:11.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.162750 21877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:11:11.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.162853 21877 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 6 FOLLOWER]: Advancing to term 8
02:11:11.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:11.163442 21627 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:11.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.164556 21877 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 8.
02:11:11.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.164834 21627 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 3aea1f3e173e4803a7ae66840502f546
02:11:11.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.164994 21936 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 FOLLOWER]: Leader election won for term 8
02:11:11.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.165158 21936 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 LEADER]: Becoming Leader. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Running, Role: LEADER
02:11:11.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.165401 21936 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:11.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.167563 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 reported cstate change: term changed from 7 to 8, leader changed from 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195) to 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193). New cstate: current_term: 8 leader_uuid: "8fa0772e177e429798c32dd085f59ae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:11:11.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:11.641520 21627 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 -> Peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Couldn't send request to peer 3aea1f3e173e4803a7ae66840502f546. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:11.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.727257 21877 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Refusing update from remote peer 8fa0772e177e429798c32dd085f59ae6: Log matching property violated. Preceding OpId in replica: term: 6 index: 20. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
02:11:11.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:11.728009 21936 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 20, Time since last communication: 0.000s
02:11:12.440 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:12.440 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:12.440 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=19, TimeoutTracker(timeout=30000, elapsed=2544), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [21ms] refreshing cache from master, [21ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [23ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [25ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [26ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [26ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [40ms] refreshing cache from master, [41ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [43ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [44ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [45ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [45ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [81ms] refreshing cache from master, [81ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [83ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [85ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [87ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [87ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [181ms] refreshing cache from master, [181ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [184ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [185ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [187ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [187ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [681ms] refreshing cache from master, [681ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [684ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [685ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [686ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [687ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2541ms] refreshing cache from master, [2541ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [2543ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@819385650(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@6307b5fc -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434) -> (continuation of Deferred@1846959752 after retry RPC@223969917) -> (continuation of Deferred@1127818851 after retry RPC@36838931) -> (continuation of Deferred@1519655471 after retry RPC@1212433896), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@21977c19 -> wakeup thread Time-limited test -> (continuation of Deferred@1930578998 after retry RPC@1778352633) -> (continuation of Deferred@1193511966 after retry RPC@1388719434) -> (continuation of Deferred@1846959752 after retry RPC@223969917) -> (continuation of Deferred@1127818851 after retry RPC@36838931) -> (continuation of Deferred@1519655471 after retry RPC@1212433896))) after lookup
02:11:12.441 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:12.442 [DEBUG - kudu-nio-6] (Connection.java:757) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] Successfully connected to peer
02:11:12.467 [DEBUG - kudu-nio-6] (Negotiator.java:919) Authenticated connection [id: 0xd8d5a4c2, L:/127.0.0.1:60444 - R:/127.19.210.193:41573] using TOKEN/null
02:11:12.489 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.195:42923
02:11:12.489 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.490383 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:12.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:42923
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44723
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:12.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:12.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.587472 21969 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:12.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.587666 21969 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:12.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.587708 21969 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:12.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.590958 21969 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:12.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.591060 21969 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.195
02:11:12.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.594777 21969 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:42923
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44723
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.21969
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:12.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:12.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:12.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:12.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:12.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:12.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.595808 21969 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:12.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.596869 21969 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:12.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.602908 21974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:12.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.603005 21975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:12.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:12.603183 21977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:12.603 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.603556 21969 server_base.cc:1061] running on GCE node
02:11:12.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.603930 21969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:12.604 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.604555 21969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:12.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.605720 21969 hybrid_clock.cc:648] HybridClock initialized: now 1783476672605707 us; error 22 us; skew 500 ppm
02:11:12.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.607638 21969 webserver.cc:534] Webserver started at http://127.19.210.195:44723/ using document root <none> and password file <none>
02:11:12.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.608239 21969 fs_manager.cc:362] Metadata directory not provided
02:11:12.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.608315 21969 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:12.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.611642 21969 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
02:11:12.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.613503 21983 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:12.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.614600 21969 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.614708 21969 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3aea1f3e173e4803a7ae66840502f546"
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.615119 21969 fs_report.cc:389] FS layout report
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/data
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:12.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:12.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.626870 21969 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:12.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.627434 21969 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:12.627 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.627600 21969 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:12.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.628134 21969 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:12.629 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.629388 21990 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:12.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.632342 21969 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:12.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.632407 21969 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
02:11:12.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.632472 21969 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:12.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.633867 21969 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:12.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.633909 21969 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
02:11:12.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.634230 21990 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap starting.
02:11:12.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.643388 21969 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.195:42923
02:11:12.643 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.643426 22097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.195:42923 every 8 connection(s)
02:11:12.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.644414 21969 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:12.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.645231 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 21969
02:11:12.646 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:12.651 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.651288 21990 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Log is configured to *not* fsync() on all Append() calls
02:11:12.654 [DEBUG - kudu-nio-6] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 21 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:12.655 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.193:41573
02:11:12.655 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:11:12.656 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.656153 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 21609
02:11:12.665 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:12.665 [DEBUG - kudu-nio-6] (Connection.java:239) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] handling channelInactive
02:11:12.665 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state READY due to: connection closed
02:11:12.665 [DEBUG - Time-limited test] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:12.667 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.667 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.667 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.669992 22098 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:12.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.670009 22100 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:12.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.670286 22098 heartbeater.cc:461] Registering TS with master...
02:11:12.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.670289 22100 heartbeater.cc:461] Registering TS with master...
02:11:12.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.670894 22098 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:12.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.670895 22100 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:12.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.671671 21990 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=24 ignored=0} inserts{seen=18 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:12.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.672012 21990 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap complete.
02:11:12.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.672286 22099 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:12.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.672364 22099 heartbeater.cc:461] Registering TS with master...
02:11:12.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.672545 22099 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:12.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.673026 20489 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:12.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.673101 21990 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.022s
02:11:12.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.673184 20416 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:12.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.673743 20337 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:12.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.674093 20416 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.195:52841
02:11:12.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.675328 21990 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:12.675 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.675626 21990 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Initialized, Role: FOLLOWER
02:11:12.676 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.675987 21990 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:12.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.676594 22100 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:12.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:12.676748 21990 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
02:11:12.680 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:12.680 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:12.680 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=15), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1724080438(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@c421303 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@339f83ff -> wakeup thread Time-limited test)) after lookup
02:11:12.681 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:12.681 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.681 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.682 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.699 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:12.700 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:12.700 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=34), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [17ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [17ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1724080438(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@c421303 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@339f83ff -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989))) after lookup
02:11:12.700 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:12.701 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.701 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.701 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.739 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:12.739 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:12.740 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=74), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [17ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [17ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [36ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [36ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [72ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [74ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1724080438(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@c421303 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@339f83ff -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759))) after lookup
02:11:12.740 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:12.741 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.741 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.741 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.799 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:12.800 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:12.800 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=135), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [17ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [17ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [36ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [36ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [72ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [74ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [75ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [76ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [76ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [134ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1724080438(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@c421303 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759) -> (continuation of Deferred@2118794598 after retry RPC@1333578510), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@339f83ff -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759) -> (continuation of Deferred@2118794598 after retry RPC@1333578510))) after lookup
02:11:12.801 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:12.801 [INFO - kudu-nio-3] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.801 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:12.802 [INFO - kudu-nio-3] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:13.079 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:13.079 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:13.080 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=414), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [17ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [17ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [36ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [36ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [72ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [74ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [75ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [76ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [76ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [134ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [135ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [137ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [137ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [412ms] refreshing cache from master, [412ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [414ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1724080438(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@c421303 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759) -> (continuation of Deferred@2118794598 after retry RPC@1333578510) -> (continuation of Deferred@2130474698 after retry RPC@1475471008), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@339f83ff -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759) -> (continuation of Deferred@2118794598 after retry RPC@1333578510) -> (continuation of Deferred@2130474698 after retry RPC@1475471008))) after lookup
02:11:13.080 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:13.081 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:13.081 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:13.081 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:13.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.989352 22111 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:11:13.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.989486 22111 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:13.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.990664 22111 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:13.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:13.991586 21810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:13.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:13.992440 21810 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:13.995 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.995504 22052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546" is_pre_election: true
02:11:13.995 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.995746 22052 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 7.
02:11:13.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.996412 21812 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:11:13.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.996668 22111 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Leader pre-election won for term 9
02:11:13.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.996752 22111 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:11:13.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.996812 22111 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 8 FOLLOWER]: Advancing to term 9
02:11:13.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.997596 22111 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:13.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.997967 22111 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 9 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:13.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.998347 22052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 9 candidate_status { last_received { term: 8 index: 28 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546"
02:11:13.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:13.998481 22052 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 7 FOLLOWER]: Advancing to term 9
02:11:13.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:13.998826 21810 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:14.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.000309 22052 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 9.
02:11:14.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.000631 21812 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:11:14.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.000779 22111 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Leader election won for term 9
02:11:14.001 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.000959 22111 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 LEADER]: Becoming Leader. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Running, Role: LEADER
02:11:14.001 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.001394 22111 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 8.28, Last appended by leader: 28, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:14.003 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.003616 20417 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba reported cstate change: term changed from 8 to 9, leader changed from 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193) to 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194). New cstate: current_term: 9 leader_uuid: "3b4491bc31634e84a3162ba1d70813ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:11:14.566 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:14.565824 21810 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba -> Peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Couldn't send request to peer 8fa0772e177e429798c32dd085f59ae6. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:14.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.620247 22052 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Refusing update from remote peer 3b4491bc31634e84a3162ba1d70813ba: Log matching property violated. Preceding OpId in replica: term: 7 index: 24. Preceding OpId from leader: term: 9 index: 29. (index mismatch)
02:11:14.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:14.620914 22128 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 24, Time since last communication: 0.000s
02:11:15.139 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:15.140 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:15.140 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=22, TimeoutTracker(timeout=30000, elapsed=2474), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [2ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [17ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [17ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [36ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [36ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [72ms] refreshing cache from master, [72ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [74ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [75ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [76ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [76ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [131ms] refreshing cache from master, [132ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [134ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [135ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [137ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [137ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [412ms] refreshing cache from master, [412ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [414ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [415ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [416ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [416ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2472ms] refreshing cache from master, [2472ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [2474ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1724080438(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@c421303 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759) -> (continuation of Deferred@2118794598 after retry RPC@1333578510) -> (continuation of Deferred@2130474698 after retry RPC@1475471008) -> (continuation of Deferred@128440813 after retry RPC@1606151007), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@339f83ff -> wakeup thread Time-limited test -> (continuation of Deferred@998143343 after retry RPC@653048989) -> (continuation of Deferred@1593758041 after retry RPC@498439759) -> (continuation of Deferred@2118794598 after retry RPC@1333578510) -> (continuation of Deferred@2130474698 after retry RPC@1475471008) -> (continuation of Deferred@128440813 after retry RPC@1606151007))) after lookup
02:11:15.140 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:15.141 [DEBUG - kudu-nio-5] (Connection.java:757) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] Successfully connected to peer
02:11:15.168 [DEBUG - kudu-nio-5] (Negotiator.java:919) Authenticated connection [id: 0xfc997886, L:/127.0.0.1:52228 - R:/127.19.210.194:43205] using TOKEN/null
02:11:15.190 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.193:41573
02:11:15.190 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.191351 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:15.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:15.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:15.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.291182 22148 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:15.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.291363 22148 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:15.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.291404 22148 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:15.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.294579 22148 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:15.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.294675 22148 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.193
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.298362 22148 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:15.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.22148
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:15.299 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.299345 22148 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:15.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.300709 22148 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:15.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.307582 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:15.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.307725 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:15.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.307790 22148 server_base.cc:1061] running on GCE node
02:11:15.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:15.307585 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:15.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.308387 22148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:15.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.308995 22148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:15.310 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.310207 22148 hybrid_clock.cc:648] HybridClock initialized: now 1783476675310180 us; error 42 us; skew 500 ppm
02:11:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.311954 22148 webserver.cc:534] Webserver started at http://127.19.210.193:36261/ using document root <none> and password file <none>
02:11:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.312449 22148 fs_manager.cc:362] Metadata directory not provided
02:11:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.312500 22148 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:15.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.315856 22148 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
02:11:15.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.317947 22162 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.318959 22148 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.319044 22148 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.319397 22148 fs_report.cc:389] FS layout report
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/data
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:15.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:15.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.339457 22148 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:15.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.340093 22148 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:15.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.340245 22148 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:15.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.340860 22148 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:15.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.342139 22169 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:15.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.345409 22148 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:15.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.345467 22148 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
02:11:15.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.345526 22148 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:15.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.346998 22148 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:15.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.347039 22148 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
02:11:15.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.347354 22169 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap starting.
02:11:15.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.357220 22148 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.193:41573
02:11:15.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.358217 22148 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:15.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.359069 22276 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.193:41573 every 8 connection(s)
02:11:15.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.362414 22169 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Log is configured to *not* fsync() on all Append() calls
02:11:15.367 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.366863 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 22148
02:11:15.367 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:15.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.369117 22279 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:15.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.369331 22279 heartbeater.cc:461] Registering TS with master...
02:11:15.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.369865 22279 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:15.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.372324 20416 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:15.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.373241 20416 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.193:43801
02:11:15.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.373342 22277 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:15.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.373405 22277 heartbeater.cc:461] Registering TS with master...
02:11:15.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.373576 22277 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:15.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.373875 22278 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:15.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.373946 22278 heartbeater.cc:461] Registering TS with master...
02:11:15.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.374116 22278 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:15.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.374979 20489 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:15.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.375063 20345 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:15.377 [DEBUG - kudu-nio-5] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 24 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:15.378 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.194:43205
02:11:15.378 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

02:11:15.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.378939 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 21793
02:11:15.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.382959 22169 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=21 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:15.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.383286 22169 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap complete.
02:11:15.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.384156 22169 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent bootstrapping tablet: real 0.037s	user 0.028s	sys 0.004s
02:11:15.384 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:15.385 [DEBUG - kudu-nio-5] (Connection.java:239) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] handling channelInactive
02:11:15.385 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state READY due to: connection closed
02:11:15.385 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: connection closed
02:11:15.387 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.386857 22169 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:15.387 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.387179 22169 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Initialized, Role: FOLLOWER
02:11:15.387 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.387585 22169 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 8.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:15.388 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.388229 22279 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:15.388 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:15.388448 22169 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.003s
02:11:15.399 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:15.400 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:15.400 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=15), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [0ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@792366653(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2f5b7996 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@14168a66 -> wakeup thread Time-limited test)) after lookup
02:11:15.400 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:15.401 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.401 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.401 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.419 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:15.420 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:15.420 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=35), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [0ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [16ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [16ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@792366653(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2f5b7996 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@14168a66 -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936))) after lookup
02:11:15.420 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:15.422 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.422 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.422 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.439 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:15.440 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:15.440 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=55), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [0ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [16ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [16ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [37ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [54ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@792366653(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2f5b7996 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@14168a66 -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965))) after lookup
02:11:15.440 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:15.441 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.442 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.442 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.540 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:15.540 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:15.540 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=155), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [0ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [16ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [16ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [37ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [54ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [55ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [57ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [57ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [152ms] refreshing cache from master, [152ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [154ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@792366653(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2f5b7996 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965) -> (continuation of Deferred@1258228154 after retry RPC@1125481761), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@14168a66 -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965) -> (continuation of Deferred@1258228154 after retry RPC@1125481761))) after lookup
02:11:15.541 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:15.542 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.542 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.542 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.820 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:15.820 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:15.820 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=435), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [0ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [16ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [16ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [37ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [54ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [55ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [57ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [57ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [152ms] refreshing cache from master, [152ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [154ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [155ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [157ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [157ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [432ms] refreshing cache from master, [432ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [434ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@792366653(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2f5b7996 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965) -> (continuation of Deferred@1258228154 after retry RPC@1125481761) -> (continuation of Deferred@1427540404 after retry RPC@1031936578), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@14168a66 -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965) -> (continuation of Deferred@1258228154 after retry RPC@1125481761) -> (continuation of Deferred@1427540404 after retry RPC@1031936578))) after lookup
02:11:15.821 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:15.822 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.822 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:15.822 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205
02:11:16.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.689831 22290 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b4491bc31634e84a3162ba1d70813ba)
02:11:16.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.689975 22290 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:16.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.690950 22290 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:16.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:16.691694 21987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:11:16.692 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:16.692589 21987 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:11:16.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.695672 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 10 candidate_status { last_received { term: 9 index: 32 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6" is_pre_election: true
02:11:16.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.695904 22231 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 8.
02:11:16.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.696516 21985 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6; no voters: 3b4491bc31634e84a3162ba1d70813ba
02:11:16.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.696784 22290 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Leader pre-election won for term 10
02:11:16.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.696838 22290 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b4491bc31634e84a3162ba1d70813ba)
02:11:16.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.696866 22290 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 9 FOLLOWER]: Advancing to term 10
02:11:16.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.697587 22290 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:16.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.697877 22290 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 10 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:16.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.698206 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3aea1f3e173e4803a7ae66840502f546" candidate_term: 10 candidate_status { last_received { term: 9 index: 32 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6"
02:11:16.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.698323 22231 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 8 FOLLOWER]: Advancing to term 10
02:11:16.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:16.698812 21987 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111)
02:11:16.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.699867 22231 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aea1f3e173e4803a7ae66840502f546 in term 10.
02:11:16.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.700162 21985 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6; no voters: 3b4491bc31634e84a3162ba1d70813ba
02:11:16.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.700318 22290 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 FOLLOWER]: Leader election won for term 10
02:11:16.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.700501 22290 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 LEADER]: Becoming Leader. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Running, Role: LEADER
02:11:16.700 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.700786 22290 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 9.32, Last appended by leader: 32, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:16.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:16.702950 20416 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 reported cstate change: term changed from 9 to 10, leader changed from 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194) to 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195). New cstate: current_term: 10 leader_uuid: "3aea1f3e173e4803a7ae66840502f546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: HEALTHY } } }
02:11:17.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.274343 21987 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 -> Peer 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205): Couldn't send request to peer 3b4491bc31634e84a3162ba1d70813ba. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.194:43205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:17.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.277273 22231 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Refusing update from remote peer 3aea1f3e173e4803a7ae66840502f546: Log matching property violated. Preceding OpId in replica: term: 8 index: 28. Preceding OpId from leader: term: 10 index: 33. (index mismatch)
02:11:17.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.277966 22309 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 28, Time since last communication: 0.000s
02:11:17.839 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:17.840 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:17.840 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=25, TimeoutTracker(timeout=30000, elapsed=2455), Traces: [0ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [0ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: connection closed, [1ms] delaying RPC due to: Network error: connection closed, [12ms] refreshing cache from master, [12ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [14ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [15ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [16ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [16ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [32ms] refreshing cache from master, [32ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [34ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [35ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [37ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [37ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [52ms] refreshing cache from master, [52ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [54ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [55ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [57ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [57ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [152ms] refreshing cache from master, [152ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [154ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [155ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [157ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [157ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [432ms] refreshing cache from master, [432ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [434ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [435ms] sending RPC to server 3b4491bc31634e84a3162ba1d70813ba, [437ms] received response from server 3b4491bc31634e84a3162ba1d70813ba: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [437ms] delaying RPC due to: Network error: Failed to connect to peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205): Connection refused: /127.19.210.194:43205, [2452ms] refreshing cache from master, [2452ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [2454ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@792366653(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@2f5b7996 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965) -> (continuation of Deferred@1258228154 after retry RPC@1125481761) -> (continuation of Deferred@1427540404 after retry RPC@1031936578) -> (continuation of Deferred@1795683888 after retry RPC@1190830360), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@14168a66 -> wakeup thread Time-limited test -> (continuation of Deferred@646585393 after retry RPC@34706936) -> (continuation of Deferred@344128144 after retry RPC@447873965) -> (continuation of Deferred@1258228154 after retry RPC@1125481761) -> (continuation of Deferred@1427540404 after retry RPC@1031936578) -> (continuation of Deferred@1795683888 after retry RPC@1190830360))) after lookup
02:11:17.841 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:17.842 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] Successfully connected to peer
02:11:17.856 [DEBUG - kudu-nio-3] (Negotiator.java:919) Authenticated connection [id: 0x9b0e7b5e, L:/127.0.0.1:42842 - R:/127.19.210.195:42923] using TOKEN/null
02:11:17.873 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.194:43205
02:11:17.873 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 1
  }
}

02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.874054 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:43205
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39205
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:11:17.874 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:17.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:17.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:17.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:17.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.973112 22329 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:17.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.973312 22329 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:17.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.973354 22329 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:17.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.976490 22329 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:17.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.976599 22329 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.194
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.980265 22329 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.194:43205
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.194
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39205
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.22329
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:17.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.194
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/logs
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:17.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.981297 22329 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:17.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.982357 22329 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:17.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.988504 22335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:17.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.988674 22329 server_base.cc:1061] running on GCE node
02:11:17.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.988785 22334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:17.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:17.988557 22337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:17.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.989327 22329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:17.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.989897 22329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:17.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.991044 22329 hybrid_clock.cc:648] HybridClock initialized: now 1783476677991028 us; error 30 us; skew 500 ppm
02:11:17.993 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.992915 22329 webserver.cc:534] Webserver started at http://127.19.210.194:39205/ using document root <none> and password file <none>
02:11:17.993 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.993450 22329 fs_manager.cc:362] Metadata directory not provided
02:11:17.993 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.993511 22329 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:17.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.996762 22329 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
02:11:17.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.998628 22343 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:17.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.999586 22329 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:17.999696 22329 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.000099 22329 fs_report.cc:389] FS layout report
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/wal
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/data
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:18.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:18.014 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.014235 22329 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:18.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.014894 22329 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:18.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.015054 22329 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:18.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.015638 22329 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:18.017 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.017001 22350 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:18.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.020295 22329 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:18.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.020362 22329 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
02:11:18.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.020458 22329 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:18.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.021917 22329 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:18.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.021961 22329 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
02:11:18.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.022342 22350 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap starting.
02:11:18.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.032380 22329 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.194:43205
02:11:18.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.032418 22457 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.194:43205 every 8 connection(s)
02:11:18.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.033365 22329 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-1/data/info.pb
02:11:18.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.038757 22350 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Log is configured to *not* fsync() on all Append() calls
02:11:18.039 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.039588 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 22329
02:11:18.039 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:18.048 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 27 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:18.049 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.195:42923
02:11:18.049 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

02:11:18.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.049914 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 21969
02:11:18.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.054575 22458 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:18.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.054837 22458 heartbeater.cc:461] Registering TS with master...
02:11:18.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.055485 22458 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:18.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.057312 20489 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:18.062 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] handling channelInactive
02:11:18.062 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state READY due to: connection closed
02:11:18.063 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:18.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.062976 22459 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:18.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.063086 22459 heartbeater.cc:461] Registering TS with master...
02:11:18.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.063316 22459 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:18.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.064141 22350 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=24 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:18.064 [DEBUG - Time-limited test] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.064595 22350 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Bootstrap complete.
02:11:18.065 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.065387 20345 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:18.065 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.065 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.067417 22460 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:18.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.067445 22350 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent bootstrapping tablet: real 0.045s	user 0.040s	sys 0.000s
02:11:18.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.067750 22460 heartbeater.cc:461] Registering TS with master...
02:11:18.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.068005 22460 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:18.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.068975 20416 ts_manager.cc:194] Re-registered known tserver with Master: 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205)
02:11:18.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.069895 20416 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.194:58221
02:11:18.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.070458 22350 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:18.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.070842 22350 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Initialized, Role: FOLLOWER
02:11:18.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.071369 22350 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 9.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:18.072 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.072273 22350 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
02:11:18.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:18.072305 22460 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:18.079 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:18.080 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:18.080 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=2, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=16), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test)) after lookup
02:11:18.080 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.081 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.081 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.082 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.100 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:18.100 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:18.101 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=4, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=37), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [17ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [19ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [19ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [37ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930))) after lookup
02:11:18.101 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.102 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.102 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.102 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.139 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:18.140 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:18.140 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=6, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=76), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [17ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [19ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [19ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [37ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [38ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [39ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [39ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [76ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163))) after lookup
02:11:18.140 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.141 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.142 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.142 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.179 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:18.179 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:18.180 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=8, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=116), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [17ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [19ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [19ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [37ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [38ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [39ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [39ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [76ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [77ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [79ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [79ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [116ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995))) after lookup
02:11:18.180 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.181 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.181 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.181 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.639 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:18.640 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:18.640 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=10, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=576), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [17ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [19ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [19ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [37ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [38ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [39ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [39ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [76ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [77ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [79ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [79ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [116ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [117ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [118ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [118ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [574ms] refreshing cache from master, [574ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [576ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995) -> (continuation of Deferred@856290471 after retry RPC@344401887), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995) -> (continuation of Deferred@856290471 after retry RPC@344401887))) after lookup
02:11:18.641 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.642 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.642 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.642 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.899 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:18.900 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)[L],3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:18.900 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=12, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=836), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [17ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [19ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [19ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [37ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [38ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [39ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [39ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [76ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [77ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [79ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [79ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [116ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [117ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [118ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [118ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [574ms] refreshing cache from master, [574ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [576ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [577ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [579ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [579ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [834ms] refreshing cache from master, [834ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [836ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995) -> (continuation of Deferred@856290471 after retry RPC@344401887) -> (continuation of Deferred@1909229724 after retry RPC@1549328549), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995) -> (continuation of Deferred@856290471 after retry RPC@344401887) -> (continuation of Deferred@1909229724 after retry RPC@1549328549))) after lookup
02:11:18.901 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] connecting to peer
02:11:18.902 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.902 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] cleaning up while in state CONNECTING due to: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:18.902 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923
02:11:19.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.323536 22471 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
02:11:19.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.323698 22471 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:19.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.324786 22471 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:19.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:19.325537 22347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:19.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:19.326320 22347 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:19.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.328553 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 10 candidate_status { last_received { term: 9 index: 32 } } ignore_live_leader: false dest_uuid: "8fa0772e177e429798c32dd085f59ae6" is_pre_election: true
02:11:19.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.329026 22345 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba; no voters: 3aea1f3e173e4803a7ae66840502f546, 8fa0772e177e429798c32dd085f59ae6
02:11:19.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.329284 22471 raft_consensus.cc:2749] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
02:11:19.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.373639 22476 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3aea1f3e173e4803a7ae66840502f546)
02:11:19.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.373770 22476 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:19.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.374804 22476 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:19.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:19.375795 22166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:19.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:19.376657 22166 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:19.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.379478 22412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 11 candidate_status { last_received { term: 10 index: 36 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba" is_pre_election: true
02:11:19.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.379669 22412 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 9.
02:11:19.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.380234 22166 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 3aea1f3e173e4803a7ae66840502f546
02:11:19.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.380468 22476 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Leader pre-election won for term 11
02:11:19.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.380550 22476 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3aea1f3e173e4803a7ae66840502f546)
02:11:19.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.380594 22476 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 10 FOLLOWER]: Advancing to term 11
02:11:19.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.381354 22476 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:19.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.381700 22476 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 11 election: Requested vote from peers 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194:43205), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:19.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.382020 22412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "8fa0772e177e429798c32dd085f59ae6" candidate_term: 11 candidate_status { last_received { term: 10 index: 36 } } ignore_live_leader: false dest_uuid: "3b4491bc31634e84a3162ba1d70813ba"
02:11:19.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.382136 22412 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 9 FOLLOWER]: Advancing to term 11
02:11:19.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:19.382586 22166 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111)
02:11:19.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.383736 22412 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fa0772e177e429798c32dd085f59ae6 in term 11.
02:11:19.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.384019 22166 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b4491bc31634e84a3162ba1d70813ba, 8fa0772e177e429798c32dd085f59ae6; no voters: 3aea1f3e173e4803a7ae66840502f546
02:11:19.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.384177 22476 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 11 FOLLOWER]: Leader election won for term 11
02:11:19.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.384383 22476 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 11 LEADER]: Becoming Leader. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Running, Role: LEADER
02:11:19.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.384703 22476 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 10.36, Last appended by leader: 36, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:19.387 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.386936 20416 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 reported cstate change: term changed from 10 to 11, leader changed from 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195) to 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193). New cstate: current_term: 11 leader_uuid: "8fa0772e177e429798c32dd085f59ae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:11:19.931 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:19.931375 22166 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 -> Peer 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923): Couldn't send request to peer 3aea1f3e173e4803a7ae66840502f546. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.195:42923: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:19.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.946486 22412 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Refusing update from remote peer 8fa0772e177e429798c32dd085f59ae6: Log matching property violated. Preceding OpId in replica: term: 9 index: 32. Preceding OpId from leader: term: 11 index: 37. (index mismatch)
02:11:19.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:19.947171 22494 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 32, Time since last communication: 0.000s
02:11:21.619 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:21.619 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:21.620 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=14, sequence_id=28, TimeoutTracker(timeout=30000, elapsed=3556), Traces: [0ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [2ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [2ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [14ms] refreshing cache from master, [14ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [16ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [17ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [19ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [19ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [34ms] refreshing cache from master, [34ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [37ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [38ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [39ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [39ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [74ms] refreshing cache from master, [74ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [76ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [77ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [79ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [79ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [114ms] refreshing cache from master, [114ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [116ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [117ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [118ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [118ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [574ms] refreshing cache from master, [574ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [576ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [577ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [579ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [579ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [834ms] refreshing cache from master, [834ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [836ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [837ms] sending RPC to server 3aea1f3e173e4803a7ae66840502f546, [839ms] received response from server 3aea1f3e173e4803a7ae66840502f546: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [839ms] delaying RPC due to: Network error: Failed to connect to peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923): Connection refused: /127.19.210.195:42923, [3554ms] refreshing cache from master, [3554ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [3556ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@2065115479(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@68f6a54c -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995) -> (continuation of Deferred@856290471 after retry RPC@344401887) -> (continuation of Deferred@1909229724 after retry RPC@1549328549) -> (continuation of Deferred@1741283732 after retry RPC@2081388544), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@28209c10 -> wakeup thread Time-limited test -> (continuation of Deferred@934068851 after retry RPC@465569930) -> (continuation of Deferred@1992554065 after retry RPC@1834755163) -> (continuation of Deferred@1638575817 after retry RPC@1834491995) -> (continuation of Deferred@856290471 after retry RPC@344401887) -> (continuation of Deferred@1909229724 after retry RPC@1549328549) -> (continuation of Deferred@1741283732 after retry RPC@2081388544))) after lookup
02:11:21.620 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:21.621 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] Successfully connected to peer
02:11:21.638 [DEBUG - kudu-nio-3] (Negotiator.java:919) Authenticated connection [id: 0xf84dddb1, L:/127.0.0.1:60030 - R:/127.19.210.193:41573] using TOKEN/null
02:11:21.653 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.195:42923
02:11:21.653 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 2
  }
}

02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.654386 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:21.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:42923
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44723
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:21.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:21.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.755645 22514 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:21.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.755825 22514 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:21.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.755867 22514 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:21.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.759395 22514 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:21.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.759500 22514 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.195
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.763303 22514 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.195:42923
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:21.763 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.195
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=44723
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.22514
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.195
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/logs
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:21.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.764324 22514 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:21.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.765439 22514 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:21.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.771564 22520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:21.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.771590 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:21.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:21.771590 22522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:21.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.771636 22514 server_base.cc:1061] running on GCE node
02:11:21.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.772204 22514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:21.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.772704 22514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:21.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.773836 22514 hybrid_clock.cc:648] HybridClock initialized: now 1783476681773820 us; error 33 us; skew 500 ppm
02:11:21.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.775300 22514 webserver.cc:534] Webserver started at http://127.19.210.195:44723/ using document root <none> and password file <none>
02:11:21.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.775768 22514 fs_manager.cc:362] Metadata directory not provided
02:11:21.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.775815 22514 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:21.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.778738 22514 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.000s
02:11:21.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.780340 22528 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:21.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.781248 22514 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:11:21.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.781344 22514 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:21.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3aea1f3e173e4803a7ae66840502f546"
02:11:21.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:21.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.781729 22514 fs_report.cc:389] FS layout report
02:11:21.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/wal
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/data
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:21.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:21.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.805996 22514 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:21.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.806694 22514 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:21.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.806871 22514 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:21.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.807464 22514 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:21.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.808736 22535 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:21.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.811898 22514 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:21.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.811956 22514 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
02:11:21.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.812016 22514 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:21.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.813556 22514 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:21.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.813601 22514 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.001s
02:11:21.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.813926 22535 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap starting.
02:11:21.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.823730 22514 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.195:42923
02:11:21.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.823752 22642 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.195:42923 every 8 connection(s)
02:11:21.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.824785 22514 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-2/data/info.pb
02:11:21.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.829694 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 22514
02:11:21.830 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:21.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.831517 22535 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Log is configured to *not* fsync() on all Append() calls
02:11:21.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.835167 22643 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:21.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.835407 22643 heartbeater.cc:461] Registering TS with master...
02:11:21.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.835857 22643 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:21.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.837525 20489 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:21.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.837522 22644 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:21.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.837620 22644 heartbeater.cc:461] Registering TS with master...
02:11:21.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.837792 22644 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:21.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.838673 20337 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:21.839 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 30 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:21.840 [INFO - Time-limited test] (MiniKuduCluster.java:475) Killing tablet server 127.19.210.193:41573
02:11:21.840 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:11:21.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.844625 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 22148
02:11:21.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.844789 22645 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:21.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.844853 22645 heartbeater.cc:461] Registering TS with master...
02:11:21.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.845010 22645 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:21.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.847208 20416 ts_manager.cc:194] Re-registered known tserver with Master: 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:21.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.848137 20416 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.195:58635
02:11:21.852 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:21.853 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] handling channelInactive
02:11:21.853 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state READY due to: connection closed
02:11:21.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.857189 22535 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=27 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:21.857 [DEBUG - pool-2-thread-1] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:21.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.857604 22535 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Bootstrap complete.
02:11:21.858 [INFO - kudu-nio-4] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.858 [DEBUG - kudu-nio-4] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.858628 22535 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent bootstrapping tablet: real 0.045s	user 0.029s	sys 0.011s
02:11:21.858 [INFO - kudu-nio-4] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.860814 22535 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:21.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.861136 22535 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aea1f3e173e4803a7ae66840502f546, State: Initialized, Role: FOLLOWER
02:11:21.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.861503 22535 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 10.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:21.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.862118 22645 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:21.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:21.862280 22535 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
02:11:21.879 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:21.879 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:21.879 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=3, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=26), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test, errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test)) after lookup
02:11:21.880 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:21.881 [INFO - kudu-nio-5] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.881 [DEBUG - kudu-nio-5] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.881 [INFO - kudu-nio-5] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.899 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:21.900 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:21.900 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=5, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=47), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [27ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [28ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208))) after lookup
02:11:21.900 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:21.901 [INFO - kudu-nio-6] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.901 [DEBUG - kudu-nio-6] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.902 [INFO - kudu-nio-6] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.939 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:21.939 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:21.940 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=7, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=86), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [27ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [28ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [48ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [49ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [84ms] refreshing cache from master, [84ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129))) after lookup
02:11:21.940 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:21.941 [INFO - kudu-nio-7] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.941 [DEBUG - kudu-nio-7] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.941 [INFO - kudu-nio-7] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:21.999 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:21.999 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:22.001 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=9, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=148), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [27ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [28ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [48ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [49ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [84ms] refreshing cache from master, [84ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [87ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [88ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [88ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [144ms] refreshing cache from master, [144ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [146ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086))) after lookup
02:11:22.002 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:22.003 [INFO - kudu-nio-0] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.003 [DEBUG - kudu-nio-0] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.003 [INFO - kudu-nio-0] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.360 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:22.360 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:22.360 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=11, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=507), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [27ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [28ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [48ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [49ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [84ms] refreshing cache from master, [84ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [87ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [88ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [88ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [144ms] refreshing cache from master, [144ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [146ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [149ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [150ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [150ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [504ms] refreshing cache from master, [504ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [507ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086) -> (continuation of Deferred@1405234985 after retry RPC@1913903025), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086) -> (continuation of Deferred@1405234985 after retry RPC@1913903025))) after lookup
02:11:22.361 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:22.362 [INFO - kudu-nio-1] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.362 [DEBUG - kudu-nio-1] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.362 [INFO - kudu-nio-1] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.580 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:22.580 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205),8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)[L]]}]
02:11:22.580 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=13, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=727), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [27ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [28ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [48ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [49ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [84ms] refreshing cache from master, [84ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [87ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [88ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [88ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [144ms] refreshing cache from master, [144ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [146ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [149ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [150ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [150ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [504ms] refreshing cache from master, [504ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [507ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [508ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [509ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [509ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [724ms] refreshing cache from master, [724ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [726ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086) -> (continuation of Deferred@1405234985 after retry RPC@1913903025) -> (continuation of Deferred@691700160 after retry RPC@1165518089), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086) -> (continuation of Deferred@1405234985 after retry RPC@1913903025) -> (continuation of Deferred@691700160 after retry RPC@1165518089))) after lookup
02:11:22.581 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] connecting to peer
02:11:22.581 [INFO - kudu-nio-2] (Connection.java:429) Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.581 [DEBUG - kudu-nio-2] (Connection.java:709) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] cleaning up while in state CONNECTING due to: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:22.582 [INFO - kudu-nio-2] (AsyncKuduClient.java:2355) Invalidating location 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573) for tablet 49ba254cf896489cb98aaa7fafc55cc1: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573
02:11:23.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.154203 22656 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:11:23.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.154352 22656 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:23.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.154759 22656 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:23.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:23.156025 22345 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:23.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.159639 22597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 12 candidate_status { last_received { term: 11 index: 40 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546" is_pre_election: true
02:11:23.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.159937 22597 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 10.
02:11:23.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.160693 22347 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:11:23.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.160882 22656 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Leader pre-election won for term 12
02:11:23.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.160948 22656 raft_consensus.cc:493] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8fa0772e177e429798c32dd085f59ae6)
02:11:23.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.160976 22656 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 11 FOLLOWER]: Advancing to term 12
02:11:23.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.161728 22656 raft_consensus.cc:515] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:23.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.162035 22656 leader_election.cc:290] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 12 election: Requested vote from peers 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573), 3aea1f3e173e4803a7ae66840502f546 (127.19.210.195:42923)
02:11:23.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.162375 22597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49ba254cf896489cb98aaa7fafc55cc1" candidate_uuid: "3b4491bc31634e84a3162ba1d70813ba" candidate_term: 12 candidate_status { last_received { term: 11 index: 40 } } ignore_live_leader: false dest_uuid: "3aea1f3e173e4803a7ae66840502f546"
02:11:23.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.162504 22597 raft_consensus.cc:3060] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 10 FOLLOWER]: Advancing to term 12
02:11:23.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:23.162878 22345 leader_election.cc:336] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111)
02:11:23.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.164218 22597 raft_consensus.cc:2468] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b4491bc31634e84a3162ba1d70813ba in term 12.
02:11:23.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.164510 22347 leader_election.cc:304] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aea1f3e173e4803a7ae66840502f546, 3b4491bc31634e84a3162ba1d70813ba; no voters: 8fa0772e177e429798c32dd085f59ae6
02:11:23.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.164706 22656 raft_consensus.cc:2804] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 12 FOLLOWER]: Leader election won for term 12
02:11:23.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.164947 22656 raft_consensus.cc:697] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [term 12 LEADER]: Becoming Leader. State: Replica: 3b4491bc31634e84a3162ba1d70813ba, State: Running, Role: LEADER
02:11:23.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.165249 22656 consensus_queue.cc:237] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 11.40, Last appended by leader: 40, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:23.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.167490 20416 catalog_manager.cc:5696] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba reported cstate change: term changed from 11 to 12, leader changed from 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193) to 3b4491bc31634e84a3162ba1d70813ba (127.19.210.194). New cstate: current_term: 12 leader_uuid: "3b4491bc31634e84a3162ba1d70813ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } health_report { overall_health: UNKNOWN } } }
02:11:23.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.704228 22597 raft_consensus.cc:1275] T 49ba254cf896489cb98aaa7fafc55cc1 P 3aea1f3e173e4803a7ae66840502f546 [term 12 FOLLOWER]: Refusing update from remote peer 3b4491bc31634e84a3162ba1d70813ba: Log matching property violated. Preceding OpId in replica: term: 10 index: 36. Preceding OpId from leader: term: 12 index: 41. (index mismatch)
02:11:23.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:23.704993 22673 consensus_queue.cc:1048] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 36, Time since last communication: 0.000s
02:11:23.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:23.713923 22345 consensus_peers.cc:597] T 49ba254cf896489cb98aaa7fafc55cc1 P 3b4491bc31634e84a3162ba1d70813ba -> Peer 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573): Couldn't send request to peer 8fa0772e177e429798c32dd085f59ae6. Status: Network error: Client connection negotiation failed: client connection to 127.19.210.193:41573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
02:11:24.440 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2538) Learned about tablet 49ba254cf896489cb98aaa7fafc55cc1 for table 'org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364' with partition [<start>, <end>)
02:11:24.440 [DEBUG - kudu-nio-0] (TableLocationsCache.java:192) Discovered table locations:	[Tablet{lowerBoundPartitionKey=0x, upperBoundPartitionKey=0x, ttl=299999, tablet=49ba254cf896489cb98aaa7fafc55cc1@[3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923),3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)[L],8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)]}]
02:11:24.441 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1488) Retrying sending RPC KuduRpc(method=Write, tablet=49ba254cf896489cb98aaa7fafc55cc1, attempt=15, sequence_id=31, TimeoutTracker(timeout=30000, elapsed=2587), Traces: [0ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [0ms] delaying RPC due to: Illegal state: connection is terminated, [4ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [5ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [5ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [24ms] refreshing cache from master, [24ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [26ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [27ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [28ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [28ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [44ms] refreshing cache from master, [44ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [46ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [47ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [48ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [49ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [84ms] refreshing cache from master, [84ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [86ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [87ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [88ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [88ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [144ms] refreshing cache from master, [144ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [146ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [149ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [150ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [150ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [504ms] refreshing cache from master, [504ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [507ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [508ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [509ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [509ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [724ms] refreshing cache from master, [724ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [726ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, [727ms] sending RPC to server 8fa0772e177e429798c32dd085f59ae6, [728ms] received response from server 8fa0772e177e429798c32dd085f59ae6: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [729ms] delaying RPC due to: Network error: Failed to connect to peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573): Connection refused: /127.19.210.193:41573, [2584ms] refreshing cache from master, [2584ms] Sub RPC GetTableLocations: sending RPC to server master-127.19.210.253:42855, [2586ms] Sub RPC GetTableLocations: received response from server master-127.19.210.253:42855: OK, deferred=Deferred@1513401751(state=PENDING, result=null, callback=org.apache.kudu.client.AsyncKuduSession$$Lambda$232/0x00007f7ab036b560@5d7c11a9 -> passthrough -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086) -> (continuation of Deferred@1405234985 after retry RPC@1913903025) -> (continuation of Deferred@691700160 after retry RPC@1165518089) -> (continuation of Deferred@942882449 after retry RPC@1271437399), errback=passthrough -> org.apache.kudu.client.AsyncKuduSession$SingleOperationErrCallback@4fc3266f -> wakeup thread Time-limited test -> (continuation of Deferred@921885671 after retry RPC@1616147208) -> (continuation of Deferred@86198982 after retry RPC@1177335129) -> (continuation of Deferred@199291038 after retry RPC@684006086) -> (continuation of Deferred@1405234985 after retry RPC@1913903025) -> (continuation of Deferred@691700160 after retry RPC@1165518089) -> (continuation of Deferred@942882449 after retry RPC@1271437399))) after lookup
02:11:24.441 [DEBUG - kudu-nio-0] (Connection.java:748) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] connecting to peer
02:11:24.442 [DEBUG - kudu-nio-3] (Connection.java:757) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] Successfully connected to peer
02:11:24.456 [DEBUG - kudu-nio-3] (Negotiator.java:919) Authenticated connection [id: 0x28329640, L:/127.0.0.1:42402 - R:/127.19.210.194:43205] using TOKEN/null
02:11:24.471 [INFO - Time-limited test] (MiniKuduCluster.java:456) Starting tablet server 127.19.210.193:41573
02:11:24.471 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
  id {
    type: TSERVER
    index: 0
  }
}

02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.472321 20299 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:11:24.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:24.473 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
02:11:24.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.571448 22692 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:11:24.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.571625 22692 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:11:24.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.571666 22692 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:11:24.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.575003 22692 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:11:24.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.575103 22692 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.210.193
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.578833 22692 tablet_server_runner.cc:78] Tablet server non-default flags:
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.19.210.212:39895
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.19.210.193:41573
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.19.210.193
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36261
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.19.210.254:37903,127.19.210.253:42855,127.19.210.252:44531
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --heap_profile_path=/tmp/kudu.22692
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.19.210.193
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/logs
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision a3f35afc2b6dad6aa403ec58378221544a26edd0
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type DEBUG
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 08 Jul 2026 01:40:33 UTC on bdcb31816ec0
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 16847
02:11:24.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.579799 22692 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:11:24.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.580889 22692 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:11:24.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.586926 22698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:24.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.587002 22692 server_base.cc:1061] running on GCE node
02:11:24.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.586928 22697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:24.587 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.587145 22700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:11:24.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.587971 22692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:11:24.588 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.588474 22692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:11:24.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.589634 22692 hybrid_clock.cc:648] HybridClock initialized: now 1783476684589618 us; error 29 us; skew 500 ppm
02:11:24.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.591187 22692 webserver.cc:534] Webserver started at http://127.19.210.193:36261/ using document root <none> and password file <none>
02:11:24.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.591704 22692 fs_manager.cc:362] Metadata directory not provided
02:11:24.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.591761 22692 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:11:24.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.594889 22692 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
02:11:24.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.596706 22706 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
02:11:24.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.597656 22692 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.597750 22692 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data,/tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8fa0772e177e429798c32dd085f59ae6"
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-07-08 02:10:52 on dist-test-slave-7tzg"
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.598129 22692 fs_report.cc:389] FS layout report
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/wal
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/data
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:11:24.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:11:24.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.611241 22692 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
02:11:24.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.611783 22692 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:11:24.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.611953 22692 kserver.cc:163] Server-wide thread pool size limit: 3276
02:11:24.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.612650 22692 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
02:11:24.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.613835 22713 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
02:11:24.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.616935 22692 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
02:11:24.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.616991 22692 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s	user 0.000s	sys 0.000s
02:11:24.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.617049 22692 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
02:11:24.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.618423 22692 ts_tablet_manager.cc:616] Registered 1 tablets
02:11:24.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.618464 22692 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
02:11:24.618 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.618805 22713 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap starting.
02:11:24.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.627985 22692 rpc_server.cc:307] RPC server started. Bound to: 127.19.210.193:41573
02:11:24.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.628026 22820 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.210.193:41573 every 8 connection(s)
02:11:24.629 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.628891 22692 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasknKVXmV/test-tmp/mini-kudu-cluster15219309224714599347/ts-0/data/info.pb
02:11:24.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.637045 22713 log.cc:826] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Log is configured to *not* fsync() on all Append() calls
02:11:24.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.637415 20299 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu as pid 22692
02:11:24.637 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: 
02:11:24.644 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.644745 22822 heartbeater.cc:344] Connected to a master server at 127.19.210.254:37903
02:11:24.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.644984 22822 heartbeater.cc:461] Registering TS with master...
02:11:24.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.645555 22822 heartbeater.cc:507] Master 127.19.210.254:37903 requested a full tablet report, sending...
02:11:24.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.645579 22821 heartbeater.cc:344] Connected to a master server at 127.19.210.252:44531
02:11:24.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.646106 22821 heartbeater.cc:461] Registering TS with master...
02:11:24.646 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.646301 22821 heartbeater.cc:507] Master 127.19.210.252:44531 requested a full tablet report, sending...
02:11:24.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.647385 20345 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:24.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.647384 20490 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:24.648 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:24.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.649313 22823 heartbeater.cc:344] Connected to a master server at 127.19.210.253:42855
02:11:24.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.649390 22823 heartbeater.cc:461] Registering TS with master...
02:11:24.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.649577 22823 heartbeater.cc:507] Master 127.19.210.253:42855 requested a full tablet report, sending...
02:11:24.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.650288 20416 ts_manager.cc:194] Re-registered known tserver with Master: 8fa0772e177e429798c32dd085f59ae6 (127.19.210.193:41573)
02:11:24.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.650985 20416 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.19.210.193:38631
02:11:24.656 [DEBUG - kudu-nio-3] (AsyncKuduScanner.java:1370) AsyncKuduScanner$Response(scannerId = "", data = RowwiseRowResultIterator for 33 rows, more = false) for scanner KuduScanner(table=org.apache.kudu.client.TestMultipleLeaderFailover-1783476653364, tablet=49ba254cf896489cb98aaa7fafc55cc1, scannerId=null, scanRequestTimeout=30000, startPrimaryKey=<start>, endPrimaryKey=<end>, lastPrimaryKey=<last>)
02:11:24.662 [DEBUG - main] (Connection.java:575) [peer master-127.19.210.253:42855(127.19.210.253:42855)] disconnecting while in state READY
02:11:24.663 [DEBUG - main] (Connection.java:575) [peer 8fa0772e177e429798c32dd085f59ae6(127.19.210.193:41573)] disconnecting while in state TERMINATED
02:11:24.664 [DEBUG - main] (Connection.java:575) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] disconnecting while in state READY
02:11:24.664 [DEBUG - main] (Connection.java:575) [peer master-127.19.210.254:37903(127.19.210.254:37903)] disconnecting while in state READY
02:11:24.665 [DEBUG - main] (Connection.java:575) [peer 3aea1f3e173e4803a7ae66840502f546(127.19.210.195:42923)] disconnecting while in state TERMINATED
02:11:24.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.665186 22713 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=30 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
02:11:24.665 [DEBUG - main] (Connection.java:575) [peer master-127.19.210.252:44531(127.19.210.252:44531)] disconnecting while in state READY
02:11:24.666 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.19.210.253:42855(127.19.210.253:42855)] handling channelInactive
02:11:24.666 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.19.210.252:44531(127.19.210.252:44531)] handling channelInactive
02:11:24.666 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.19.210.253:42855(127.19.210.253:42855)] cleaning up while in state READY due to: connection closed
02:11:24.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.665776 22713 tablet_bootstrap.cc:492] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Bootstrap complete.
02:11:24.666 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.19.210.252:44531(127.19.210.252:44531)] cleaning up while in state READY due to: connection closed
02:11:24.666 [DEBUG - kudu-nio-2] (Connection.java:239) [peer master-127.19.210.254:37903(127.19.210.254:37903)] handling channelInactive
02:11:24.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.666910 22713 ts_tablet_manager.cc:1403] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent bootstrapping tablet: real 0.048s	user 0.040s	sys 0.004s
02:11:24.667 [DEBUG - kudu-nio-2] (Connection.java:709) [peer master-127.19.210.254:37903(127.19.210.254:37903)] cleaning up while in state READY due to: connection closed
02:11:24.667 [DEBUG - kudu-nio-3] (Connection.java:239) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] handling channelInactive
02:11:24.667 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
02:11:24.667 [DEBUG - kudu-nio-3] (Connection.java:709) [peer 3b4491bc31634e84a3162ba1d70813ba(127.19.210.194:43205)] cleaning up while in state READY due to: connection closed
02:11:24.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.669697 22713 raft_consensus.cc:359] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 11 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:24.670 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.670166 22713 raft_consensus.cc:740] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa0772e177e429798c32dd085f59ae6, State: Initialized, Role: FOLLOWER
02:11:24.671 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.670688 22713 consensus_queue.cc:260] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 11.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fa0772e177e429798c32dd085f59ae6" member_type: VOTER last_known_addr { host: "127.19.210.193" port: 41573 } } peers { permanent_uuid: "3b4491bc31634e84a3162ba1d70813ba" member_type: VOTER last_known_addr { host: "127.19.210.194" port: 43205 } } peers { permanent_uuid: "3aea1f3e173e4803a7ae66840502f546" member_type: VOTER last_known_addr { host: "127.19.210.195" port: 42923 } }
02:11:24.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.671999 22713 ts_tablet_manager.cc:1434] T 49ba254cf896489cb98aaa7fafc55cc1 P 8fa0772e177e429798c32dd085f59ae6: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
02:11:24.672 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.672432 22823 heartbeater.cc:499] Master 127.19.210.253:42855 was elected leader, sending a full tablet report...
02:11:24.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.674001 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 22692
02:11:24.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.680064 20477 connection.cc:573] server connection from 127.19.210.193:59091 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
02:11:24.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.680073 20329 connection.cc:573] server connection from 127.19.210.193:46979 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
02:11:24.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260708 02:11:24.680073 20402 connection.cc:573] server connection from 127.19.210.193:38631 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
02:11:24.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.680495 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 22329
02:11:24.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.686254 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 22514
02:11:24.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.691342 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 20310
02:11:24.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.698135 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 20385
02:11:24.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260708 02:11:24.702877 20299 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasknKVXmV/build/debug/bin/kudu with pid 20456
02:11:24.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-07-08T02:11:24Z chronyd exiting

Time: 33.905

OK (1 test)

